Display options
Cite
Zwetsloot PP, Van Der Naald M, Sena ES, et al. Standardized mean differences cause funnel plot distortion in publication bias assessments. Elife. 2017;6doi: 10.7554/eLife.24260.
Zwetsloot, P. P., Van Der Naald, M., Sena, E. S., Howells, D. W., IntHout, J., De Groot, J. A., Chamuleau, S. A., MacLeod, M. R., & Wever, K. E. (2017). Standardized mean differences cause funnel plot distortion in publication bias assessments. eLife, 6. https://doi.org/10.7554/eLife.24260
Zwetsloot, Peter-Paul, et al. "Standardized mean differences cause funnel plot distortion in publication bias assessments." eLife vol. 6 (2017). doi: https://doi.org/10.7554/eLife.24260
Zwetsloot PP, Van Der Naald M, Sena ES, Howells DW, IntHout J, De Groot JA, Chamuleau SA, MacLeod MR, Wever KE. Standardized mean differences cause funnel plot distortion in publication bias assessments. Elife. 2017 Sep 08;6. doi: 10.7554/eLife.24260. PMID: 28884685; PMCID: PMC5621838.
Copy Download .nbib Format: NLM
Share it on

Elife. 2017 Sep 08;6. doi: 10.7554/eLife.24260.

Standardized mean differences cause funnel plot distortion in publication bias assessments.

eLife

Peter-Paul Zwetsloot, Mira Van Der Naald, Emily S Sena, David W Howells, Joanna IntHout, Joris Ah De Groot, Steven Aj Chamuleau, Malcolm R MacLeod, Kimberley E Wever

Affiliations

  1. Cardiology, Experimental Cardiology Laboratory, University Medical Center Utrecht, Utrecht, Netherlands.
  2. Netherlands Heart Institute, Utrecht, Netherlands.
  3. Center for Clinical Brain Sciences, University of Edinburgh, Edinburgh, United Kingdom.
  4. School of Medicine, University of Tasmania, Hobart, Australia.
  5. Department for Health Evidence, Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, Netherlands.
  6. Julius Center for Health Sciences and Primary care, University Medical Center Utrecht, Utrecht, Netherlands.
  7. Regenerative Medicine Center, University Medical Center Utrecht, Utrecht, Netherlands.
  8. Systematic Review Centre for Laboratory Animal Experimentation (SYRCLE), Radboud Institute for Health Sciences, Radboud University Medical Center, Nijmegen, Netherlands.

PMID: 28884685 PMCID: PMC5621838 DOI: 10.7554/eLife.24260

Abstract

Meta-analyses are increasingly used for synthesis of evidence from biomedical research, and often include an assessment of publication bias based on visual or analytical detection of asymmetry in funnel plots. We studied the influence of different normalisation approaches, sample size and intervention effects on funnel plot asymmetry, using empirical datasets and illustrative simulations. We found that funnel plots of the Standardized Mean Difference (SMD) plotted against the standard error (SE) are susceptible to distortion, leading to overestimation of the existence and extent of publication bias. Distortion was more severe when the primary studies had a small sample size and when an intervention effect was present. We show that using the Normalised Mean Difference measure as effect size (when possible), or plotting the SMD against a sample size-based precision estimate, are more reliable alternatives. We conclude that funnel plots using the SMD in combination with the SE are unsuitable for publication bias assessments and can lead to false-positive results.

Keywords: data simulation; epidemiology; funnel plot; global health; human biology; medicine; meta-analysis; none; publication bias

References

  1. Lancet. 2014 Jan 11;383(9912):156-65 - PubMed
  2. Sci Rep. 2015 Aug 27;5:13428 - PubMed
  3. PLoS One. 2015 Nov 18;10(11):e0142021 - PubMed
  4. BMJ. 2006 Sep 16;333(7568):597-600 - PubMed
  5. BMJ. 2011 Jul 22;343:d4002 - PubMed
  6. PLoS One. 2012;7(2):e32296 - PubMed
  7. Evid Based Preclin Med. 2016 Aug;3(1):e00015 - PubMed
  8. Biometrics. 1994 Dec;50(4):1088-101 - PubMed
  9. J Neurosci Methods. 2014 Jan 15;221:92-102 - PubMed
  10. Biometrics. 2000 Jun;56(2):455-63 - PubMed
  11. PLoS Biol. 2015 Oct 02;13(10):e1002264 - PubMed
  12. PLoS One. 2013 Jul 05;8(7):e66844 - PubMed
  13. Clin Trials. 2007;4(3):245-53 - PubMed
  14. Eur J Pharmacol. 2015 Apr 15;753:191-208 - PubMed
  15. JAMA. 2006 Feb 8;295(6):676-80 - PubMed
  16. Tissue Eng Part B Rev. 2015 Oct;21(5):411-26 - PubMed
  17. BMJ. 1997 Sep 13;315(7109):629-34 - PubMed
  18. PLoS Med. 2010 Sep 21;7(9):e1000326 - PubMed
  19. Circ Res. 2016 Apr 15;118(8):1223-32 - PubMed
  20. PLoS Biol. 2010 Mar 30;8(3):e1000344 - PubMed
  21. Lab Anim. 2011 Oct;45(4):225-30 - PubMed

MeSH terms

Publication Types

Grant support