1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Public Health
  4. Volume 39, 2018
  5. Article

Annual Review of Public Health

Volume 39, 2018

Meta-Analysis of Complex Interventions

Abstract

Meta-analysis is a prominent method for estimating the effects of public health interventions, yet these interventions are often complex in ways that pose challenges to using conventional meta-analytic methods. This article discusses meta-analytic techniques that can be used in research syntheses on the effects of complex public health interventions. We first introduce the use of complexity frameworks to conceptualize public health interventions. We then present a menu of meta-analytic procedures for addressing various sources of complexity when answering questions about the effects of public health interventions in research syntheses. We conclude with a review of important practices and key resources for conducting meta-analyses on complex interventions, as well as future directions for research synthesis more generally. Overall, we argue that it is possible to conduct meaningful quantitative syntheses of research on the effects of public health interventions, though these meta-analyses may require the use of advanced techniques to properly consider and attend to issues of complexity.

Keyword(s): complex intervention, meta-analysis, research synthesis, systematic review
Loading

Article metrics loading...

/content/journals/10.1146/annurev-publhealth-040617-014112
2018-04-01
2024-04-19
Loading full text...

Full text loading...

/deliver/fulltext/publhealth/39/1/annurev-publhealth-040617-014112.html?itemId=/content/journals/10.1146/annurev-publhealth-040617-014112&mimeType=html&fmt=ahah

Literature Cited

  1. Abelson J, Wagner F, DeJean D, Boesveld S, Gauvin FP. 1.  et al. 2016. Public and patient involvement in health technology assessment: a framework for action. Int. J. Technol. Assess. Health Care 32:256–64 [Google Scholar]
  2. Anderson LM, Oliver SR, Michie S, Rehfuess E, Noyes J, Shemilt I. 2.  2013. Investigating complexity in systematic reviews of interventions by using a spectrum of methods. J. Clin. Epidemiol. 66:1223–29 [Google Scholar]
  3. Anderson LM, Petticrew M, Rehfuess E, Armstrong R, Ueffing E. 3.  et al. 2011. Using logic models to capture complexity in systematic reviews. Res. Synth. Methods 2:33–42 [Google Scholar]
  4. Barlow J, Smailagic N, Huband N, Roloff V, Bennett C. 4.  2012. Group-based parent training programmes for improving parental psychosocial health. Cochrane Database Syst. Rev.5CD002020 [Google Scholar]
  5. Beretvas SN, Pastor DA. 5.  2003. Using mixed-effects models in reliability generalization studies. Educ. Psychol. Meas. 63:75–95 [Google Scholar]
  6. Berlin JA, Santanna J, Schmid CH, Szczech LA, Feldman HI. 6.  2002. Individual patient- versus group-level data meta-regressions for the investigation of treatment effect modifiers: Ecological bias rears its ugly head. Stat. Med. 21:371–87 [Google Scholar]
  7. Blackwood B. 7.  2006. Methodological issues in evaluating complex healthcare interventions. J. Adv. Nurs. 54:612–22 [Google Scholar]
  8. Booth A, Clarke M, Dooley G, Ghersi D, Moher D. 8.  et al. 2012. The nuts and bolts of PROSPERO: an international prospective register of systematic reviews. Syst. Rev. 1:2 [Google Scholar]
  9. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. 9.  2009. Introduction to Meta-Analysis West Sussex, UK: Wiley
  10. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. 10.  2010. A basic introduction to fixed-effect and random-effects models for meta-analysis. Res. Synth. Methods 1:97–111 [Google Scholar]
  11. Borenstein M, Higgins JPT. 11.  2013. Meta-analysis and subgroups. Prev. Sci. 14:134–43 [Google Scholar]
  12. Borenstein M, Higgins JPT, Hedges LV, Rothstein HR. 12.  2017. Basics of meta-analysis: I2 is not an absolute measure of heterogeneity. Res. Synth. Methods 8:5–18 [Google Scholar]
  13. Caldwell DM. 13.  2014. An overview of conducting systematic reviews with network meta-analysis. Syst. Rev. 3:109 [Google Scholar]
  14. Caldwell DM, Welton NJ. 14.  2016. Approaches for synthesising complex mental health interventions in meta-analysis. Evid. Based Ment. Health 19:16–21 [Google Scholar]
  15. 15. Cent. Rev. Dissem. 2008. Systematic Reviews: CRD's Guidance for Undertaking Reviews in Health Care York, UK: Univ. York
  16. Chandler CIR, Kizito J, Taaka L, Nabirye C, Kayendeke M. 16.  et al. 2013. Aspirations for quality health care in Uganda: How do we get there?. Hum. Resour. Health 11:13 [Google Scholar]
  17. Cheung MW. 17.  2014. Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. Psychol. Methods 19:211–29 [Google Scholar]
  18. Cheung MWL, Chan W. 18.  2009. A two-stage approach to synthesizing covariance matrices in meta-analytic structural equation modeling. Struct. Equ. Model. 16:28–53 [Google Scholar]
  19. Cohen J, Cohen P, West SG, Aiken LS. 19.  2003. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences Mahwah, NJ: Erlbaum. , 3rd ed..
  20. Concannon TW, Fuster M, Saunders T, Patel K, Wong JB. 20.  et al. 2014. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J. Gen. Intern. Med. 29:1692–701 [Google Scholar]
  21. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM. 21.  et al. 2012. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J. Gen. Intern. Med. 27:985–91 [Google Scholar]
  22. Coon JT, Gwernan-Jones R, Moore D, Richardson M, Shotton MC. 22.  et al. 2016. End-user involvement in a systematic review of quantitative and qualitative research of non-pharmacological interventions for attention deficit hyperactivity disorder delivered in school settings: reflections on the impacts and challenges. Health Expect 19:1084–97 [Google Scholar]
  23. Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S. 23.  et al. 2014. Defining the Benefits of Stakeholder Engagement in Systematic Reviews AHRQ Publ. No. 14-EHC006-EF Rockville, MD: Agency Healthc. Res. Quality (AHRQ)
  24. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. 24.  2008. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 337:a1655 [Google Scholar]
  25. Cronce JM, Bittinger JN, Liu J, Kilmer JR. 25.  2014. Electronic feedback in college student drinking prevention and intervention. Alcohol Res. Curr. Rev. 36:47–62 [Google Scholar]
  26. Datta J, Petticrew M. 26.  2013. Challenges to evaluating complex interventions: a content analysis of published papers. BMC Public Health 13:568 [Google Scholar]
  27. Debray TPA, Moons KGM, van Valkenhoef G, Efthimiou O, Hummel N. 27.  et al. 2015. Get real in individual participant data (IPD) meta-analysis: a review of the methodology. Res. Synth. Methods 6:293–309 [Google Scholar]
  28. Dias S, Welton NJ, Caldwell DM, Ades AE. 28.  2010. Checking consistency in mixed treatment comparison meta-analysis. Stat. Med. 29:932–44 [Google Scholar]
  29. Dickersin K, Renne D. 29.  2003. Registering clinical trials. JAMA 290:516–23 [Google Scholar]
  30. Dimeff LA, Baer JS, Kivlahan DR, Marlatt GA. 30.  1999. Brief Alcohol Screening and Intervention for College Students (BASICS): A Harm Reduction Approach New York: Guilford Press
  31. Donegan S, Williamson P, D'Alessandro U, Tudur Smith C. 31.  2013. Assessing key assumptions of network meta-analysis: a review of methods. Res. Synth. Methods 4:291–323 [Google Scholar]
  32. Duncan OD. 32.  1966. Path analysis: sociological examples. Am. J. Sociol. 72:1–16 [Google Scholar]
  33. Efthimiou O, Debray TPA, van Valkenhoef G, Trelle S, Panayidou K. 33.  et al. 2016. GetReal in network meta-analysis: a review of the methodology. Res. Synth. Methods 7:236–63 [Google Scholar]
  34. Egan M, Bambra C, Petticrew M, Whitehead M. 34.  2009. Reviewing evidence on complex social interventions: appraising implementation in systematic reviews of health effects of organisational-level workplace interventions. J. Epidemiol. Community Health 63:4–11 [Google Scholar]
  35. Embry DD, Biglan A. 35.  2008. Evidence-based kernels: fundamental units of behavioral influence. Clin. Child Fam. Psychol. Rev. 11:75–113 [Google Scholar]
  36. Fischer R, Boer D. 36.  2011. What is more important for national well-being: money or autonomy? A meta-analysis of well-being, burnout, and anxiety across 63 societies. J. Pers. Soc. Psychol. 101:164–84 [Google Scholar]
  37. Fisher DJ, Copas AJ, Tierney JF, Parmar MKB. 37.  2011. A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners. J. Clin. Epidemiol. 64:949–67 [Google Scholar]
  38. Goldstein H, Yang M, Omar R, Turner R, Thompson S. 38.  2000. Meta-analysis using multilevel models with an application to the study of class size effects. Appl. Stat. 49:339–412 [Google Scholar]
  39. Goodman SN, Fanelli D, Ioannidis JPA. 39.  2016. What does research reproducibility mean?. Sci. Transl. Med. 8:341ps12 [Google Scholar]
  40. Gough D, Oliver S, Thomas J. 40.  2017. An Introduction to Systematic Reviews London: Sage
  41. Grant ES, Calderbank-Batista T. 41.  2013. Network meta-analysis for complex social interventions: problems and potential. J. Soc. Soc. Work Res. 4:406–20 [Google Scholar]
  42. Grant S. 42.  2014. Development of a CONSORT Extension for Social and Psychological Interventions Oxford, UK: University of Oxford
  43. Grant SP, Mayo-Wilson E, Melendez-Torres GJ, Montgomery P. 43.  2013. Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications. PLOS ONE 8:e65442 [Google Scholar]
  44. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. 44.  2005. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc. Sci. Med. 61:417–30 [Google Scholar]
  45. Guise JM, Chang C, Viswanathan M, Glick S, Treadwell J. 45.  et al. 2014. Agency for Healthcare Research and Quality Evidence-Based Practice Center methods for systematically reviewing complex multicomponent health care interventions. J. Clin. Epidemiol. 67:1181–91 [Google Scholar]
  46. Hagger MS, Chan DK, Protogerou C, Chatzisarantis NL. 46.  2016. Using meta-analytic path analysis to test theoretical predictions in health behavior: an illustration based on meta-analyses of the theory of planned behavior. Prev. Med. 89:154–61 [Google Scholar]
  47. Hedges LV, Vevea JL. 47.  1998. Fixed- and random-effects models in meta-analysis. Psychol. Methods 3:468–504 [Google Scholar]
  48. Higgins JPT, Whitehead A, Turner RM, Omar RZ, Thompson SG. 48.  2001. Meta-analysis of continuous outcome data from individual patients. Stat. Med. 20:2219–41 [Google Scholar]
  49. Humphreys DK, Panter J, Sahlqvist S, Goodman A, Ogilvie D. 49.  2016. Changing the environment to improve population health: a framework for considering exposure in natural experimental studies. J. Epidemiol. Community Health 70:941–46 [Google Scholar]
  50. Hutton B, Salanti G, Caldwell DM, Chaimani A, Schmid CH. 50.  et al. 2015. The PRISMA extension statement for reporting of systematic reviews incorporating network meta-analyses of health care interventions: checklist and explanations. Ann. Intern. Med. 162:777–84 [Google Scholar]
  51. Jackson D, Riley R, White IR. 51.  2011. Multivariate meta-analysis: potential and promise. Stat. Med. 30:2481–98 [Google Scholar]
  52. Jak S. 52.  2015. Meta-Analytic Structural Equation Modelling Dordrecht, Neth.: Springer
  53. Jonkman NH, Schuurmans MJ, Groenwold RH, Hoes AW, Trappenburg JC. 53.  2016. Identifying components of self-management interventions that improve health-related quality of life in chronically ill patients: systematic review and meta-regression analysis. Patient Educ. Couns. 99:1087–98 [Google Scholar]
  54. Keown K, Van Eerd D, Irvin E. 54.  2008. Stakeholder engagement opportunities in systematic reviews: knowledge transfer for policy and practice. J. Contin. Educ. Health Prof. 28:67–72 [Google Scholar]
  55. Konstantopoulous S. 55.  2011. Fixed effects and variance components in three-level meta-analysis. Res. Synth. Methods 2:61–76 [Google Scholar]
  56. Kreis J, Puhan MA, Schünemann HJ, Dickersin K. 56.  2013. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect 16:323–37 [Google Scholar]
  57. Kühne F, Ehmcke R, Härter M, Kriston L. 57.  2015. Conceptual decomposition of complex health care interventions for evidence synthesis: a literature review. J. Eval. Clin. Pract. 21:817–23 [Google Scholar]
  58. Kuyken W, Warren FC, Taylor RS, Whalley B, Crane C. 58.  et al. 2016. Efficacy of mindfulness-based cognitive therapy in prevention of depressive relapse: an individual patient data meta-analysis from randomized trials. JAMA Psychiatry 73:565–74 [Google Scholar]
  59. Larimer ME, Turner AP, Anderson BK, Fader JS, Kilmer JR. 59.  et al. 2001. Evaluating a brief alcohol intervention with fraternities. J. Stud. Alcohol 62:370–80 [Google Scholar]
  60. Lenz M, Steckelberg A, Richter B, Mühlhauser I. 60.  2007. Meta-analysis does not allow appraisal of complex interventions in diabetes and hypertension self-management: a methodological review. Diabetologia 50:1375–83 [Google Scholar]
  61. Li X, Dusseldorp E, Meulman JJ. 61.  2017. Meta-CART: a tool to identify interactions between moderators in meta-analysis. Br. J. Math. Stat. Psychol. 70:118–36 [Google Scholar]
  62. Lipsey MW, Wilson DB. 62.  2001. Practical Meta-Analysis Thousand Oaks, CA: Sage
  63. Littell JH, Corcoran J, Pillai V. 63.  2008. Systematic Reviews and Meta-Analysis New York: Oxford Univ. Press
  64. Loehlin JC. 64.  2004. Latent Variable Models: An Introduction to Factor, Path, and Structural Equation Analysis Mahwah, NJ: Erlbaum. , 4th ed..
  65. Lorenc T, Felix L, Petticrew M, Melendez-Torres GJ, Thomas J. 65.  et al. 2016. Meta-analysis, complexity, and heterogeneity: a qualitative interview study of researchers’ methodological values and practices. Syst. Rev. 5:192 [Google Scholar]
  66. Mavridis D, Giannatsi M, Cipriani A, Salanti G. 66.  2015. A primer on network meta-analysis with emphasis on mental health. Evid. Based Ment. Health 18:40–46 [Google Scholar]
  67. Mayo-Wilson E, Montgomery P. 67.  2013. Media-delivered cognitive behavioural therapy and behavioural therapy (self-help) for anxiety disorders in adults. Cochrane Database Syst. Rev.9CD005330 [Google Scholar]
  68. Melendez-Torres GJ, Bonell C, Thomas J. 68.  2015. Emergent approaches to the meta-analysis of multiple heterogeneous complex interventions. BMC Med. Res. Methodol. 15:47 [Google Scholar]
  69. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A. 69.  et al. 2015. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst. Rev. 4:1 [Google Scholar]
  70. Möhler R, Köpke S, Meyer G. 70.  2015. Criteria for Reporting the Development and Evaluation of Complex Interventions in healthcare: revised guideline (CReDECI 2). Trials 16:204 [Google Scholar]
  71. Montgomery P, Grant S, Hopewell S, Macdonald G, Moher D. 71.  et al. 2013. Protocol for CONSORT-SPI: an extension for social and psychological interventions. Implement. Sci. 8:99 [Google Scholar]
  72. Morley RF, Norman G, Golder S, Griffith P. 72.  2016. A systematic scoping review of the evidence for consumer involvement in organisations undertaking systematic reviews: focus on Cochrane. Res. Involv. Engagem. 2:36 [Google Scholar]
  73. Noyes J, Gough D, Lewin S, Mayhew A, Michie S. 73.  et al. 2013. A research and development agenda for systematic reviews that ask complex questions about complex interventions. J. Clin. Epidemiol. 66:1262–70 [Google Scholar]
  74. Noyes J, Hendry M, Booth A, Chandler J, Lewin S. 74.  et al. 2016. Current use was established and Cochrane guidance on selection of social theories for systematic reviews of complex interventions was developed. J. Clin. Epidemiol. 75:78–92 [Google Scholar]
  75. Okwaro FM, Chandler CIR, Hutchinson E, Nabirye C, Taaka L. 75.  et al. 2015. Challenging logics of complex intervention trials: community perspectives of a health care improvement intervention in rural Uganda. Soc. Sci. Med. 131:10–17 [Google Scholar]
  76. Pandor A, Gomersall T, Stevens JW, Wang J, Al-Mohammad A. 76.  et al. 2013. Remote monitoring after recent hospital discharge in patients with heart failure: a systematic review and network meta-analysis. Heart 99:1717–26 [Google Scholar]
  77. Petticrew M, Anderson L, Elder R, Grimshaw J, Hopkins D. 77.  et al. 2013. Complex interventions and their implications for systematic reviews: a pragmatic approach. J. Clin. Epidemiol. 66:1209–14 [Google Scholar]
  78. Petticrew M, Shemilt I, Lorenc T, Marteau TM, Melendez-Torres GJ. 78.  et al. 2017. Alcohol advertising and public health: systems perspectives versus narrow perspectives. J. Epidemiol. Community Health 71:308–12 [Google Scholar]
  79. Rehfuess EA, Akl EA. 79.  2013. Current experience with applying the GRADE approach to public health interventions: an empirical study. BMC Public Health 13:9 [Google Scholar]
  80. Rice K, Higgins JPT, Lumley T. 80.  2018. A re-evaluation of fixed effect(s) meta-analysis. J. R. Stat. Soc. A 181:205–27 [Google Scholar]
  81. Riley RD, Lambert PC, Abo-Zaid G. 81.  2010. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ 340:c221 [Google Scholar]
  82. Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F. 82.  et al. 2008. Meta-analysis of continuous outcomes combining individual patient data and aggregated data. Stat. Med. 27:1870–93 [Google Scholar]
  83. Riley RD, Steyerberg EW. 83.  2010. Meta-analysis of a binary outcome using individual participant data and aggregate data. Res. Synth. Methods 1:2–19 [Google Scholar]
  84. Rohwer A, Pfadenhauer L, Burns J, Brereton L, Gerhardus A. 84.  et al. 2017. Logic models help make sense of complexity in systematic reviews and health technology assessments. J. Clin. Epidemiol. 83:37–47 [Google Scholar]
  85. Rychetnik L, Hawe P, Waters E, Barratt A, Frommer M. 85.  2004. A glossary for evidence based public health. J. Epidemiol. Community Health 58:538–45 [Google Scholar]
  86. Shepperd S, Lewin S, Straus S, Clarke M, Eccles MP. 86.  et al. 2009. Can we systematically review studies that evaluate complex interventions?. PLOS Med 6:e1000086 [Google Scholar]
  87. Squires JE, Valentine JC, Grimshaw JM. 87.  2013. Systematic reviews of complex interventions: framing the review question. J. Clin. Epidemiol. 66:1215–22 [Google Scholar]
  88. Ståhlbrandt H, Johnsson KO, Berglund M. 88.  2007. Two-year outcome of alcohol interventions in Swedish university halls of residence: a cluster randomized trial of a brief skills training program, twelve-step–influenced intervention, and controls. Alcohol. Clin. Exp. Res. 3:458–66 [Google Scholar]
  89. Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M. 89.  et al. 2015. Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement. JAMA 313:1657–65 [Google Scholar]
  90. Sutton AJ, Kendrick D, Coupland CAC. 90.  2008. Meta-analysis of individual- and aggregate-level data. Stat. Med. 27:651–69 [Google Scholar]
  91. Tanner-Smith EE, Lipsey MW. 91.  2015. Brief alcohol interventions for adolescents and young adults: a systematic review and meta-analysis. J. Subst. Abuse Treat. 51:1–18 [Google Scholar]
  92. Tanner-Smith EE, Lipsey MW, Wilson DB. 92.  2016. Meta-Analysis of Research on the Effectiveness of Juvenile Drug Courts Nashville, TN: Peabody Res. Inst., Vanderbilt Univ.
  93. Tanner-Smith EE, Tipton E, Polanin JR. 93.  2016. Handling complex meta-analytic data structures using robust variance estimates: a tutorial in R. J. Dev. Life-Course Criminol. 2:85–112 [Google Scholar]
  94. Thompson SG, Higgins JPT. 94.  2002. How should meta-regression analyses be undertaken and interpreted?. Stat. Med. 21:1559–73 [Google Scholar]
  95. Thomson H, Thomas S, Sellstron E, Petticrew M. 95.  2013. Housing improvements for health and associated socio-economic outcomes. Cochrane Database Syst. Rev.2CD008657 [Google Scholar]
  96. Valentine JC, Pigott TD, Rothstein HR. 96.  2010. How many studies do you need? A primer on statistical power for meta-analysis. J. Educ. Behav. Stat. 35:215–47 [Google Scholar]
  97. Van den Noortgate W, López-López JA, Marín-Martínez F, Sánchez-Meca J. 97.  2015. Meta-analysis of multiple outcomes: a multilevel approach. Behav. Res. Methods 47:1274–94 [Google Scholar]
  98. Van den Noortgate W, Onghena P. 98.  2003. Multilevel meta-analysis: a comparison with traditional meta-analytical procedures. Educ. Psychol. Meas. 63:765–90 [Google Scholar]
  99. Waters E, Petticrew M, Priest N, Weightman A, Harden A, Doyle J. 99.  2008. Evidence synthesis, upstream determinants and health inequalities: the role of a proposed new Cochrane Public Health Review Group. Eur. J. Public Health 18:221–23 [Google Scholar]
  100. Welch VA, Ghogomu E, Hossain A, Awasthi S, Bhutta Z. 100.  et al. 2016. Deworming and adjuvant interventions for improving the developmental health and well-being of children in low-and middle-income countries: a systematic review and network meta-analysis. Campbell Syst. Rev. 2016:7 [Google Scholar]
  101. Wilson DB, Tanner-Smith EE, Mavridis D. 101.  2016. Network Meta-Analysis Campbell Methods Ser. Policy Note 1 (Version 1.0) Oslo: Campbell Collab.
  102. Wilson SJ, Tanner-Smith EE. 102.  2014. Meta-analysis in prevention science. Defining Prevention Science, Advances in Prevention Science Z Sloboda, H Petras 431–52 New York: Springer [Google Scholar]
/content/journals/10.1146/annurev-publhealth-040617-014112
Loading
Meta-Analysis of Complex Interventions
Annual Review of Public Health 39, 135 (2018); https://doi.org/10.1146/annurev-publhealth-040617-014112
/content/journals/10.1146/annurev-publhealth-040617-014112
/content/journals/10.1146/annurev-publhealth-040617-014112
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error