1932

Abstract

This article provides an overview of mixed methods research and mixed studies reviews. These two approaches are used to combine the strengths of quantitative and qualitative methods and to compensate for their respective limitations. This article is structured in three main parts. First, the epistemological background for mixed methods will be presented. Afterward, we present the main types of mixed methods research designs and techniques as well as guidance for planning, conducting, and appraising mixed methods research. In the last part, we describe the main types of mixed studies reviews and provide a tool kit and examples. Future research needs to offer guidance for assessing mixed methods research and reporting mixed studies reviews, among other challenges.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-publhealth-032013-182440
2014-03-18
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/publhealth/35/1/annurev-publhealth-032013-182440.html?itemId=/content/journals/10.1146/annurev-publhealth-032013-182440&mimeType=html&fmt=ahah

Literature Cited

  1. Abbott A. 1.  1988. The System of Professions Chicago: Univ. Chicago Press
  2. Abbott A. 2.  1998. The causal devolution. Sociol. Methods Res. 27:148–81 [Google Scholar]
  3. Armstrong R, Waters E, Doyle J. 3.  2008. Reviews in public health and health promotion. Cochrane Handbook for Systematic Reviews of Interventions JPT Higgins, S Green 593–606 Chichester, UK: Wiley- Blackwell [Google Scholar]
  4. Bélanger E, Rodríguez C, Groleau D. 4.  2011. Shared decision-making in palliative care: a systematic mixed studies review using narrative synthesis. Palliat. Med. 25:242–61 [Google Scholar]
  5. Bindiganavile Sridhar S. 5.  2011. In pursuit of a valid information assessment method Master's diss. McGill Univ., Montreal 92
  6. Boaz A, Ashby D, Denyer D, Egan M, Harden A. 6.  et al. 2006. A multitude of syntheses: a comparison of five approaches from diverse policy fields. Evid. Policy 2:479–502 [Google Scholar]
  7. Boyatzis RE. 7.  1998. Transforming Qualitative Information: Thematic Analysis and Code Development Thousand Oaks, CA: Sage
  8. Boyko JA, Lavis JN, Abelson J, Dobbins M, Carter N. 8.  2012. Deliberative dialogues as a mechanism for knowledge translation and exchange in health systems decision-making. Soc. Sci. Med. 75:1938–45 [Google Scholar]
  9. Bryman A. 9.  2006. Integrating quantitative and qualitative research: How is it done?. Qual. Res. 6:97–113 [Google Scholar]
  10. Buelens M, Woestyne M, Mestdagh S, Bouckenooghe D. 10.  2008. Methodological issues in negotiation research: a state-of-the-art-review. Group Decis. Negot. 17:321–45 [Google Scholar]
  11. Campbell DT. 11.  1988. Qualitative knowing in action research. Methodology and Epistemology for Social Science: Selected Papers of Donald T. Campbell S Overman 360–76 Chicago: Univ. Chicago Press [Google Scholar]
  12. Campbell F, Johnson M, Messina J, Guillaume L, Goyder E. 12.  2011. Behavioural interventions for weight management in pregnancy: a systematic review of quantitative and qualitative data. BMC Public Health 11:491 [Google Scholar]
  13. Carr SM, Lhussier M, Forster N, Geddes L, Deane K. 13.  et al. 2011. An evidence synthesis of qualitative and quantitative research on component intervention techniques, effectiveness, cost-effectiveness, equity and acceptability of different versions of health-related lifestyle advisor role in improving health. Health Technol. Assess. 15:iii–iv [Google Scholar]
  14. Chang Y, Voils CI, Sandelowski M, Hasselblad V, Crandell JL. 14.  2009. Transforming verbal counts in reports of qualitative descriptive studies into numbers. West. J. Nurs. Res. 31:837–52 [Google Scholar]
  15. Classen S, Lopez EDS. 15.  2006. Mixed methods approach explaining process of an older driver safety systematic literature review. Top. Geriatr. Rehabil. 22:99–112 [Google Scholar]
  16. Coles E, Themessl-Huber M, Freeman R. 16.  2012. Investigating community-based health and health promotion for homeless people: a mixed methods review. Health Educ. Res. 27:624–44 [Google Scholar]
  17. Contandriopoulos D, Lemire M, Denis JL, Tremblay E. 17.  2010. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Q. 88:444–83 [Google Scholar]
  18. Creswell JW, Plano Clark VL. 18.  2010. Designing and Conducting Mixed Methods Research Thousand Oaks, CA: Sage, 2nd ed..
  19. Creswell JW, Klassen AC, Plano Clark VL, Clegg Smith K. 19.  2011. Best Practices for Mixed Methods Research in the Health Sciences Bethesda, MD: Off. Behav. Soc. Sci. Res., Natl. Inst. Health
  20. Dagenais C, Nault-Brière F, Dupont D, Dutil J. 20.  2008. Implementation and effects of a service coordination program for youths and their families in complex situations: a mixed evaluation design. Child Youth Serv. Rev. 30:903–13 [Google Scholar]
  21. Denzin NK, Lincoln YS. 21.  1994. Handbook of Qualitative Research Thousand Oaks, CA: Sage
  22. Dixon-Woods M, Agarwal S, Jones D, Young B, Sutton A. 22.  2005. Synthesising qualitative and quantitative evidence: a review of possible methods. J. Health Serv. Res. Policy 10:45–53 [Google Scholar]
  23. Dixon-Woods M, Cavers D, Agarwal S, Annandale E, Arthur A. 23.  et al. 2006. Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med. Res. Methodol. 6:35 [Google Scholar]
  24. Fries CJ. 24.  2009. Bourdieu's reflexive sociology as a theoretical basis for mixed methods research: an application to complementary and alternative medicine. J. Mix. Methods Res. 3:326–48 [Google Scholar]
  25. Grant MJ, Booth A. 25.  2009. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info. Libr. J. 26:91–108 [Google Scholar]
  26. Greene JC. 26.  2006. Toward a methodology of mixed methods social inquiry. Res. Sch. 13:93–98 [Google Scholar]
  27. Greene JC. 27.  2008. Is mixed methods social inquiry a distinctive methodology?. J. Mix. Methods Res. 2:7–22 [Google Scholar]
  28. Greenhalgh T, Potts HW, Wong G, Bark P, Swinglehurst D. 28.  2009. Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q. 87:729–88 [Google Scholar]
  29. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. 29.  2004. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 82:581–629 [Google Scholar]
  30. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O, Peacock R. 30.  2005. Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review. Soc. Sci. Med. 61:417–30 [Google Scholar]
  31. Groleau D, Pluye P, Nadeau L. 31.  2007. A mix-method approach to the cultural understanding of distress and the non-use of mental health services. J. Ment. Health 16:731–41 [Google Scholar]
  32. Hacking I. 32.  1999. The Social Construction of What? Cambridge, MA: Harvard Univ. Press
  33. Hall B, Howard K. 33.  2008. A synergistic approach: conducting mixed methods research with typological and systemic design considerations. J. Mix. Methods Res. 2:248–69 [Google Scholar]
  34. Harden A, Brunton G, Fletcher A, Oakley A. 34.  2009. Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies. BMJ 339:b4254 [Google Scholar]
  35. Heyvaert M, Maes B, Onghena P. 35.  2013. Mixed methods research synthesis: definition, framework, and potential. Qual. Quant. 47:659–76 [Google Scholar]
  36. Huberman AM, Miles MB. 36.  1991. Analyse des données qualitatives: recueil de nouvelles méthodes Bruxelles: De Boeck Univ.
  37. Jagosh J, Macaulay A, Pluye P, Salsberg J, Bush P. 37.  et al. 2012. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 90:311–46 [Google Scholar]
  38. Jagosh J, Pluye P, Macaulay A, Salsberg J, Henderson J. 38.  et al. 2011. Assessing the outcomes of participatory research: protocol for identifying, selecting, appraising and synthesizing the literature for realist review. Implement. Sci. 6:24 [Google Scholar]
  39. Jagosh J, Pluye P, Wong G, Cargo M, Salsberg J. 39.  et al. 2014. Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment. Res. Synthesis Methods. In press
  40. Johnson RB. 40.  2008. Editorial: Living with tensions: the dialectic approach. J. Mix. Methods Res. 2:203–7 [Google Scholar]
  41. Johnson RB, Onwuegbuzie AJ, Turner LA. 41.  2007. Toward a definition of mixed methods research. J. Mix. Methods Res. 1:112–33 [Google Scholar]
  42. Kennedy F, McDonnell A, Gerrish K, Howarth A, Pollard C, Redman J. 42.  2012. Evaluation of the impact of nurse consultant roles in the United Kingdom: a mixed methods systematic literature review. J. Adv. Nurs. 68:721–42 [Google Scholar]
  43. Kompier MAJ, Cooper CL, Geurts SAE. 43.  2000. A multiple case study approach to work stress prevention in Europe. Eur. J. Work Organ. Psychol. 9:371–400 [Google Scholar]
  44. Kuhn T. 44.  1983. La structure des révolutions scientifiques Paris: Flammarion
  45. Lakshman R, Ogilvie D, Ong KK. 45.  2009. Mothers' experiences of bottle feeding: a systematic review of qualitative and quantitative studies. Arch. Dis. Child. 94:596–601 [Google Scholar]
  46. Lewis SA, Noyes J, Mackereth S. 46.  2010. Knowledge and information needs of young people with epilepsy and their parents: mixed-method systematic review. BMC Pediatr. 10:103 [Google Scholar]
  47. Mays N, Pope C, Popay J. 47.  2005. Systematically reviewing qualitative and quantitative evidence to inform management and policy-making in the health field. J. Health Serv. Res. Policy 10:Suppl. 16–20 [Google Scholar]
  48. McEvoy P, Richards D. 48.  2006. A critical realist rationale for using a combination of quantitative and qualitative methods. J. Res. Nurs. 11:66–78 [Google Scholar]
  49. Mills EJ, Seely D, Rachlis B, Griffith L, Wu P. 49.  et al. 2006. Barriers to participation in clinical trials of cancer: a meta-analysis and systematic review of patient-reported factors. Lancet Oncol. 7:141–48 [Google Scholar]
  50. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. 50.  2009. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Ann. Intern. Med. 151:264–69, W64 [Google Scholar]
  51. Morgan DL. 51.  2007. Paradigms lost and pragmatism regained: methodological implications of combining qualitative and quantitative methods. J. Mix. Methods Res. 1:48–76 [Google Scholar]
  52. Neuendorf KA. 52.  2002. The Content Analysis Guidebook Thousand Oaks, CA: Sage
  53. Newman TB. 53.  2003. The power of stories over statistics. BMJ 327:1424–27 [Google Scholar]
  54. Niglas K. 54.  2010. The multidimensional model of research methodology: an integrated set of continua. See Ref. 83 215–36
  55. Noblit GW, Hare RD. 55.  1988. Meta-Ethnography: Synthesising Qualitative Studies Newbury Park, CA: Sage
  56. O'Cathain A. 56.  2010. Assessing the quality of mixed methods research: towards a comprehensive framework. See Ref. 83 531–55
  57. O'Cathain A, Murphy E, Nicholl J. 57.  2010. Three techniques for integrating data in mixed methods studies. BMJ 341:1147–50 [Google Scholar]
  58. Oliver S, Harden A, Rees R, Shepherd J, Brunton G. 58.  et al. 2005. An emerging framework for including different types of evidence in systematic reviews for public policy. Evaluation 11:428–46 [Google Scholar]
  59. Pace R, Pluye P, Bartlett G, Macaulay AC, Salsberg J. 59.  et al. 2012. Testing the reliability and efficiency of the pilot Mixed Methods Appraisal Tool (MMAT) for systematic mixed studies review. Int. J. Nurs. Stud. 49:47–53 [Google Scholar]
  60. Patton MQ. 60.  2002. Qualitative Research and Evaluation Methods Thousand Oaks, CA: Sage
  61. Pawson R, Greenhalgh T, Harvey G, Walshe K. 61.  2005. Realist review—a new method of systematic review designed for complex policy interventions. J. Health Serv. Res. Policy 10:Suppl. 121–34 [Google Scholar]
  62. Pluye P. 62.  2013. Critical appraisal tools for assessing the methodological quality of qualitative, quantitative and mixed methods studies included in systematic mixed studies reviews. J. Eval. Clin. Pract. 19:4722 [Google Scholar]
  63. Pluye P. 63.  2012. Les méthodes mixtes. Approches et pratiques en évaluation de programme V Ridde, C Dagenais 125–43 Montréal: Press. Univ. Montréal [Google Scholar]
  64. Pluye P. 64.  2012. Méthodes mixtes en santé mondiale. Regards croisés sur la socio-anthropologie réflexive et la recherche en santé mondiale JC Suárez-Herrera, MJ Blain 129–35 Montréal: Cahiers sci. ACFAS [Google Scholar]
  65. Pluye P, Gagnon MP, Griffiths F, Johnson-Lafleur J. 65.  2009. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int. J. Nurs. Stud. 46:529–46 [Google Scholar]
  66. Pluye P, Grad R, Repchinsky C, Jovaisas B, Johnson-Lafleur J. 66.  et al. 2013. Four levels of outcomes of information-seeking: a mixed methods study in primary health care. J. Am. Soc. Inf. Sci. Technol. 64:108–25 [Google Scholar]
  67. Pluye P, Grad RM, Dunikowski LG, Stephenson R. 67.  2005. Impact of clinical information-retrieval technology on physicians: a literature review of quantitative, qualitative and mixed methods studies. Int. J. Med. Inform. 74:745–68 [Google Scholar]
  68. Pluye P, Grad RM, Levine A, Nicolau B. 68.  2009. Understanding divergence of quantitative and qualitative data (or results) in mixed methods studies. Int. J. Mult. Res. Approaches 3:58–72 [Google Scholar]
  69. Pluye P, Robert E, Cargo M, Bartlett G, O'Cathain A. 69.  et al. 2011. Proposal: A mixed methods appraisal tool for systematic mixed studies reviews McGill Univ., Montreal. http://mixedmethodsappraisaltoolpublic. pbworks.com/f/MMAT%202011%20criteria%20and%20tutorial%202011-06-29.pdf. Arch. by WebCite® at http://www.webcitation.org/5tTRTc9yJ
  70. Pope C, Mays N, Popay J. 70.  2007. Synthesizing Qualitative and Quantitative Health Evidence: A Guide to Methods Adelaide: Ramsay
  71. Porta M. 71.  2008. A Dictionary of Epidemiology New York: Oxford Univ. Press, 5th ed..
  72. Rihoux B, Marx A. 72.  2013. QCA, 25 years after “The Comparative Method”: mapping, challenges, and innovations—mini-symposium. Polit. Res. Q. 66:167–235 [Google Scholar]
  73. Rihoux B, Ragin CC. 73.  2009. Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques Thousand Oaks, CA: Sage [Google Scholar]
  74. Rivard S, Lapointe L. 74.  2012. Information technology implementers' responses to user resistance: nature and effects. MIS Q. 36:897–920 [Google Scholar]
  75. Roberts KA, Dixon-Woods M, Fitzpatrick R, Abrams KR, Jones DR. 75.  2002. Factors affecting uptake of childhood immunisation: a Bayesian synthesis of qualitative and quantitative evidence. Lancet 360:1596–99 [Google Scholar]
  76. Rosewilliam S, Roskell CA, Pandyan AD. 76.  2011. A systematic review and synthesis of the quantitative and qualitative evidence behind patient-centred goal setting in stroke rehabilitation. Clin. Rehabil. 25:501–14 [Google Scholar]
  77. Saini M, Shlonsky A. 77.  2012. Overview of mixed-method systematic review designs. Systematic Synthesis of Qualitative Research M Saini, A Shlonsky 1–14 New York: Oxford Univ. Press [Google Scholar]
  78. Sandelowski M, Barroso J, Voils CI. 78.  2007. Using qualitative metasummary to synthesize qualitative and quantitative descriptive findings. Res. Nurs. Health 30:99–111 [Google Scholar]
  79. Sayer A. 79.  2000. Realism and Social Science London: Sage
  80. Schwandt TA. 80.  2007. The SAGE Dictionary of Qualitative Inquiry Thousand Oaks, CA: Sage, 3rd ed..
  81. Scott PJ, Briggs JS. 81.  2009. A pragmatist argument for mixed methodology in medical informatics. J. Mix. Methods Res. 3:223–41 [Google Scholar]
  82. Sheldon TA. 82.  2005. Making evidence synthesis more useful for management and policy-making. J. Health Serv. Res. Policy 10:Suppl. 11–5 [Google Scholar]
  83. Tashakkori A, Teddlie C. 83.  2010. Handbook of Mixed Methods in Social and Behavioral Research Thousand Oaks, CA: Sage
  84. Thomas J, Harden A, Oakley A, Oliver S, Sutcliffe K. 84.  et al. 2004. Integrating qualitative research with trials in systematic reviews. BMJ 328:1010–12 [Google Scholar]
  85. Vogt DS, King DW, King LA. 85.  2004. Focus groups in psychological assessment: enhancing content validity by consulting members of the target population. Psychol. Assess. 16:231–43 [Google Scholar]
  86. Voils C, Hasselblad V, Crandell J, Chang Y, Lee E, Sandelowski M. 86.  2009. A Bayesian method for the synthesis of evidence from qualitative and quantitative reports: the example of antiretroviral medication adherence. J. Health Serv. Res. Policy 14:226–33 [Google Scholar]
  87. Wang HF, Yeh MC. 87.  2012. Psychological resistance to insulin therapy in adults with type 2 diabetes: mixed-method systematic review. J. Adv. Nurs. 68:743–57 [Google Scholar]
  88. Weber M. 88.  1995. Économie et société: les catégories de la sociologie Paris: Plon
  89. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. 89.  2013. RAMESES publication standards: meta-narrative reviews. BMC Med. 11:20 [Google Scholar]
  90. Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. 90.  2013. RAMESES publication standards: realist syntheses. BMC Med. 11:21 [Google Scholar]
  91. Yin RK, Bingham E, Heald KA. 91.  1976. The difference that quality makes: the case of literature reviews. Sociol. Method. Res. 5:139–56 [Google Scholar]
/content/journals/10.1146/annurev-publhealth-032013-182440
Loading
/content/journals/10.1146/annurev-publhealth-032013-182440
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error