Innovations in Mixed Methods Evaluations

Literature Cited

1. 1. 

   Aarons GA, Fettes DL, Sommerfeld DH, Palinkas LA 2012. Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services. Child Maltreat 17:67–79
   [Google Scholar]
2. 2. 

   Aarons GA, Wells R, Zagursky K, Fettes DL, Palinkas LA 2009. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am. J. Public Health 99:2087–95
   [Google Scholar]
3. 3. 

   Ackerman SL, Sarkar U, Tieu L, Handley MA, Schillinger D et al. 2017. Meaningful use in the safety net: a rapid ethnography of patient portal implementation at five community health centers in California. J. Am. Med. Inform. Assoc. 24:5903–12
   [Google Scholar]
4. 4. 

   Albright K, Gechter K, Kempe A 2013. Importance of mixed methods in pragmatic trials and dissemination and implementation research. Acad. Pediatr. 13:400–7
   [Google Scholar]
5. 5. 

   Beebe J 1995. Basic concepts and techniques of rapid appraisal. Hum. Org. 54:42–51
   [Google Scholar]
6. 6. 

   Bogart LM, Fu CM, Eyraud J, Cowgill BO, Hawes-Dawson J et al. 2018. Evaluation of the dissemination of SNaX, a middle school-based obesity prevention intervention, within a large US school district. Transl. Behav. Med. 8:724–32
   [Google Scholar]
7. 7. 

   Broder-Fingert S, Walls M, Augustyn M, Beidas R, Mandell D et al. 2018. A hybrid type I randomized effectiveness-implementation trial of patient navigation to improve access to services for children with autism spectrum disorder. BMC Psychiatry 18:79
   [Google Scholar]
8. 8. 

   Brown CH, Curran G, Palinkas LA, Aarons GA, Wells KB et al. 2017. An overview of research and evaluation designs for dissemination and implementation. Annu. Rev. Public Health 38:1–22
   [Google Scholar]
9. 9. 

   Chamberlain P, Brown C, Saldana L 2011. Observational measure of implementation progress in community-based settings: the stages of implementation completion (SIC). Implement. Sci. 6:116
   [Google Scholar]
10. 10. 

    Chamberlain P, Mihalic SF 1998. Blueprints for Violence Prevention, Book Eight: Multidimensional Treatment Foster Care Boulder, CO: Cent. Study Prev. Viol.
11. 11. 

    Choy I, Kitto S, Adu-Aryee N, Okrainec A 2013. Barriers to uptake of laparoscopic surgery in a lower-middle-income country. Surg. Endosc. 27:4009–15
    [Google Scholar]
12. 12. 

    Cohen JA, Mannarino AP, Deblinger E 2006. Treating Trauma and Traumatic Grief in Children and Adolescents New York: Guilford Press
13. 13. 

    Cook PA, Hargreaves SC, Burns EJ, de Vocht F, Parrott S et al. 2018. Communities in charge of alcohol (CICA): a protocol for a stepped-wedge randomized control trial of an alcohol health champions programme. BMC Public Health 18:522
    [Google Scholar]
14. 14. 

    Corbin J, Strauss A 2008. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory Thousand Oaks, CA: Sage
15. 15. 

    Cragun D, Pal T, Vadaparampil ST, Baldwin J, Hampel H, DeBate RD 2016. Qualitative comparative analysis: a hybrid method for identifying factors associated with program effectiveness. J. Mix. Methods Res. 10:3251–72
    [Google Scholar]
16. 16. 

    Creswell JW, Klassen AC, Plano Clark VL, Clegg Smith K 2011. Best practices for mixed methods research in the health sciences Rep., Off. Behav. Soc. Sci. Res. Natl. Inst. Health Bethesda, MD: https://obssr.od.nih.gov/wp-content/uploads/2016/02/Best_Practices_for_Mixed_Methods_Research.pdf
17. 17. 

    Creswell JW, Plano Clark VL 2011. Designing and Conducting Mixed Methods Research Thousand Oaks, CA: Sage, 2nd ed..
18. 18. 

    Creswell JW, Tashakkori A 2007. Developing publishable mixed methods manuscripts. J. Mix. Methods Res. 1:107–11
    [Google Scholar]
19. 19. 

    Cully JA, Armento ME, Mott J, Nadorff MR, Naik AD et al. 2012. Brief cognitive behavioral therapy in primary care: a hybrid type 2 patient-randomized effectiveness-implementation trial. Implement. Sci. 7:64
    [Google Scholar]
20. 20. 

    Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C 2012. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50:3217–26
    [Google Scholar]
21. 21. 

    Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC 2009. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 4:50
    [Google Scholar]
22. 22. 

    Damschroder LJ, Lowery JC 2013. Evaluation of a large-scale weight management program using the consolidated framework for implementation research (CFIR). Implement. Sci. 8:51
    [Google Scholar]
23. 23. 

    Damschroder LJ, Moin T, Datta SK, Reardon CM, Steinle N et al. 2015. Implementation and evaluation of the VA DPP clinical demonstration: protocol for a multi-site non-randomized hybrid effectiveness-implementation type III trial. Implement. Sci. 10:68
    [Google Scholar]
24. 24. 

    Damschroder LJ, Reardon CM, Sperber N, Robinson CH, Fickel JJ, Oddone EZ 2017. Implementation evaluation of the Telephone Lifestyle Coaching (TLC) program: organizational factors associated with successful implementation. Transl. Behav. Med. 7:233–41
    [Google Scholar]
25. 25. 

    Dannifer R, Abrami A, Rapoport R, Sriphanlop P, Sacks R, Johns M 2015. A mixed-methods evaluation of a SNAP-Ed farmers’ market-based nutrition education program. J. Nutr. Educ. Behav. 47:516–25
    [Google Scholar]
26. 26. 

    Delbecq AL, Van de Ven AH, Gustafson DH 1975. Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes Glenview, IL: Foresman
27. 27. 

    Duan N, Bhaumik DK, Palinkas LA, Hoagwood K 2015. Optimal design and purposeful sampling: complementary methodologies for implementation research. Admin. Policy Ment. Health Ment. Health Serv. Res. 42:524–32
    [Google Scholar]
28. 28. 

    Dy SM, Garg P, Nyberg D, Dawson PB, Pronovost PJ et al. 2005. Critical pathway effectiveness: assessing the impact of patient, hospital care, and pathway characteristics using qualitative comparative analysis. Health Serv. Res. 40:2499–516
    [Google Scholar]
29. 29. 

    Elinder LS, Patterson E, Nyberg G, Norman A 2018. A Healthy Start Plus for prevention of childhood overweight and obesity in disadvantaged areas through parental support in the school setting—study protocol for a parallel group cluster randomized trial. BMC Public Health 18:459
    [Google Scholar]
30. 30. 

    Fetters MD, Molina-Azorin JF 2018. Description. Journal of Mixed Methods Research https://au.sagepub.com/en-gb/oce/journal-of-mixed-methods-research/journal201775#description
31. 31. 

    Ford EW, Duncan WJ, Ginter PM 2005. Health departments’ implementation of public health's core functions: an assessment of health impacts. Public Health 119:11–21
    [Google Scholar]
32. 32. 

    Forman J, Heisley M, Damschroder LJ, Kaselitz E, Kerr EA 2017. Development and application of the RE-AIM QuEST mixed methods framework for program evaluation. Prev. Med. Rep. 6:322–28
    [Google Scholar]
33. 33. 

    Fox AB, Hamilton AB, Frayne SN, Wiltsey-Stirman S, Bean-Mayberry B et al. 2016. Effectiveness of an evidence-based quality improvement approach to cultural competence training: the Veterans Affairs’ “Caring for Women Veterans” program. J. Contin. Educ. Health Prof. 36:96–103
    [Google Scholar]
34. 34. 

    Gale NK, Heath G, Cameron E, Rashid S, Redwood S 2013. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med. Res. Methodol. 13:117
    [Google Scholar]
35. 35. 

    Glaser BG, Strauss AL 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research New York: Aldine de Gruyter
36. 36. 

    Glasgow RE, Vogt SM, Bowles TM 1999. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am. J. Public Health 89:91322–27
    [Google Scholar]
37. 37. 

    Goepp JG, Meykler S, Mooney NE, Lyon C, Raso R, Julliard K 2008. Provider insights about palliative care barriers and facilitators: results of a rapid ethnographic assessment. Am. J. Hosp. Palliat. Care 25:309–14
    [Google Scholar]
38. 38. 

    Granholm E, Holden JL, Sommerfeld D, Rufener C, Perivoliotis D et al. 2015. Enhancing assertive community treatment with cognitive behavioral social skills training for schizophrenia: study protocol for a randomized controlled trial. Trials 16:438
    [Google Scholar]
39. 39. 

    Greene JC, Caracelli VJ, Graham WF 1989. Toward a conceptual framework for mixed-method evaluation designs. Educ. Eval. Policy Anal. 11:3255–74
    [Google Scholar]
40. 40. 

    Hamilton AB 2013. Qualitative methods in rapid turn-around health services research Seminar transcript, Spotlight on Women's Health, VA Health Serv. Res. Dev., Dec. 11. https://www.hsrd.research.va.gov/for_researchers/cyber_seminars/archives/video_archive.cfm?SessionID=780
41. 41. 

    Hamilton AB, Cohen AN, Glover DL, Whelan F, Chemerinski E et al. 2013. Implementation of evidence-based employment services in specialty mental health. Health Serv. Res. 48:2224–44
    [Google Scholar]
42. 42. 

    Hanson RF, Schoenwald S, Saunders BE, Chapman J, Palinkas LA et al. 2016. Testing the Community-Based Learning Collaborative (CBLC) implementation model: a study protocol. Int. J. Ment. Health Syst. 10:52
    [Google Scholar]
43. 43. 

    Henggler SW, Schoenwald SK, Borduin CM, Rowland MD, Cunningham PB 2009. Multisystemic Therapy for Antisocial Behavior in Children and Adolescents New York: Guilford Press, 2nd ed..
44. 44. 

    Hsieh HF, Shannon SE 2005. Three approaches to qualitative content analysis. Qual. Health Res. 15:1277–88
    [Google Scholar]
45. 45. 

    Jayawardena A, Wijayasinghe SR, Tennakoon D, Cook T, Morcuendo JA 2013. Early effects of a ‘train the trainer’ approach on Ponseti method dissemination: a case study of Sri Lanka. Iowa Orthop. J. 33:153–60
    [Google Scholar]
46. 46. 

    Kahwati LC, Lewis MA, Kane H, Williams PA, Nerz P et al. 2011. Best practices in the Veterans Health Administration's MOVE! weight management program. Am. J. Prev. Med. 41:457–64
    [Google Scholar]
47. 47. 

    Kane H, Hinnant L, Day K, Council M, Tzeng J et al. 2017. Pathways to program success: a qualitative comparative analysis (QCA) of Communities Putting Prevention to Work case study programs. J. Public Health. Manag. Pract. 23:104–11
    [Google Scholar]
48. 48. 

    Kane H, Lewis MA, Williams PA, Kahwati LC 2014. Using qualitative comparative analysis to understand and quantify translation and implementation. Transl. Behav. Med. 4:201–8
    [Google Scholar]
49. 49. 

    Koenig CJ, Abraham T, Zamora KA, Hill C, Kelly PA et al. 2016. Pre-implementation strategies to adapt and implement a veteran peer coaching intervention to improve mental health treatment engagement among rural veterans. J. Rural Health 32:4418–28
    [Google Scholar]
50. 50. 

    Kozica SL, Lombard CB, Harrison CL, Teede HJ 2016. Evaluation of a large healthy lifestyle program: informing program implementation and scale-up in the prevention of obesity. Implement. Sci. 11:151
    [Google Scholar]
51. 51. 

    Lewis CC, Scott K, Marty CN, Marriott BR, Kroenke K et al. 2015. Implementing measurement-based care (iMBC) for depression in community mental health: a dynamic cluster randomized trial study protocol. Implement. Sci. 10:127
    [Google Scholar]
52. 52. 

    Liddle HA 2002. Multidimensional family therapy treatment for adolescent cannabis users: Rep., Cent. Subst. Abuse Treat. Subst. Abuse Ment. Health Serv. Adm. Rockville, MD: https://files.eric.ed.gov/fulltext/ED478685.pdf
53. 53. 

    Lincoln YS, Guba EG 1985. Naturalistic Inquiry Beverly Hills, CA: Sage
54. 54. 

    Martinez JL, Duncan LR, Rivers SE, Bertoli MC, Latimer-Cheung AE, Salovey P 2017. Healthy Eating for Life English as a second language curriculum: applying the RE-AIM framework to evaluate a nutrition education intervention targeting cancer risk reduction. Transl. Behav. Med. 7:657–66
    [Google Scholar]
55. 55. 

    Marx A 2010. Crisp-set qualitative comparative analysis (csQCA) and model specification: benchmarks for future csQCA applications. Int. J. Mult. Res. Approaches 4:2138–58
    [Google Scholar]
56. 56. 

    Miles MB, Huberman AM 1994. Qualitative Data Analysis: An Expanded Sourcebook Thousand Oaks, CA: Sage, 2nd ed..
57. 57. 

    Moore GF, Audrey S, Barker M, Bond L, Bonell C et al. 2015. Process evaluation of complex interventions: Medical Research Council guidance. BMJ 350:h1258
    [Google Scholar]
58. 58. 

    Morse JM 2005. Evolving trends in qualitative research: advances in mixed-method design. Qual. Health Res. 15:5583–85
    [Google Scholar]
59. 59. 

    Morse JM, Niehaus L 2009. Mixed Method Design: Principles and Procedures Walnut Creek, CA: Left Coast Press
60. 60. 

    O'Cathain A, Murphy E, Nicholl J 2008. The quality of mixed methods studies in health services research. J. Health Serv. Res. Policy 13:292–98
    [Google Scholar]
61. 61. 

    Onwuegbuzie AJ, Poth C 2016. Editors’ afterword: toward evidence-based guidelines for reviewing mixed methods research manuscripts submitted to journals. Int. J. Qual. Methods 15:1 https://doi.org/10.1177/1609406916628986
    [Crossref] [Google Scholar]
62. 62. 

    Palinkas LA 2014. Qualitative and mixed methods in mental health services and implementation research. J. Clin. Child Adolesc. Psychol. 43:851–61
    [Google Scholar]
63. 63. 

    Palinkas LA, Aarons GA, Horwitz SM, Chamberlain P, Hurlburt M, Landsverk J 2011. Mixed method designs in implementation research. Adm. Policy Ment. Health 38:44–53
    [Google Scholar]
64. 64. 

    Palinkas LA, Campbell M, Saldana L 2018. Agency leaders’ assessments of feasibility and desirability of implementation of evidence-based practices in youth-serving organizations using the stages of implementation completion. Front. Public Health 6:161
    [Google Scholar]
65. 65. 

    Palinkas LA, Cooper BR 2018. Mixed methods evaluation in dissemination and implementation science. Dissemination and Implementation Research in Health: Translating Science to Practice RC Brownson, GA Colditz, EK Proctor 335–53 New York: Oxford Univ. Press, 2nd ed..
    [Google Scholar]
66. 66. 

    Palinkas LA, Darnell D, Zatzick D 2017. Developing clinical ethnographic implementation methods for rapid assessments in acute care clinical trials Paper presented at the 10th Conference on the Science of Dissemination and Implementation, Washington, DC, Dec. 5
67. 67. 

    Palinkas LA, Holloway IW, Rice E, Fuentes D, Wu Q, Chamberlain P 2011. Social networks and implementation of evidence-based practices in public youth-serving systems: a mixed methods study. Implement. Sci. 6:113
    [Google Scholar]
68. 68. 

    Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood KE 2015. Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Admin. Policy Ment. Health 42:533–44
    [Google Scholar]
69. 69. 

    Palinkas LA, Spear SE, Mendon SJ, Villamar J, Brown CH 2018. Development of a system for measuring sustainment of prevention programs and initiatives. Implement. Sci. 13:Suppl. 3A16 (Abstr.)
    [Google Scholar]
70. 70. 

    Palinkas LA, Spear SE, Mendon SJ, Villamar J, Valente T et al. 2016. Measuring sustainment of prevention programs and initiatives: a study protocol. Implement. Sci. 11:95
    [Google Scholar]
71. 71. 

    Patton MQ 2002. Qualitative Research and Evaluation Methods Thousand Oaks, CA: Sage, 3rd ed..
72. 72. 

    Pelto PJ 2015. What is so new about mixed methods?. Qual. Health Res. 25:6734–45
    [Google Scholar]
73. 73. 

    Pluye P, Hong QN 2014. Combining the power of stories and the power of numbers: mixed methods research and mixed studies reviews. Annu. Rev. Public Health 35:29–45
    [Google Scholar]
74. 74. 

    Powell BJ, Stanick CF, Halko HM, Dorsey CN, Weiner BJ et al. 2017. Toward criteria for pragmatic measurement in implementation research and practice: a stakeholder-driven approach using concept mapping. Implement. Sci. 12:118
    [Google Scholar]
75. 75. 

    Ragin CC 1987. The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies Berkeley: Univ. Calif. Press
76. 76. 

    Ragin CC 2009. Redesigning Social Inquiry: Fuzzy Sets and Beyond Chicago: Univ. Chicago Press
77. 77. 

    Rihoux B, Ragin CC 2008. Configurational Comparative Methods: Qualitative Comparative Analysis (QCA) and Related Techniques. Thousand Oaks, CA: Sage
78. 78. 

    Rogers E, Fernandez S, Gillespie C, Smelson D, Hagedorn HJ et al. 2013. Telephone care coordination for smokers in VA mental health clinics: protocol for a hybrid type-2 effectiveness-implementation trial. Addict. Sci. Clin. Pract. 8:7
    [Google Scholar]
79. 79. 

    Rosas LG, Lv N, Xiao L, Lewis MA, Zavella P et al. 2016. Evaluation of a culturally-adapted lifestyle intervention to treat elevated cardiometabolic risk of Latino adults in primary care (Vida Sana): a randomized controlled trial. Contemp. Clin. Trials. 48:30–40
    [Google Scholar]
80. 80. 

    Sandelowski M 2000. Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res. Nurs. Health 23:3246–55
    [Google Scholar]
81. 81. 

    Schneider CQ, Wagemann C 2010. Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comp. Sociol. 9:3397–418
    [Google Scholar]
82. 82. 

    Schwingel A, Gálvez P, Linares D, Sebastião E 2017. Using a mixed-methods RE-AIM framework to evaluate community health programs for older Latinas. J. Aging Health 29:4551–93
    [Google Scholar]
83. 83. 

    Schwitters A, Lederer P, Zilversmit L, Gudo PS, Ramiro I et al. 2015. Barriers to health care in rural Mozambique: a rapid assessment of planned mobile health clinics for ART. Glob. Health Sci. Pract. 3:109–16
    [Google Scholar]
84. 84. 

    Scrimshaw SCM, Hurtado E 1987. Rapid Assessment Procedures for Nutrition and Primary Health Care: Anthropological Approaches to Improving Programme Effectiveness Los Angeles, CA: UCLA Lat. Am. Cent.
85. 85. 

    Shanks CB, Harden S 2016. A reach, effectiveness, adoption, implementation, maintenance evaluation of weekend backpack food assistance programs. Am. J. Health Promot. 30:7511–20
    [Google Scholar]
86. 86. 

    Stange KC, Crabtree BF, Miller WL 2006. Publishing multimethod research. Ann. Fam. Med. 4:292–94
    [Google Scholar]
87. 87. 

    Swindle T, Johnson SL, Whiteside-Mansell L, Curran GM 2017. A mixed methods protocol for developing and testing implementation strategies for evidence-based obesity prevention in childcare: a cluster randomized hybrid type III trial. Implement. Sci. 12:90
    [Google Scholar]
88. 88. 

    Teddlie C, Tashakkori A 2003. Major issues and controversies in the use of mixed methods in the social and behavioral sciences. Handbook of Mixed Methods in the Social and Behavioral Sciences A Tashakkori, C Teddlie 3–50 Thousand Oaks, CA: Sage
    [Google Scholar]
89. 89. 

    Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD et al. 2009. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J. Clin. Epidemiol. 62:464–75
    [Google Scholar]
90. 90. 

    Thygeson NM, Solberg LL, Asche SE, Fontaine P, Pawlson LG, Scholle SH 2012. Using fuzzy set qualitative comparative analysis (fs/QCA) to explore the relationship between medical “homeness” and quality. Health Serv. Res. 47:1 Pt. 122–45
    [Google Scholar]
91. 91. 

    Trochim WMK 1989. An introduction to concept mapping for planning and evaluation. Eval. Prog. Plann. 12:1–16
    [Google Scholar]
92. 92. 

    Van der Kleij RM, Crone MR, Paulussen TG, van de Gar VM, Reis R 2015. A stitch in time saves nine? A repeated cross-sectional case study on the implementation of the intersectoral community approach Youth At a Healthy Weight. BMC Public Health 15:1032
    [Google Scholar]
93. 93. 

    Vindrola-Padros C, Vindrola-Padros B 2018. Quick and dirty? A systematic review of the use of rapid ethnographies in healthcare organization and delivery. BMJ Qual. Saf. 27:321–30
    [Google Scholar]
94. 94. 

    Waltz TJ, Powell BJ, Matthieu MM, Damschroder LJ, Chinman MJ et al. 2015. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement. Sci. 10:109
    [Google Scholar]
95. 95. 

    Watts BV, Shiner B, Zubkoff L, Carpenter-Song E, Ronconi JM, Coldwell CM 2014. Implementation of evidence-based psychotherapies for posttraumatic stress disorder in VA specialty clinics. Psychiatr. Serv. 65:5648–53
    [Google Scholar]
96. 96. 

    WHO (World Health Organ.). 2012. Changing mindsets: strategy on health policy and systems research Rep., WHO Geneva: http://www.who.int/alliance-hpsr/alliancehpsr_changingmindsets_strategyhpsr.pdf
97. 97. 

    Wright A, Sittig DF, Ash JS, Erikson JL, Hickman TT et al. 2015. Lessons learned from implementing service-oriented clinical decision support at four sites: a qualitative study. Int. J. Med. Inform. 84:901–11
    [Google Scholar]
98. 98. 

    Zatzick D, Rivera F, Jurkovich G, Russo J, Trusz SG et al. 2011. Enhancing the population impact of collaborative care interventions: mixed method development and implementation of stepped care targeting posttraumatic stress disorder and related comorbidities after acute trauma. Gen. Hosp. Psychiatry 33:123–34
    [Google Scholar]
99. 99. 

    Zatzick DF, Russo J, Darnell D, Chambers DA, Palinkas LA et al. 2016. An effectiveness-implementation hybrid trial study protocol targeting posttraumatic stress disorder and comorbidity. Implement. Sci. 11:58
    [Google Scholar]