1932

Abstract

The goal of this review is to enable clinical psychology researchers to more rigorously test competing hypotheses when studying risk factors in observational studies. We argue that there is a critical need for researchers to leverage recent advances in epidemiology/biostatistics related to causal inference and to use innovative approaches to address a key limitation of observational research: the need to account for confounding. We first review theoretical issues related to the study of causation, how causal diagrams can facilitate the identification and testing of competing hypotheses, and the current limitations of observational research in the field. We then describe two broad approaches that help account for confounding: analytic approaches that account for measured traits and designs that account for unmeasured factors. We provide descriptions of several such approaches and highlight their strengths and limitations, particularly as they relate to the etiology and treatment of behavioral health problems.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-clinpsy-032816-045030
2020-05-07
2024-06-21
Loading full text...

Full text loading...

/deliver/fulltext/clinpsy/16/1/annurev-clinpsy-032816-045030.html?itemId=/content/journals/10.1146/annurev-clinpsy-032816-045030&mimeType=html&fmt=ahah

Literature Cited

  1. Austin PC. 2011. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivar. Behav. Res. 46:399–424
    [Google Scholar]
  2. Bell RQ. 1968. A reinterpretation of the direction of effects in studies of socialization. Psychol. Rev. 75:81–95
    [Google Scholar]
  3. Brown HK, Ray JG, Wilton AS, Lunsky Y, Gomes T, Vigod SN 2017. Association between serotonergic antidepressant use during pregnancy and autism spectrum disorder in children. JAMA 317:1544–52
    [Google Scholar]
  4. Bulik-Sullivan B, Finucane HK, Anttila V, Gusev A, Day FR et al. 2015. An atlas of genetic correlations across human diseases and traits. Nat. Genet. 47:1236–41
    [Google Scholar]
  5. Caspi A, Moffitt TE, Morgan J, Rutter M, Taylor A et al. 2004. Maternal expressed emotion predicts children's antisocial behavior problems: using monozygotic-twin differences to identify environmental effects on behavioral development. Dev. Psychol. 40:149–61
    [Google Scholar]
  6. Clausson B, Lichtenstein P, Cnattingius S 2000. Genetic influence on birthweight and gestational length determined by studies in offspring of twins. BJOG 107:375–81
    [Google Scholar]
  7. Cohen J. 1988. Statistical Power Analysis for the Behavioral Sciences Hillsdale, NJ: Lawrence Erlbaum
    [Google Scholar]
  8. Cohen J, Cohen P, West SG, Aiken LS 2003. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences New York: Routledge
    [Google Scholar]
  9. Cole SR, Platt RW, Schisterman EF, Chu H, Westreich D et al. 2009. Illustrating bias due to conditioning on a collider. Int. J. Epidemiol. 39:417–20
    [Google Scholar]
  10. Cope MB, Allison DB. 2009. White hat bias: examples of its presence in obesity research and a call for renewed commitment to faithfulness in research reporting. Int. J. Obes. 34:84–88
    [Google Scholar]
  11. Cronbach LJ, Meehl PE. 1955. Construct validity in psychological tests. Psychol. Bull. 52:281–302
    [Google Scholar]
  12. Cumming G. 2014. The new statistics: why and how. Psychol. Sci 25:7–29
    [Google Scholar]
  13. Deaton A, Cartwright N. 2018. Understanding and misunderstanding randomized controlled trials. Soc. Sci. Med. 210:2–21Discussion of the assumptions and limitations inherent in randomized controlled trials.
    [Google Scholar]
  14. D'Onofrio BM, Class QA, Lahey BB, Larsson H 2014. Testing the developmental origins of health and disease hypothesis for psychopathology using family-based, quasi-experimental designs. Child Dev. Perspect. 8:151–57
    [Google Scholar]
  15. D'Onofrio BM, Class QA, Rickert ME, Larsson H, Langstrom N, Lichtenstein P 2013. Preterm birth and mortality and morbidity: a population-based quasi-experimental study. JAMA Psychiatry 70:1231–40
    [Google Scholar]
  16. D'Onofrio BM, Lahey BB, Turkheimer E, Lichtenstein P 2013. The critical need for family-based, quasi-experimental research in integrating genetic and social science research. Am. J. Public Health 103:S46–55Review of how family-based designs can help account for unmeasured confounding factors.
    [Google Scholar]
  17. D'Onofrio BM, Turkheimer E, Eaves LJ, Corey LA, Berg K et al. 2003. The role of the Children of Twins design in elucidating causal relations between parent characteristics and child outcomes. J. Child Psychol. Psychiatry 44:1130–44
    [Google Scholar]
  18. D'Onofrio BM, Viken RJ, Hetrick WP 2017. Science in clinical psychology. Toward a More Perfect Psychology: Improving Trust, Accuracy, and Transparency in Research MC Makel, JA Plucker 187–98 Washington, DC: Am. Psychol. Assoc.
    [Google Scholar]
  19. Elwert F, Winship C. 2014. Endogenous selection bias: the problem of conditioning on a collider variable. Annu. Rev. Sociol. 40:31–53
    [Google Scholar]
  20. Fewell Z, Davey Smith G, Sterne JAC 2007. The impact of residual and unmeasured confounding in epidemiologic studies: a simulation study. Am. J. Epidemiol 166:646–55
    [Google Scholar]
  21. Frisell T, Oberg S, Kuja-Halkola R, Sjolander A 2012. Sibling comparison designs: bias from non-shared confounders and measurement error. Epidemiology 23:713–20
    [Google Scholar]
  22. Gage SH, Munafò MR, Smith GD 2016. Causal inference in developmental origins of health and disease (DOHaD) research. Annu. Rev. Psychol. 67:567–85
    [Google Scholar]
  23. Glass GV, Willson VL, Gottman IM 1975. Design and Analysis of Time-Series Experiments Boulder, CO: Colorado Assoc. Univ. Press
    [Google Scholar]
  24. Gottesman II, Bertelsen A. 1989. Confirming unexpressed genotypes for schizophrenia: risks in the offspring of Fischer's Danish identical and fraternal discordant twins. Arch. Gen. Psychiatry 46:867–72
    [Google Scholar]
  25. Greenland S, Pearl J, Robins JM 1999. Causal diagrams for epidemiologic research. Epidemiology 10:37–48
    [Google Scholar]
  26. Hannigan LJ, Walaker N, Waszczuk MA, McAdams TA, Eley TC 2017. Aetiological influences on stability and change in emotional and behavioural problems across development: a systematic review. Psychopathol. Rev 4:52–108
    [Google Scholar]
  27. Hernan MA, Hernandez-Diaz S, Robins JM 2004. A structural approach to selection bias. Epidemiology 15:615–25
    [Google Scholar]
  28. Howards PP, Schisterman EF, Heagerty PJ 2007. Potential confounding by exposure history and prior outcomes: an example from perinatal epidemiology. Epidemiology 18:544–51
    [Google Scholar]
  29. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI et al. 2001. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 286:821–30
    [Google Scholar]
  30. Jaffee SR, Price TS. 2012. The implications of genotype–environment correlation for establishing causal processes in psychopathology. Dev. Psychopathol. 24:1253–64
    [Google Scholar]
  31. Kazdin AE, Kopel SA. 1975. On resolving ambiguities of the multiple-baseline design: problems and recommendations. Behav. Ther. 6:601–8
    [Google Scholar]
  32. Kazdin AE, Wilson GT. 1978. Evaluation of Behavior Therapy: Issues, Evidence, and Research Strategies Oxford, England: Ballinger
    [Google Scholar]
  33. Kendler KS. 2005. Toward a philosophical structure for psychiatry. Am. J. Psychiatry 162:433–40
    [Google Scholar]
  34. Kendler KS. 2017. Causal inference in psychiatric epidemiology. JAMA Psychiatry 74:561–62
    [Google Scholar]
  35. Kendler KS. 2019. From many to one to many—the search for causes of psychiatric illness. JAMA Psychiatry 76:1085–91
    [Google Scholar]
  36. Kendler KS, Gardner CO. 2010. Dependent stressful life events and prior depressive episodes in the prediction of major depression: the problem of causal inference in psychiatric epidemiology. Arch. Gen. Psychiatry 67:1120–27
    [Google Scholar]
  37. Knopik VS, Neiderhiser JM, DeFries JC, Plomin R 2016. Behavioral Genetics New York: Worth Publ.
    [Google Scholar]
  38. Kraemer HC, Kazdin AE, Offord DR, Kessler RC, Jensen PS, Kupfer DJ 1997. Coming to terms with the terms of risk. Arch. Gen. Psychiatry 54:337–43
    [Google Scholar]
  39. Kratochwill TR, Piersel WC. 1983. Time-series research: contributions to empirical clinical practice. Behav. Assess. 5:165–76
    [Google Scholar]
  40. Lahey BB, D'Onofrio BM. 2010. All in the family: comparing siblings to test causal hypotheses regarding environmental influences on behavior. Curr. Dir. Psychol. Sci. 19:319–23
    [Google Scholar]
  41. Lee J, Little TD. 2017. A practical guide to propensity score analysis for applied clinical research. Behav. Res. Ther. 98:76–90
    [Google Scholar]
  42. Leppert B, Havdahl A, Riglin L, Jones HJ, Zheng J et al. 2019. Association of maternal neurodevelopmental risk alleles with early-life exposures. JAMA Psychiatry 76:834–42
    [Google Scholar]
  43. Leve LD, Neiderhiser JM, Shaw DS, Ganiban J, Natsuaki MN, Reiss D 2013. The early growth and development study: a prospective adoption study from birth through middle childhood. Twin Res. Hum. Genet. 16:412–23
    [Google Scholar]
  44. Luellen JK, Shadish WR, Clark MH 2005. Propensity scores: an introduction and experimental test. Eval. Rev. 29:530–58
    [Google Scholar]
  45. Mayo D. 1996. Error and Growth of Experimental Knowledge Chicago: Univ. Chicago Press
    [Google Scholar]
  46. McAdams TA, Neiderhiser JM, Rijsdijk FV, Narusyte J, Lichtenstein P, Eley TC 2014. Accounting for genetic and environmental confounds in associations between parent and child characteristics: a systematic review of children-of-twins studies. Psychol. Bull. 140:1138–73
    [Google Scholar]
  47. McAdams TA, Rijsdijk FV, Neiderhiser JM, Narusyte J, Shaw DS et al. 2015. The relationship between parental depressive symptoms and offspring psychopathology: evidence from a children-of-twins study and an adoption study. Psychol. Med. 45:2583–94
    [Google Scholar]
  48. McGue M, Osler M, Christensen K 2010. Causal inference and observational research: the utility of twins. Perspect. Psychol. Sci. 5:546–56
    [Google Scholar]
  49. O'Donahue W. 2013. Clinical Psychology and the Philosophy of Science New York: Springer
    [Google Scholar]
  50. O'Donnell KJ, Meaney MJ. 2017. Fetal origins of mental health: the developmental origins of health and disease hypothesis. Am. J. Psychiatry 174:319–28
    [Google Scholar]
  51. Pearl J. 2009. Causality: Models, Reasoning, and Inference New York: Cambridge Univ. PressA comprehensive review of the analysis of causation.
    [Google Scholar]
  52. Platt JR. 1964. Strong inference. Science 146:347–53Discussion of how research progress is greatly facilitated by testing and ruling out competing hypotheses.
    [Google Scholar]
  53. Plomin R, Bergeman CS. 1991. The nature of nurture: genetic influence on “environmental” measures. Behav. Brain Sci. 14:3373–86
    [Google Scholar]
  54. Polderman TJC, Benyamin B, de Leeuw CA, Sullivan PF, van Bochoven A et al. 2015. Meta-analysis of the heritability of human traits based on fifty years of twin studies. Nat. Genet. 47:702–9
    [Google Scholar]
  55. Popper K. 1962. Conjectures and Refutations: The Growth of Scientific Knowledge New York: Basic Books
    [Google Scholar]
  56. Pridemore WA, Chamlin MB, Andreev E 2013. Reduction in male suicide mortality following the 2006 Russian alcohol policy: an interrupted time series analysis. Am. J. Public Health 103:2021–26
    [Google Scholar]
  57. Pridemore WA, Snowden AJ. 2009. Reduction in suicide mortality following a new national alcohol policy in Slovenia: an interrupted time-series analysis. Am. J. Public Health 99:915–20
    [Google Scholar]
  58. Rai D, Lee BK, Dalman C, Newschaffer C, Lewis G, Magnusson C 2017. Antidepressants during pregnancy and autism in offspring: population based cohort study. BMJ 358:j2811
    [Google Scholar]
  59. Rawlins M. 2008. De testimonio: on the evidence for decisions about the use of therapeutic interventions. Lancet 372:2152–61
    [Google Scholar]
  60. Robins JM. 2001. Data, design, and background knowledge in etiologic inference. Epidemiology 12:313–20
    [Google Scholar]
  61. Rohrer JM. 2018. Thinking clearly about correlations and causation: graphical causal models for observational data. Adv. Methods Pract. Psychol. Sci. 1:27–42Introduction to the use of causal diagrams for psychological researchers.
    [Google Scholar]
  62. Rosenbaum PR. 1984. The consequences of adjustment for a concomitant variable that has been affected by the treatment. J. R. Stat. Soc. A 147:656–66
    [Google Scholar]
  63. Rosenbaum PR, Rubin DB. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70:41–55
    [Google Scholar]
  64. Rothman KJ, Greenland S. 2005. Causation and causal inference in epidemiology. Am. J. Public Health 95:S144–50
    [Google Scholar]
  65. Rutter M. 2000. Psychosocial influences: critiques, findings, and research needs. Dev. Psychopathol. 12:375–405
    [Google Scholar]
  66. Rutter M, Pickles A, Murray R, Eaves LJ 2001. Testing hypotheses on specific environmental causal effects on behavior. Psychol. Bull. 127:291–324Review of the concepts of causation and tests of causal effects in psychological science.
    [Google Scholar]
  67. Rutter M, Silberg J, Simonoff E 1993. Whither behavior genetics? A developmental psychopathology perspective. Nature, Nurture, and Psychology R Plomin, GE McClearn 433–56 Washington, DC: Am. Psychol. Assoc.
    [Google Scholar]
  68. Schneeweiss S, Rassen JA, Glynn RJ, Avorn J, Mogun H, Brookhart MA 2009. High-dimensional propensity score adjustment in studies of treatment effects using health care claims data. Epidemiology 20:512–22
    [Google Scholar]
  69. Schwartz SJ, Lilienfeld SO, Meca A, Sauvigné KC 2016. The role of neuroscience within psychology: a call for inclusiveness over exclusiveness. Am. Psychol. 71:52–70
    [Google Scholar]
  70. Shadish WR, Cook TD, Campbell DT 2002. Experimental and Quasi-Experimental Designs for Generalized Causal Inference New York: Houghton MifflinFoundational text on causal inference for social science researchers.
    [Google Scholar]
  71. Silberg JL, Maes H, Eaves LJ 2010. Genetic and environmental influences on the transmission of parental depression to children's depression and conduct disturbance: an extended Children of Twins study. J. Child Psychol. Psychiatry 51:734–44
    [Google Scholar]
  72. Silberg JL, Parr T, Neale MC, Rutter M, Angold A, Eaves LJ 2003. Maternal smoking during pregnancy and risk to boys' conduct disturbance: an examination of the causal hypothesis. Biol. Psychiatry 53:130–35
    [Google Scholar]
  73. Singh AL, D'Onofrio BM, Slutske WS, Turkheimer E, Emery RE et al. 2011. Parental depression and offspring psychopathology: a Children of Twins study. Psychol. Med. 41:1385–95
    [Google Scholar]
  74. Sjolander A, Frisell T, Kuja-Halkola R, Oberg S, Zetterqvist J 2016. Carryover effects in sibling comparison designs. Epidemiology 27:852–58
    [Google Scholar]
  75. Smith JD. 2012. Single-case experimental designs: a systematic review of published research and current standards. Psychol. Methods 17:510–50
    [Google Scholar]
  76. Sujan AC, Öberg AS, Quinn PD, D'Onofrio BM 2019. Annual research review: maternal antidepressant use during pregnancy and offspring neurodevelopmental problems—a critical review and recommendations for future research. J. Child Psychol. Psychiatry 60:356–76
    [Google Scholar]
  77. Sujan AC, Rickert ME, Öberg AS, Quinn PD, Hernández-Díaz S et al. 2017. Associations of maternal antidepressant use during the first trimester of pregnancy with preterm birth, small for gestational age, autism spectrum disorder, and attention-deficit/hyperactivity disorder in offspring. JAMA 317:1553–62
    [Google Scholar]
  78. Tackett JL, Lilienfeld SO, Patrick CJ, Johnson SL, Krueger RF et al. 2017. It's time to broaden the replicability conversation: thoughts for and from clinical psychological science. Perspect. Psychol. Sci. 12:742–56
    [Google Scholar]
  79. Thapar A, Rutter M. 2019. Do natural experiments have an important future in the study of mental disorders?. Psychol. Med 49:1079–88
    [Google Scholar]
  80. Turkheimer E. 2000. Three laws of behavior genetics and what they mean. Curr. Dir. Psychol. Sci. 9:160–64
    [Google Scholar]
  81. West SG. 2009. Alternatives to randomized experiments. Curr. Dir. Psychol. Sci. 18:299–304
    [Google Scholar]
  82. West SG, Cham H, Thoemmes F, Renneberg B, Schulze J, Weiler M 2014. Propensity scores as a basis for equating groups: basic principles and application in clinical treatment outcome research. J. Consult. Clin. Psychol. 82:906–19Review of how propensity scores can help account for measured traits in clinical psychology research.
    [Google Scholar]
  83. Westfall J, Yarkoni T. 2016. Statistically controlling for confounding constructs is harder than you think. PLOS ONE 11:e0152719Review of the limitations of statistically adjusting for measured covariates due to measurement error.
    [Google Scholar]
  84. Wing C, Simon K, Bello-Gomez RA 2018. Designing difference in difference studies: best practices for public health policy research. Annu. Rev. Public Health 39:453–69
    [Google Scholar]
/content/journals/10.1146/annurev-clinpsy-032816-045030
Loading
/content/journals/10.1146/annurev-clinpsy-032816-045030
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error