1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Clinical Psychology
  4. Volume 12, 2016
  5. Article

Annual Review of Clinical Psychology

Volume 12, 2016

Computerized Adaptive Diagnosis and Testing of Mental Health Disorders

Abstract

In this review we explore recent developments in computerized adaptive diagnostic screening and computerized adaptive testing for the presence and severity of mental health disorders such as depression, anxiety, and mania. The statistical methodology is unique in that it is based on multidimensional item response theory (severity) and random forests (diagnosis) instead of traditional mental health measurement based on classical test theory (a simple total score) or unidimensional item response theory. We show that the information contained in large item banks consisting of hundreds of symptom items can be efficiently calibrated using multidimensional item response theory, and the information contained in these large item banks can be precisely extracted using adaptive administration of a small set of items for each individual. In terms of diagnosis, computerized adaptive diagnostic screening can accurately track an hour-long face-to-face clinician diagnostic interview for major depressive disorder (as an example) in less than a minute using an average of four questions with unprecedented high sensitivity and specificity. Directions for future research and applications are discussed.

Keyword(s): anxiety, CAT, computerized adaptive testing, depression, differential item functioning, IRT, item response theory, mania, mental health diagnosis, mental health measurement
Loading

Article metrics loading...

/content/journals/10.1146/annurev-clinpsy-021815-093634
2016-03-28
2024-04-25
Loading full text...

Full text loading...

/deliver/fulltext/clinpsy/12/1/annurev-clinpsy-021815-093634.html?itemId=/content/journals/10.1146/annurev-clinpsy-021815-093634&mimeType=html&fmt=ahah

Literature Cited

  1. Achtyes ED, Halstead S, Smart L, Moore T, Frank E. et al. 2015. Validation of computerized adaptive testing in an outpatient non-academic setting: the VOCATIONS Trial. Psychiatr. Serv. 661091–96
  2. Am. Psychiatr. Assoc 1994. Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: Am. Psychiatr. Publ., 4th ed..
  3. Am. Psychiatr. Assoc 2013. Diagnostic and Statistical Manual of Mental Disorders. Washington, DC: Am. Psychiatr. Publ., 5th ed..
  4. Andrich D. 1978a. A rating formulation for ordered response categories. Psychometrika 43:561–71 [Google Scholar]
  5. Andrich D. 1978b. Application of a psychometric rating model to ordered categories which are scored with successive integers. Appl. Psychol. Meas. 2:581–94 [Google Scholar]
  6. Andrich D. 1988. The application of an unfolding model of the IRT type to the measurement of attitude. Appl. Psychol. Meas. 12:33–51 [Google Scholar]
  7. Baek SG. 1997. Computerized adaptive testing using the partial credit model for attitude measurement. Objective Measurement: Theory into Practice M Wilson, G Engelhard Jr, K Draney 37–52 Norwood, NJ: Ablex [Google Scholar]
  8. Baker FB. 1992. Item Response Theory: Parameter Estimation Techniques. New York: Marcel Dekker
  9. Bock JO, Luppa M, Brettschneider C, Riedel-Heller S, Bickel H. et al. 2014. Impact of depression on health care utilization and costs among multimorbid patients—results from the MultiCare Cohort Study. PLOS ONE 9:e91973 [Google Scholar]
  10. Bock RD, Aitkin M. 1981. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika 46:443–59 [Google Scholar]
  11. Brieman L. 1996. Bagging predictors. Mach. Learn. 24:123–40 [Google Scholar]
  12. Brieman L. 2001. Random forests. Mach. Learn. 45:5–32 [Google Scholar]
  13. Brieman L, Friedman JH, Olshen R, Stone C. 1984. Classification and Regression Trees Belmont CA: Wadsworth
  14. Brown JM, Weiss DJ. 1977. An adaptive testing strategy for achievement test batteries Res. Rep. 77-6. Minneapolis: Univ. Minn., Dep. Psychol., Psychom. Meth. Program, Comput. Adapt. Test. Lab.
  15. Dodd BG, DeAyala RJ, Koch WR. 1995. Computerized adaptive testing with polytomous items. Appl. Psychol. Meas. 19:5–22 [Google Scholar]
  16. Elderon L, Smolderen KG, Na B, Whooley MA. 2011. Accuracy and prognostic value of American Heart Association—recommended depression screening in patients with coronary heart disease: data from the Heart and Soul Study. Circ.: Cardiovasc. Qual. Outcomes 4:533–40 [Google Scholar]
  17. Embretson S, Reise S. 2000. Item Response Theory for Psychologists Mahwah, NJ: Erlbaum
  18. First MB, Spitzer RL, Gibbon M, Williams JB. 1996. Structured Clinical Interview for the DSM-IV Axis I Disorders Clinician Version (SCID-CV) Washington, DC: Am. Psychiatr. Publ. [Google Scholar]
  19. Fliege H, Becker J, Walter OB, Bjorner JB, Burghard F, Rose M. 2005. Development of a computer-adaptive test for depression (D-CAT). Qual. Life Res. 14:102277–91 [Google Scholar]
  20. Freund Y, Shapire R. 1996. Experiments with a new boosting algorithm, machine learning. Proc. 13th Int. Conf. Mach. Learn.148–56 Burlington, MA: Morgan Kauffman [Google Scholar]
  21. Gardner W, Shear K, Kelleher KJ, Pajer KA, Mammen O. et al. 2004. Computerized adaptive measurement of depression: a simulation study. BMC Psychiatry 4:13 [Google Scholar]
  22. Gibbons RD, Bock D, Hedeker D, Weiss D, Segawa E. et al. 2007. Full-information item bifactor analysis of graded response data. Appl. Psychol. Meas. 31:4–19 [Google Scholar]
  23. Gibbons RD, Brown CH, Hur K. 2012a. Is the rate of suicide among veterans elevated?. Am. J. Public Health 102:S17–19 [Google Scholar]
  24. Gibbons RD, Hedeker D. 1992. Full information item bi-factor analysis. Psychometrika 57:423–36 [Google Scholar]
  25. Gibbons RD, Hooker G, Finkelman MD, Weiss DJ, Pilkonis PA. et al. 2013. The CAD-MDD: a computerized adaptive diagnostic screening tool for depression. J. Clin. Psychiatry 74:669–74 [Google Scholar]
  26. Gibbons RD, Weiss DJ, Pilkonis PA, Frank E, Moore T. et al. 2012b. The CAT-DI: a computerized adaptive test for depression. Arch. Gen. Psychiatry 69:1104–12 [Google Scholar]
  27. Gibbons RD, Weiss DJ, Pilkonis PA, Frank E, Moore T. et al. 2014. Development of the CAT-ANX: a computerized adaptive test for anxiety. Am. J. Psychiatry 171:187–94 [Google Scholar]
  28. Hambleton RK, Swaminathan H. 1985. Item Response Theory: Principles and Applications Boston: Kluwer-Nijhoff
  29. Hastie T, Tibshirani R, Friedman JH. 2009. Elements of Statistical Learning New York: Springer
  30. Hawton K. 1992. Suicide and attempted suicide. Handbook of Affective Disorders ES Paykel 635–50 Edinburgh: Churchill Livingstone [Google Scholar]
  31. Holzinger KJ, Swineford F. 1937. The bi-factor method. Psychometrika 2:41–54 [Google Scholar]
  32. Joiner T. 2010. Myths about Suicide Cambridge, MA: Harvard Univ. Press
  33. Joreskog K. 1969. A general approach to confirmatory maximum likelihood factor analysis. Psychometrika 34:183–02 [Google Scholar]
  34. Kingsbury GG, Weiss DJ. 1980. An alternate-forms reliability and concurrent validity comparison of Bayesian adaptive and conventional ability tests. Res. Rep. 80-5 Minneapolis: Univ. Minn., Dep. Psychol., Psychom. Meth. Program, Comput. Adapt. Test. Lab.
  35. Kingsbury GG, Weiss DJ. 1983. A comparison of IRT-based adaptive mastery testing and a sequential mastery testing procedure. See Weiss 1983 257–83
  36. Mathers CD, Loncar D. 2006. Projections of global mortality and burden of disease from 2002 to 2030. Public Libr. Sci. Med. 3:e442 [Google Scholar]
  37. McBride JR, Martin JR. 1983. Reliability and validity of adaptive ability tests in a military setting. See Weiss 1983 223–36
  38. Mitchell AJ, Vaze A, Rao S. 2009. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet 374:609–19 [Google Scholar]
  39. Muraki E. 1990. Fitting a polytomous item response model to Likert-type data. Appl. Psychol. Meas. 14:59–71 [Google Scholar]
  40. Muthén BO. 1989. Latent variable modeling in heterogeneous populations. Psychometrika 54:557–85 [Google Scholar]
  41. Natl. Inst. Mental Health 2013. Major depressive disorder among adults. http://www.nimh.nih.gov/health/statistics/prevalence/major-depression-among-adults.shtml
  42. Pilkonis PA, Choi SW, Reise SP, Stover AM, Riley WT, Cella D. 2011. PROMIS Cooperative Group. Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS): depression, anxiety, and anger. Assessment 18:263–83 [Google Scholar]
  43. Pratt LA, Brody DJ. 2010. Depression and smoking in the U.S. household population aged 20 and over, 2005–2008. NCHS Data Brief 34:1–8 [Google Scholar]
  44. Pyne JM, Rost KM, Zhang M, Williams DK, Smith J, Fortney J. 2003. Cost-effectiveness of a primary care depression intervention. J. Gen. Intern. Med. 18:432–41 [Google Scholar]
  45. Quinlan R. 1993. C4.5: Programs for Machine Learning Burlington, MA: Morgan Kaufmann
  46. Reise SP, Waller NG. 1991. Fitting the two-parameter model to personality data. Appl. Psychol. Meas. 15:45–58 [Google Scholar]
  47. Samejima F. 1969. Estimation of latent ability using a response pattern of graded scores. Psychometrika Monogr. Suppl. 17:1–68 [Google Scholar]
  48. Seo DG, Weiss. 2015. Best design for multidimensional computerized adaptive testing with the bifactor model. Educ. Psychol. Meas. doi: 10.1177/0013164415575147. In press
  49. Smith JP, Smith GC. 1982. Long-term economic costs of psychological problems during childhood. Soc. Sci. Med. 71:110–15 [Google Scholar]
  50. Stewart WF, Ricci JA, Chee E, Hahn SR, Morganstein D. 2003. Cost of lost productive work time among US workers with depression. JAMA 289:3135 [Google Scholar]
  51. Stuart A. 1958. Equally correlated variates and the multinormal integral. J. R. Stat. Soc. Ser. B 20:373–78 [Google Scholar]
  52. Tucker LR. 1958. An inter-battery method of factor analysis. Psychometrika 23:111–36 [Google Scholar]
  53. Unützer J, Katon WJ, Fan MY. 2008. Long-term cost effects of collaborative care for late-life depression. Am. J. Manag. Care 14:95–100 [Google Scholar]
  54. Vale CD, Weiss DJ. 1984. A rapid item-search procedure for Bayesian adaptive testing Res. Rep. 77-4 Minneapolis: Univ. Minn., Dep. Psychol., Psychom. Meth. Program, Comput. Adapt. Test. Lab.
  55. Weiss DJ. 1983. New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing New York: Academic
  56. Weiss DJ. 1985. Adaptive testing by computer. J. Consult. Clin. Psychol. 53:774–89 [Google Scholar]
  57. Weiss DJ, Kingsbury GG. 1984. Application of computerized adaptive testing to educational problems. J. Educ. Meas. 21:361–75 [Google Scholar]
  58. Weiss DJ, McBride JR. 1984. Bias and information of Bayesian adaptive testing. Appl. Psychol. Meas. 8:272–85 [Google Scholar]
  59. Whooley M. 2012. Diagnosis and treatment of depression in adults with comorbid medical conditions. JAMA 307:1848–57 [Google Scholar]
/content/journals/10.1146/annurev-clinpsy-021815-093634
Loading
Computerized Adaptive Diagnosis and Testing of Mental Health Disorders
Annual Review of Clinical Psychology 12, 83 (2016); https://doi.org/10.1146/annurev-clinpsy-021815-093634
/content/journals/10.1146/annurev-clinpsy-021815-093634
/content/journals/10.1146/annurev-clinpsy-021815-093634
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error