1932

Abstract

This review focuses on the use of multilevel models in psychology and other social sciences. We target readers who are catching up on current best practices and sources of controversy in the specification of multilevel models. We first describe common use cases for clustered, longitudinal, and cross-classified designs, as well as their combinations. Using examples from both clustered and longitudinal designs, we then address issues of centering for observed predictor variables: its use in creating interpretable fixed and random effects of predictors, its relationship to endogeneity problems (correlations between predictors and model error terms), and its translation into multivariate multilevel models (using latent-centering within multilevel structural equation models). Finally, we describe novel extensions—mixed-effects location–scale models—designed for predicting differential amounts of variability.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-psych-020821-103525
2022-01-04
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/psych/73/1/annurev-psych-020821-103525.html?itemId=/content/journals/10.1146/annurev-psych-020821-103525&mimeType=html&fmt=ahah

Literature Cited

  1. Aguinis H, Gottfredson RK, Culpepper SA. 2013. Best-practice recommendations for estimating cross-level interaction effects using multilevel modeling. J. Manag. 39:61490–528
    [Google Scholar]
  2. Algina J, Swaminathan H 2011. Centering in two-level nested designs. Handbook of Advanced Multilevel Analysis J Hox, JK Roberts 285–312 New York: Taylor & Francis
    [Google Scholar]
  3. Antonakis J, Bastardoz N, Rönkkö M. 2021. On ignoring the random effects assumption in multilevel models: review, critique, and recommendations. Organ. Res. Methods 24:4443–83
    [Google Scholar]
  4. Asparouhov T, Muthén B. 2019. Latent variable centering of predictors and mediators in multilevel and time-series models. Struct. Equ. Model. 26:1119–42
    [Google Scholar]
  5. Barr DJ, Levy R, Scheepers C, Tily HJ. 2013. Random effects structure for confirmatory hypothesis testing: Keep it maximal. J. Memory Lang. 68:3255–78
    [Google Scholar]
  6. Bauer DJ. 2003. Estimating multilevel linear models as structural equation models. J. Educ. Behav. Stat. 28:2135–67
    [Google Scholar]
  7. Bell A, Fairbrother M, Jones K. 2019. Fixed and random effects models: making an informed choice. Qual. Quant. 53:1051–74
    [Google Scholar]
  8. Berkhof J, Kampen JK. 2004. Asymptotic effect of misspecification in the random part of the multilevel model. J. Educ. Behav. Stat. 29:2201–18
    [Google Scholar]
  9. Berry D, Willoughby M. 2017. On the practical interpretability of cross-lagged panel models: rethinking a developmental workhorse. Child Dev 88:41186–206
    [Google Scholar]
  10. Blalock HM. 1984. Contextual-effects models: theoretical and methodological issues. Annu. Rev. Sociol. 10:353–72
    [Google Scholar]
  11. Bliese PD, Maltarich MA, Hendricks JL. 2018. Back to basics with mixed-effects models: nine take-away points. J. Bus. Psychol. 33:1–23
    [Google Scholar]
  12. Blozis SA, McTernan M, Harring JR, Zheng Q. 2020. Two-part mixed-effects location scale models. Behav. Res. Methods 52:1836–47
    [Google Scholar]
  13. Brauer M, Curtin JJ. 2018. Linear mixed-effects models and the analysis of nonindependent data: a unified framework to analyze categorical and continuous independent variables that vary within-subjects and/or within-items. Psychol. Methods 23:3389–411
    [Google Scholar]
  14. Brincks AM, Enders CK, Llabre MM, Bulotsky-Shearer RJ, Prado G, Feaster DJ. 2017. Centering predictor variables in three-level contextual models. Multivar. Behav. Res. 52:2149–63
    [Google Scholar]
  15. Brunton-Smith I, Sturgis P, Leckie G 2017. Detecting and understanding interviewer effects on survey data by using a cross-classified mixed effects location-scale model. J. R. Stat. Soc. Ser. A 180:551–68
    [Google Scholar]
  16. Bürkner PC. 2018. Advanced Bayesian multilevel modeling with the R package brms. R J. 10:1395–411
    [Google Scholar]
  17. Burstein L. 1980. The analysis of multilevel data in educational research and evaluation. Rev. Res. Educ. 8:158–233
    [Google Scholar]
  18. Cleveland W, Denby L, Liu C. 2000. Random location and scale effects: model building methods for a general class of models. Comput. Sci. Stat. 32:3–10
    [Google Scholar]
  19. Courvoisier D, Walls TA, Cheval B, Hedeker D. 2019. A mixed-effects location scale model for time-to-event data: a smoking behavior application. Addict. Behav. 94:42–49
    [Google Scholar]
  20. Culpepper SA. 2010. Studying individual differences in predictability with gamma regression and nonlinear multilevel models. Multivar. Behav. Res. 45:1153–85
    [Google Scholar]
  21. Curran PJ, Bauer DJ. 2011. The disaggregation of within-person and between-person effects in longitudinal models of change. Annu. Rev. Psychol. 62:583–619
    [Google Scholar]
  22. Curran PJ, Howard AL, Bainter SA, Lane ST, McGinley JS 2014. The separation of between-person and within-person components of individual change over time: a latent curve model with structured residuals. J. Consult. Clin. Psychol. 82:5879–94
    [Google Scholar]
  23. Curran PJ, Lee T, Howard AL, Lane S, MacCallum R 2012. Disaggregating within-person and between-person effects in multilevel and structural equation growth models. Advances in Longitudinal Methods in the Social and Behavioral Sciences JR Harring, GR Hancock 217–53 Charlotte, NC: Inf. Age Publ.
    [Google Scholar]
  24. Cushing CC, Walters RW, Hoffman L. 2014. Aggregated N-of-1 randomized controlled trials: modern data analytics applied to a clinically valid method of intervention effectiveness. J. Pediatr. Psychol. 39:2138–50
    [Google Scholar]
  25. Eckardt R, Yammarino FJ, Dionne SD, Spain SM 2021. Multilevel methods and statistics: the next frontier. Organ. Res. Methods 24:4187–218
    [Google Scholar]
  26. Enders CK. 2013. Centering predictors and contextual effects. See Scott et al. 2013 89–107
  27. Enders CK, Tofighi D. 2007. Centering predictor variables in cross-sectional multilevel models: a new look at an old issue. Psychol. Methods 12:2121–38
    [Google Scholar]
  28. Estrada E, Hamagami F, Ferrer E. 2020. Estimating age-based developmental trajectories using latent change score models based on measurement occasion. Multivar. Behav. Res. 55:3454–77
    [Google Scholar]
  29. Foulley JL, Quaas RL. 1995. Heterogeneous variances in Gaussian linear mixed models. Genet. Sel. Evol. 27:211–28
    [Google Scholar]
  30. Grund S, Lüdtke O, Robitzsch A 2019. Missing data in multilevel research. See Humphrey & LeBreton 2019 365–86
  31. Hamaker EL, Klugkist I 2011. Bayesian estimation of multilevel models. European Association for Methodology Series. Handbook for Advanced Multilevel Analysis JJ Hox, JK Roberts 137–61 Oxfordshire, UK: Routledge/Taylor & Francis Group
    [Google Scholar]
  32. Hamaker EL, Kuiper RM, Grasman RPPP. 2015. A critique of the cross-lagged panel model. Psychol. Methods 20:1102–16
    [Google Scholar]
  33. Hamaker EL, Muthén B. 2020. The fixed versus random effects debate and how it relates to centering in multilevel modeling. Psychol. Methods 25:3365–79
    [Google Scholar]
  34. Hedeker D, Gibbons RD. 2006. Longitudinal Data Analysis Hoboken, NJ: Wiley-Interscience
  35. Hedeker D, Mermelstein RJ, Berbaum ML, Campbell RT. 2009. Modeling mood variation associated with smoking: an application of heterogeneous mixed-effects model for analysis of ecological momentary assessment (EMA) data. Addiction 104:2297–307
    [Google Scholar]
  36. Hedeker D, Mermelstein RJ, Demirtas H. 2008. An application of a mixed-effects location scale model for analysis of ecological momentary assessment (EMA) data. Biometrics 64:2627–34
    [Google Scholar]
  37. Hedeker D, Mermelstein RJ, Demirtas H, Berbaum ML. 2016. A mixed-effects location-scale model for ordinal questionnaire data. Health Serv. Outcomes Res. Methodol. 16:3117–31
    [Google Scholar]
  38. Hedeker D, Nordgren R. 2013. MIXREGLS: a program for mixed-effects location scale analysis. J. Stat. Softw. 52:121–38
    [Google Scholar]
  39. Heisig JP, Schaeffer M. 2019. Why you should always include a random slope for the lower-level variable involved in a cross-level interaction. Eur. Sociol. Rev. 35:2258–79
    [Google Scholar]
  40. Hoffman L 2012. Considering alternative metrics of time: Does anybody really know what “time” is?. Advances in Longitudinal Methods in the Social and Behavioral Sciences JR Harring, GR Hancock 255–87 Charlotte, NC: Inf. Age Publ.
    [Google Scholar]
  41. Hoffman L. 2015. Longitudinal Analysis: Modeling Within-Person Fluctuation and Change Oxfordshire, UK: Routledge/Taylor & Francis Group
  42. Hoffman L. 2019. On the interpretation of parameters in multivariate multilevel models across different combinations of model specification and estimation. Adv. Methods Pract. Psychol. Sci. 2:3288–311
    [Google Scholar]
  43. Hoffman L. 2021. Disaggregating associations of between-person differences in change over time from associations of within-person fluctuation in longitudinal data. PsyArXiv https://doi.org/10.31234/osf.io/qtc7r
    [Crossref] [Google Scholar]
  44. Hoffman L, Stawski RS. 2009. Persons as contexts: evaluating between-person and within-person effects in longitudinal analysis. Res. Hum. Dev. 6:2–397–210
    [Google Scholar]
  45. Hofmann DA, Gavin MB. 1998. Centering decisions in hierarchical linear models: implications for research in organizations. J. Manag. 24:5623–41
    [Google Scholar]
  46. Hox J. 2010. Multilevel Analysis: Techniques and Applications New York: Routledge Academic, 2nd ed..
  47. Hox J, Roberts K 2011. Multilevel analysis: where we were and where we are. Handbook of Advanced Multilevel Analysis J Hox, JK Roberts 3–14 New York: Taylor & Francis
    [Google Scholar]
  48. Humphrey SE, LeBreton JM 2019. The Handbook of Multilevel Theory, Measurement, and Analysis Washington, DC: Am. Psychol. Assoc.
  49. Judd CM, Westfall J, Kenny DA. 2017. Experiments with more than one random factor: designs, analytic models, and statistical power. Annu. Rev. Psychol. 68:601–25
    [Google Scholar]
  50. Ke Z, Wang L. 2015. Detecting individual differences in change: methods and comparisons. Struct. Equ. Model. 22:3382–400
    [Google Scholar]
  51. Kreft IGG, de Leeuw J, Aiken LS. 1995. The effect of different forms of centering in hierarchical linear models. Multivar. Behav. Res. 30:11–21
    [Google Scholar]
  52. LaHuis DM, Ferguson MW. 2009. The accuracy of significance tests for slope variance components in multilevel random coefficient models. Organ. Res. Methods 12:3418–35
    [Google Scholar]
  53. LaHuis DM, Jenkins DR, Hartman MJ, Hakoyama S, Clark PC 2020. The effects of misspecifying the random part of multilevel models. Methodology 16:3224–40
    [Google Scholar]
  54. Leckie G, French R, Charlton C, Browne W 2014. Modeling heterogeneous variance-covariance components in two-level models. J. Educ. Behav. Stat. 39:5307–32
    [Google Scholar]
  55. Lee Y, Nelder JA. 2006. Double hierarchical generalized linear models. Appl. Stat. 55:2139–85
    [Google Scholar]
  56. Lee Y, Noh M. 2012. Modelling random effect variance with double hierarchical generalized linear models. Stat. Model. 12:6487–502
    [Google Scholar]
  57. Lester HF, Cullen-Lester KL, Walters RW. 2021. From nuisance to novel research questions: using multilevel models to predict heterogeneous variances. Organ. Res. Methods 24:4342–88
    [Google Scholar]
  58. Li X, Hedeker D 2012. A three-level mixed-effects location scale model with an application to ecological momentary assessment (EMA) data. Stat. Med. 31:263192–210
    [Google Scholar]
  59. Lin X, Mermelstein RJ, Hedeker D. 2018. A three-level Bayesian mixed effects location scale model with an application to ecological momentary assessment data. Stat. Med. 37:132108–19
    [Google Scholar]
  60. Liu Y, West SG. 2015. Weekly cycles in daily report data: an overlooked issue. J. Personal 84:5560–79
    [Google Scholar]
  61. Loeys T, Josephy H, Dewitte M. 2018. More precise estimation of lower-level interaction effects in multilevel models. Multivar. Behav. Res. 53:3335–47
    [Google Scholar]
  62. Lu T. 2018. Mixed-effects location and scale Tobit joint models for heterogeneous longitudinal data with skewness, detection limits, and measurement errors. Stat. Methods Med. Res. 27:123525–43
    [Google Scholar]
  63. Lüdtke O, Marsh HW, Robitzsch A, Trautwein U 2011. A 2 × 2 taxonomy of multilevel latent contextual models: accuracy-bias trade-offs in full and partial error correction models. Psychol. Methods 16:4444–67
    [Google Scholar]
  64. Lüdtke O, Marsh HW, Robitzsch A, Trautwein U, Asparouhov T, Muthén B 2008. The multilevel latent covariate model: a new, more reliable approach to group-level effects in contextual studies. Psychol. Methods 13:3203–29
    [Google Scholar]
  65. Luo W, Kwok O. 2009. The impacts of ignoring a crossed factor in analyzing cross-classified data. Multivar. Behav. Res. 44:2182–212
    [Google Scholar]
  66. Ma Q, Dunton GF, Hedeker D. 2021. A mixed effect location-scale model with mixture distributed scale random effects to analyze (near) identical entries in ecological momentary assessments. Multivar. Behav. Res. 56:1160
    [Google Scholar]
  67. Matuschek H, Kliegl R, Vasishth S, Baayen H, Bates D. 2017. Balancing type I error and power in linear mixed models. J. Memory Lang. 94:305–15
    [Google Scholar]
  68. McNeish D. 2017a. Multilevel mediation with small samples: a cautionary note on the multilevel structural equation modeling framework. Struct. Equ. Model. 24:4609–25
    [Google Scholar]
  69. McNeish D. 2017b. Small sample methods for multilevel modeling: a colloquial elucidation of REML and the Kenward-Roger correction. Multivar. Behav. Res. 52:5661–70
    [Google Scholar]
  70. McNeish D. 2021. Specifying location-scale models for heterogeneous variances as multilevel SEMs. Organ. Res. Methods 24:3630–53
    [Google Scholar]
  71. McNeish D, Hamaker EL. 2020. A primer on two-level dynamic structural equation models for intensive longitudinal data in Mplus. Psychol. Methods 25:5610–35
    [Google Scholar]
  72. McNeish D, Kelley K. 2019. Fixed effects models versus mixed effects models for clustered data: reviewing the approaches, disentangling the differences, and making recommendations. Psychol. Methods 24:120–35
    [Google Scholar]
  73. McNeish D, Matta T. 2018. Differentiating between mixed-effects and latent-curve approaches to growth modeling. Behav. Res. Methods 50:1398–414
    [Google Scholar]
  74. Mehta PD, Neale MC 2005. People are variables too: multilevel structural equations modeling. Psychol. Methods 10:3259–84
    [Google Scholar]
  75. Meuleman B, Billiet J. 2009. A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM?. Surv. Res. Methods 3:145–58
    [Google Scholar]
  76. Meyers J, Beretvas S. 2006. The impact of inappropriate modeling of cross-classified data structures. Multivar. Behav. Res. 41:473–97
    [Google Scholar]
  77. Moerbeek M. 2004. The consequence of ignoring a level of nesting in multilevel analysis. Multivar. Behav. Res. 39:1129–49
    [Google Scholar]
  78. O'Keefe P, Rodgers JL 2017. Double decomposition of level-1 variables in multilevel models: an analysis of the Flynn effect in the NSLY data. Multivar. Behav. Res. 52:5630–47
    [Google Scholar]
  79. Park J, Cardwell R, Yu H-T. 2020. Specifying the random effect structure in linear mixed effect models for analyzing psycholinguistic data. Methodology 16:292–111
    [Google Scholar]
  80. Preacher KJ, Zhang Z, Zyphur MJ. 2011. Alternative methods for assessing mediation in multilevel data: the advantages of multilevel SEM. Struct. Equ. Model. 18:2161–82
    [Google Scholar]
  81. Preacher KJ, Zhang Z, Zyphur MJ. 2016. Multilevel structural equation models for assessing moderation within and across levels of analysis. Psychol. Methods 21:2189–205
    [Google Scholar]
  82. Preacher KJ, Zyphur MJ, Zhang Z. 2010. A general multilevel SEM framework for assessing multilevel mediation. Psychol. Methods 15:3209–33
    [Google Scholar]
  83. Rast P, Ferrer E. 2018. A mixed-effects location scale model for dyadic interactions. Multivar. Behav. Res. 53:5756–75
    [Google Scholar]
  84. Rast P, Hofer SM, Sparks C. 2012. Modeling individual differences in within-person variation of negative and positive affect in a mixed effects location scale model using BUGS/JAGS. Multivar. Behav. Res. 47:2177–200
    [Google Scholar]
  85. Raudenbush SW, Bryk AS. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods Thousand Oaks, CA: Sage, 2nd ed..
  86. Rights JD, Preacher KJ, Cole DA. 2020. The danger of conflating level-specific effects of control variables when primary interest lies in level-2 effects. Br. J. Math. Stat. Psychol. 73:194–211
    [Google Scholar]
  87. Rights JD, Sterba SK. 2019. Quantifying explained variance in multilevel models: an integrative framework for defining R-squared measures. Psychol. Methods 24:3309–38
    [Google Scholar]
  88. Rights JD, Sterba SK. 2020. New recommendations on the use of R-squared differences in multilevel model comparisons. Multivar. Behav. Res. 55:4568–99
    [Google Scholar]
  89. Rights JD, Sterba SK. 2021. R-squared measures for multilevel models with three or more levels. Multivariate Behav. Res. In press
    [Google Scholar]
  90. Schmidt-Catran AW, Fairbrother M. 2016. The random effects in multilevel models: getting them wrong and getting them right. Eur. Sociol. Rev. 32:123–38
    [Google Scholar]
  91. Scott MA, Simonof JS, Marx BP 2013. The SAGE Handbook of Multilevel Modeling London: Sage
  92. Shi Y, Leite W, Algina J 2010. The impact of omitting the interaction between crossed factors in cross-classified random effects modelling. Br. J. Math. Stat. Psychol. 63:Part 11–15
    [Google Scholar]
  93. Singer J, Willett J. 2003. Applied Longitudinal Data Analysis: Modeling Change and Event Occurrence Oxford, UK: Oxford Univ. Press
  94. Sliwinski MJ, Hoffman L, Hofer SM. 2010. Evaluating convergence of within-person change and between-person age differences in age-heterogeneous longitudinal studies. Res. Hum. Dev. 7:145–60
    [Google Scholar]
  95. Smid SC, McNeish D, Miočević M, van de Schoot R. 2020. Bayesian versus frequentist estimation for structural equation models in small sample contexts: a systematic review. Struct. Equ. Model. 27:1131–61
    [Google Scholar]
  96. Snijders TAB, Bosker RJ. 2012. Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling London: Sage, 2nd ed..
  97. Townsend Z, Buckley J, Harada M, Scott MA. 2013. The choice between fixed and random effects. See Scott et al. 2013 73–88
  98. Van Landeghem G, De Fraine B, Van Damme J. 2005. The consequence of ignoring a level of nesting in multilevel analysis: a comment. Multivar. Behav. Res. 40:4423–34
    [Google Scholar]
  99. Vandenberg RJ, Richardson HA. 2019. A primer on multilevel structural modeling: user-friendly guidelines. See Humphrey & LeBreton 2019 449–72
  100. Walters RW. 2015. Mixed-effects location-scale models for conditionally normally distributed repeated-measures data PhD Diss., Univ Nebraska-Lincoln:
  101. Walters RW, Hoffman L, Templin J. 2018. The power to detect and predict individual differences in intra-individual variability using the mixed-effects location-scale model. Multivar. Behav. Res. 53:3360–74
    [Google Scholar]
  102. Wang L, Hamaker E, Bergeman CS 2012. Investigating inter-individual differences in short-term intra-individual variability. Psychol. Methods 17:4567–81
    [Google Scholar]
  103. Wang L, Maxwell SE 2015. On disaggregating between-person and within-person effects with longitudinal data using multilevel models. Psychol. Methods 20:163–83
    [Google Scholar]
  104. Williams DR, Zimprich DR, Rast P. 2019. A Bayesian nonlinear mixed-effects location scale model for learning. Behav. Res. Methods 51:1968–86
    [Google Scholar]
  105. Yaremych HE, Preacher KJ, Hedeker D 2021. Centering categorical predictors in multilevel models: best practices and interpretation. Psychol. Methods In press
    [Google Scholar]
  106. Ye F, Daniel L 2017. The impact of inappropriate modeling of cross-classified data structures on random-slope models. J. Mod. Appl. Stat. Methods 16:2458–84
    [Google Scholar]
  107. Zigler CK, Ye F. 2019. A comparison of multilevel mediation modeling methods: recommendations for applied researchers. Multivar. Behav. Res. 54:3338–59
    [Google Scholar]
  108. Zitzmann S, Lüdtke O, Robitzsch A, Hecht M 2020. On the performance of Bayesian approaches in small samples: a comment on Smid, McNeish, Miocevic, and van de Schoot. Struct. Equ. Model. 28:140–50
    [Google Scholar]
/content/journals/10.1146/annurev-psych-020821-103525
Loading
/content/journals/10.1146/annurev-psych-020821-103525
Loading

Data & Media loading...

Supplemental Material

Supplementary Data

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error