1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Statistics and Its Application
  4. Volume 12, 2025
  5. Article

Annual Review of Statistics and Its Application

Volume 12, 2025

Generalized Additive Models

Abstract

Generalized additive models are generalized linear models in which the linear predictor includes a sum of smooth functions of covariates, where the shape of the functions is to be estimated. They have also been generalized beyond the original generalized linear model setting to distributions outside the exponential family and to situations in which multiple parameters of the response distribution may depend on sums of smooth functions of covariates. The widely used computational and inferential framework in which the smooth terms are represented as latent Gaussian processes, splines, or Gaussian random effects is reviewed, paying particular attention to the case in which computational and theoretical tractability is obtained by prior rank reduction of the model terms. An empirical Bayes approach is taken, and its relatively good frequentist performance discussed, along with some more overtly frequentist approaches to model selection. Estimation of the degree of smoothness of component functions via cross validation or marginal likelihood is covered, alongside the computational strategies required in practice, including when data and models are reasonably large. It is briefly shown how the framework extends easily to location-scale modeling, and, with more effort, to techniques such as quantile regression. Also covered are the main classes of smooths of multiple covariates that may be included in models: isotropic splines and tensor product smooth interaction terms.

Keyword(s): cross validation, marginal likelihood, regression, smooth, smoothing parameter
Loading

Article metrics loading...

/content/journals/10.1146/annurev-statistics-112723-034249
2025-03-07
2025-04-17
Loading full text...

Full text loading...

/deliver/fulltext/statistics/12/1/annurev-statistics-112723-034249.html?itemId=/content/journals/10.1146/annurev-statistics-112723-034249&mimeType=html&fmt=ahah

Literature Cited

  1. Akaike H. 1973.. Information theory and an extension of the maximum likelihood principle. . In Proceedings of the 2nd International Symposium on Information Theory, ed. B Petran, F Csaaki , pp. 267–81. Budapest:: Akadeemiai Kiadi
     [Google Scholar]
  2. Bissiri PG, Holmes C, Walker SG. 2016.. A general framework for updating belief distributions. . J. R. Stat. Soc. Ser. B 78:(5):1103–30
     [Crossref] [Google Scholar]
  3. Breslow NE, Clayton DG. 1993.. Approximate inference in generalized linear mixed models. . J. Am. Stat. Assoc. 88::9–25
     [Crossref] [Google Scholar]
  4. Chouldechova A, Hastie T. 2015.. Generalized additive model selection. . arXiv:1506.03850 [stat.ML]
  5. Claeskens G, Krivobokova T, Opsomer JD. 2009.. Asymptotic properties of penalized spline estimators. . Biometrika 96:(3):529–44
     [Crossref] [Google Scholar]
  6. Cox D. 1972.. Regression models and life tables (with discussion). . J. R. Stat. Soc. Ser. B 34:(2):187–220
     [Crossref] [Google Scholar]
  7. Crainiceanu C, Ruppert D, Claeskens G, Wand MP. 2005.. Exact likelihood ratio tests for penalised splines. . Biometrika 92:(1):91–103
     [Crossref] [Google Scholar]
  8. Craven P, Wahba G. 1979.. Smoothing noisy data with spline functions. . Numer. Math. 31:(5):377–403
     [Google Scholar]
  9. Davies RB. 1973.. Numerical inversion of a characteristic function. . Biometrika 60:(2):415–17
     [Crossref] [Google Scholar]
  10. Davies RB. 1980.. Algorithm AS 155: the distribution of a linear combination of χ2 random variables. . J. R. Stat. Soc. Ser. C 29:(3):323–33
     [Google Scholar]
  11. de Boor C. 2001.. A Practical Guide to Splines. New York:: Springer. Rev. ed.
     [Google Scholar]
  12. Duchon J. 1977.. Splines minimizing rotation-invariant semi-norms in Solobev spaces. . In Construction Theory of Functions of Several Variables, ed. W Schemp, K Zeller , pp. 85–100. Berlin:: Springer
     [Google Scholar]
  13. Dupont E, Wood SN, Augustin NH. 2022.. Spatial+: a novel approach to spatial confounding. . Biometrics 78:(4):1279–90
     [Crossref] [Google Scholar]
  14. Eilers PHC, Marx BD. 1996.. Flexible smoothing with B-splines and penalties. . Stat. Sci. 11:(2):89–121
     [Crossref] [Google Scholar]
  15. Ettinger B, Perotto S, Sangalli LM. 2016.. Spatial regression models over two-dimensional manifolds. . Biometrika 103:(1):71–88
     [Crossref] [Google Scholar]
  16. Fahrmeir L, Lang S. 2001.. Bayesian inference for generalized additive mixed models based on Markov random field priors. . J. R. Stat. Soc. Ser. C 50:(2):201–20
     [Crossref] [Google Scholar]
  17. Fasiolo M, Wood SN, Zaffran M, Nedellec R, Goude Y. 2021.. Fast calibrated additive quantile regression. . J. Am. Stat. Assoc. 116:(535):1402–12
     [Crossref] [Google Scholar]
  18. Fellner WH. 1986.. Robust estimation of variance components. . Technometrics 28:(1):51–60
     [Crossref] [Google Scholar]
  19. Fletcher D. 2012.. Estimating overdispersion when fitting a generalized linear model to sparse data. . Biometrika 99:(1):230–37
     [Crossref] [Google Scholar]
  20. Golub GH, Heath M, Wahba G. 1979.. Generalized cross validation as a method for choosing a good ridge parameter. . Technometrics 21:(2):215–23
     [Crossref] [Google Scholar]
  21. Golub GH, van Loan CF. 2013.. Matrix Computations. Baltimore, MD:: Johns Hopkins Univ. Press. , 4th ed..
     [Google Scholar]
  22. Greven S, Crainiceanu CM, Küchenhoff H, Peters A. 2008.. Restricted likelihood ratio testing for zero variance components in linear mixed models. . J. Comput. Graph. Stat. 17:(4):870–91
     [Crossref] [Google Scholar]
  23. Greven S, Kneib T. 2010.. On the behaviour of marginal and conditional AIC in linear mixed models. . Biometrika 97:(4):773–89
     [Crossref] [Google Scholar]
  24. Gu C. 1992.. Cross-validating non-Gaussian data. . J. Comput. Graph. Stat. 1:(2):169–79
     [Crossref] [Google Scholar]
  25. Gu C, Wahba G. 1991.. Minimizing GCV/GML scores with multiple smoothing parameters via the Newton method. . SIAM J. Sci. Stat. Comput. 12:(2):383–98
     [Crossref] [Google Scholar]
  26. Hastie T, Tibshirani R. 1986.. Generalized additive models (with discussion). . Stat. Sci. 1:(3):297–318
     [Google Scholar]
  27. Hastie T, Tibshirani R. 1990.. Generalized Additive Models. Boca Raton, FL:: Chapman & Hall
     [Google Scholar]
  28. Hastie T, Tibshirani R. 1993.. Varying-coefficient models (with discussion). . J. R. Stat. Soc. Ser. B 55:(4):757–96
     [Crossref] [Google Scholar]
  29. He VX, Wand MP. 2024.. Bayesian generalized additive model selection including a fast variational option. . AStA Adv. Stat. Anal. 108::63968
     [Crossref] [Google Scholar]
  30. Heckman NE, Ramsay JO. 2000.. Penalized regression with model-based penalties. . Can. J. Stat. 28:(2):241–58
     [Crossref] [Google Scholar]
  31. Hodges JS, Reich BJ. 2010.. Adding spatially-correlated errors can mess up the fixed effect you love. . Am. Stat. 64:(4):325–34
     [Crossref] [Google Scholar]
  32. Kammann EE, Wand MP. 2003.. Geoadditive models. . J. R. Stat. Soc. Ser. C 52:(1):1–18
     [Crossref] [Google Scholar]
  33. Kimeldorf GS, Wahba G. 1970.. A correspondence between Bayesian estimation on stochastic processes and smoothing by splines. . Ann. Math. Stat. 41:(2):495–502
     [Crossref] [Google Scholar]
  34. Kuonen D. 1999.. Saddlepoint approximations for distributions of quadratic forms in normal variables. . Biometrika 86:(4):929–35
     [Crossref] [Google Scholar]
  35. Li Z, Wood SN. 2020.. Faster model matrix crossproducts for large generalized linear models with discretized covariates. . Stat. Comput. 30:(1):19–25
     [Crossref] [Google Scholar]
  36. Lindgren F, Rue H, Lindström J. 2011.. An explicit link between Gaussian fields and Gaussian Markov random fields: the stochastic partial differential equation approach (with discussion). . J. R. Stat. Soc. Ser. B 73:(4):423–98
     [Crossref] [Google Scholar]
  37. Marques I, Kneib T, Klein N. 2023.. Mitigating spatial confounding by explicitly correlating Gaussian random fields. . Environmetrics 34:(4):e2801
     [Crossref] [Google Scholar]
  38. Marra G, Wood SN. 2012.. Coverage properties of confidence intervals for generalized additive model components. . Scand. J. Stat. 39:(1):53–74
     [Crossref] [Google Scholar]
  39. Marx BD, Eilers PH. 1999.. Generalized linear regression on sampled signals and curves: a P-spline approach. . Technometrics 41:(1):1–13
     [Crossref] [Google Scholar]
  40. Mayr A, Fenske N, Hofner B, Kneib T, Schmid M. 2012.. Generalized additive models for location, scale and shape for high dimensional data—a flexible approach based on boosting. . J. R. Stat. Soc. Ser. C 61:(3):403–27
     [Crossref] [Google Scholar]
  41. McCullagh P, Nelder JA. 1989.. Generalized Linear Models. London:: Chapman & Hall. , 2nd ed..
     [Google Scholar]
  42. Nelder JA, Wedderburn RWM. 1972.. Generalized linear models. . J. R. Stat. Soc. Ser. A 135:(3):370–84
     [Crossref] [Google Scholar]
  43. Nychka D. 1988.. Bayesian confidence intervals for smoothing splines. . J. Am. Stat. Assoc. 83:(404):1134–43
     [Crossref] [Google Scholar]
  44. O'Sullivan FB, Yandall B, Raynor W. 1986.. Automatic smoothing of regression functions in generalized linear models. . J. Am. Stat. Assoc. 81:(393):96–103
     [Crossref] [Google Scholar]
  45. Peng RD, Welty LJ. 2004.. The NMMAPSdata package. . R News 4:(2):10–14
     [Google Scholar]
  46. Ramsay JO, Hooker G, Campbell D, Cao J. 2007.. Parameter estimation for differential equations: a generalized smoothing approach (with discussion). . J. R. Stat. Soc. Ser. B 69:(5):741–96
     [Crossref] [Google Scholar]
  47. Ramsay T. 2002.. Spline smoothing over difficult regions. . J. R. Stat. Soc. Ser. B 64:(2):307–19
     [Crossref] [Google Scholar]
  48. Rigby R, Stasinopoulos DM. 2005.. Generalized additive models for location, scale and shape (with discussion). . J. R. Stat. Soc. Ser. C 54:(3):507–54
     [Crossref] [Google Scholar]
  49. Rue H, Held L. 2005.. Gaussian Markov Random Fields: Theory and Applications. Boca Raton, FL:: Chapman and Hall/CRC
     [Google Scholar]
  50. Rue H, Martino S, Chopin N. 2009.. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations (with discussion). . J. R. Stat. Soc. Ser. B 71:(2):319–92
     [Crossref] [Google Scholar]
  51. Ruppert D, Wand MP, Carroll RJ. 2003.. Semiparametric Regression. Cambridge, UK:: Cambridge Univ. Press
     [Google Scholar]
  52. Säfken B, Kneib T, van Waveren CS, Greven S. 2014.. A unifying approach to the estimation of the conditional Akaike information in generalized linear mixed models. . Electron. J. Stat. 8::201–25
     [Google Scholar]
  53. Sangalli LM. 2021.. Spatial regression with partial differential equation regularisation. . Int. Stat. Rev. 89:(3):505–31
     [Crossref] [Google Scholar]
  54. Schall R. 1991.. Estimation in generalized linear models with random effects. . Biometrika 78:(4):719–27
     [Crossref] [Google Scholar]
  55. Schmid M, Hothorn T. 2008.. Boosting additive models using component-wise P-splines. . Comput. Stat. Data Anal. 53:(2):298–311
     [Crossref] [Google Scholar]
  56. Silverman BW. 1985.. Some aspects of the spline smoothing approach to non-parametric regression curve fitting (with discussion). . J. R. Stat. Soc. Ser. B 47:(1):152
     [Crossref] [Google Scholar]
  57. Stasinopoulos MD, Rigby RA, Heller GZ, Voudouris V, De Bastiani F. 2017.. Flexible Regression and Smoothing: Using GAMLSS in R. Boca Raton, FL:: Chapman and Hall/CRC
     [Google Scholar]
  58. Umlauf N, Adler D, Kneib T, Lang S, Zeileis A. 2015.. Structured additive regression models: an R interface to BayesX. . J. Stat. Softw. 63:(21):1–46
     [Crossref] [Google Scholar]
  59. van der Vorst HA. 2003.. Iterative Krylov Methods for Large Linear Systems. Cambridge, UK:: Cambridge Univ. Press
     [Google Scholar]
  60. Wahba G. 1983.. Bayesian confidence intervals for the cross validated smoothing spline. . J. R. Stat. Soc. Ser. B 45:(1):133–50
     [Crossref] [Google Scholar]
  61. Wahba G. 1985.. A comparison of GCV and GML for choosing the smoothing parameter in the generalized spline smoothing problem. . Ann. Stat. 13:(4):1378–402
     [Crossref] [Google Scholar]
  62. Wahba G. 1990.. Spline Models for Observational Data. Philadelphia:: SIAM
     [Google Scholar]
  63. Wendelberger J. 1981.. Smoothing noisy data with multidimensional splines and generalized cross validation. PhD Thesis , Dep. Stat., Univ. Wis., Madison, WI:
     [Google Scholar]
  64. Wood SN. 2000.. Modelling and smoothing parameter estimation with multiple quadratic penalties. . J. R. Stat. Soc. Ser. B 62:(2):413–28
     [Crossref] [Google Scholar]
  65. Wood SN. 2003.. Thin plate regression splines. . J. R. Stat. Soc. Ser. B 65::95–114
     [Crossref] [Google Scholar]
  66. Wood SN. 2011.. Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. . J. R. Stat. Soc. Ser. B 73:(1):3–36
     [Crossref] [Google Scholar]
  67. Wood SN. 2013a.. A simple test for random effects in regression models. . Biometrika 100:(4):1005–10
     [Crossref] [Google Scholar]
  68. Wood SN. 2013b.. On p-values for smooth components of an extended generalized additive model. . Biometrika 100:(1):221–28
     [Crossref] [Google Scholar]
  69. Wood SN. 2017a.. Generalized Additive Models: An Introduction with R. Boca Raton, FL:: Chapman & Hall/CRC. , 2nd ed..
     [Google Scholar]
  70. Wood SN. 2017b.. P-splines with derivative based penalties and tensor product smoothing of unevenly distributed data. . Stat. Comput. 27:(4):985–89
     [Crossref] [Google Scholar]
  71. Wood SN. 2020.. Inference and computation with generalized additive models and their extensions. . Test 29:(2):307–39
     [Crossref] [Google Scholar]
  72. Wood SN, Bravington MV, Hedley SL. 2008.. Soap film smoothing. . J. R. Stat. Soc. Ser. B 70:(5):931–55
     [Crossref] [Google Scholar]
  73. Wood SN, Fasiolo M. 2017.. A generalized Fellner–Schall method for smoothing parameter optimization with application to Tweedie location, scale and shape models. . Biometrics 73:(4):1071–81
     [Crossref] [Google Scholar]
  74. Wood SN, Goude Y, Shaw S. 2015.. Generalized additive models for large data sets. . J. R. Stat. Soc. Ser. C 64:(1):139–55
     [Crossref] [Google Scholar]
  75. Wood SN, Li Z, Shaddick G, Augustin NH. 2017.. Generalized additive models for gigadata: modelling the UK black smoke network daily data. . J. Am. Stat. Assoc. 112:(519):1199–210
     [Crossref] [Google Scholar]
  76. Wood SN, Pya N, Säfken B. 2016.. Smoothing parameter and model selection for general smooth models (with discussion). . J. Am. Stat. Assoc. 111::1548–75
     [Crossref] [Google Scholar]
  77. Wood SN, Wit EC. 2021.. Was R < 1 before the English lockdowns? On modelling mechanistic detail, causality and inference about Covid-19. . PLOS ONE 16:(9):e0257455
     [Crossref] [Google Scholar]
  78. Yee TW. 2015.. Vector Generalized Linear and Additive Models: With an Implementation in R. New York:: Springer
     [Google Scholar]
  79. Yee TW, Wild C. 1996.. Vector generalized additive models. . J. R. Stat. Soc. Ser. B 58:(3):481–93
     [Crossref] [Google Scholar]
/content/journals/10.1146/annurev-statistics-112723-034249
Loading
Generalized Additive Models
Annual Review of Statistics and Its Application 12, 497 (2025); https://doi.org/10.1146/annurev-statistics-112723-034249
/content/journals/10.1146/annurev-statistics-112723-034249
/content/journals/10.1146/annurev-statistics-112723-034249
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error