1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Statistics and Its Application
  4. Volume 7, 2020
  5. Article

Annual Review of Statistics and Its Application

Volume 7, 2020

Bayesian Additive Regression Trees: A Review and Look Forward

Abstract

Bayesian additive regression trees (BART) provides a flexible approach to fitting a variety of regression models while avoiding strong parametric assumptions. The sum-of-trees model is embedded in a Bayesian inferential framework to support uncertainty quantification and provide a principled approach to regularization through prior specification. This article presents the basic approach and discusses further development of the original algorithm that supports a variety of data structures and assumptions. We describe augmentations of the prior specification to accommodate higher dimensional data and smoother functions. Recent theoretical developments provide justifications for the performance observed in simulations and other settings. Use of BART in causal inference provides an additional avenue for extensions and applications. We discuss software options as well as challenges and future directions.

Keyword(s): Bayesian nonparametrics, causal inference, machine learning, regression, regularization
Loading

Article metrics loading...

/content/journals/10.1146/annurev-statistics-031219-041110
2020-03-07
2024-04-26
Loading full text...

Full text loading...

/deliver/fulltext/statistics/7/1/annurev-statistics-031219-041110.html?itemId=/content/journals/10.1146/annurev-statistics-031219-041110&mimeType=html&fmt=ahah

Literature Cited

  1. Albert JH, Chib S. 1993. Bayesian analysis of binary and polychotomous response data. J. Am. Stat. Assoc. 88:669–79
     [Google Scholar]
  2. Bickel P, Kleijn B. 2012. The semiparametric Bernstein–von Mises theorem. Ann. Stat. 40:206–37
     [Google Scholar]
  3. Bonato V, Baladandayuthapani V, Broom BM, Sulman EP, Aldape KD, Do KA 2010. Bayesian ensemble methods for survival prediction in gene expression data. Bioinformatics 27:359–67
     [Google Scholar]
  4. Carnegie N. 2019. Contributions of model features to BART causal inference performance using ACIC 2016 competition data. Stat. Sci. 34:90–93
     [Google Scholar]
  5. Carnegie N, Dorie V, Hill J 2019. Examining treatment effect heterogeneity using BART. Obs. Stud. press
     [Google Scholar]
  6. Carnegie NB, Harada M, Dorie V, Hill J 2015. treatSens: sensitivity analysis for causal inference. R package https://rdrr.io/cran/treatSens/
    [Google Scholar]
  7. Carnegie NB, Harada M, Hill J 2016. Assessing sensitivity to unmeasured confounding using a simulated potential confounder. J. Res. Educ. Eff. 9:395–420
     [Google Scholar]
  8. Chipman HA, George EI, McCulloch RE 1998. Bayesian CART model search. J. Am. Stat. Assoc. 93:935–48
     [Google Scholar]
  9. Chipman HA, George EI, McCulloch R 2007. Bayesian ensemble learning. Advances in Neural Information Processing Systems 19 B Schölkopf, J Platt, T Hoffman 265–72 Cambridge, MA: MIT Press
     [Google Scholar]
  10. Chipman HA, George EI, McCulloch RE 2010. BART: Bayesian additive regression trees. Ann. Appl. Stat. 4:266–98
     [Google Scholar]
  11. Chipman HA, McCulloch R. 2010. BayesTree: Bayesian methods for tree based models. R package https://cran.r-project.org/web/packages/BayesTree/BayesTree.pdf
    [Google Scholar]
  12. Cox DR. 1972. Regression models and life-tables. J. R. Stat. Soc. B 34:187–202
     [Google Scholar]
  13. Dawid AP. 2000. Causal inference without counterfactuals. J. Am. Stat. Assoc. 95:407–24
     [Google Scholar]
  14. Denison DGT, Mallick BK, Smith AFM 1998. A Bayesian CART algorithm. Biometrika 85:363–77
     [Google Scholar]
  15. Dorie V, Carnegie NB, Harada M, Hill J 2016. A flexible, interpretable framework for assessing sensitivity to unmeasured confounding. Stat. Med. 35:3453–70
     [Google Scholar]
  16. Dorie V, Hill J, Shalit U, Scott M, Cervone D 2019. Automated versus do-it-yourself methods for causal inference: lessons learned from a data analysis competition. Stat. Sci. 34:43–68
     [Google Scholar]
  17. Friedman JH. 2001. Greedy function approximation: a gradient boosting machine. Ann. Stat. 29:1189–232
     [Google Scholar]
  18. George E, Laud P, Logan B, McCulloch R, Sparapani R 2018. Fully nonparametric Bayesian additive regression trees. arXiv:1807.00068 [stat.ML]
  19. Goldstein A, Kapelner A, Bleich J, Pitkin E 2015. Peeking inside the black box: visualizing statistical learning with plots of individual conditional expectation. J. Comput. Graph. Stat. 24:44–65
     [Google Scholar]
  20. Green DP, Kern HL. 2012. Modeling heterogeneous treatment effects in survey experiments with Bayesian additive regression trees. Public Opin. Q. 76:491–511
     [Google Scholar]
  21. Greene WH. 1994. Accounting for excess zeros and sample selection in Poisson and negative binomial regression models Work. Pap. EC-94-10 New York Univ:.
  22. Hahn PR, Dorie V, Murray JS 2019. Atlantic Causal Inference Conference (ACIC) data analysis challenge 2017. arXiv:1905.09515 [stat.ME]
  23. Hahn PR, Murray JS, Carvalho C 2017. Bayesian regression tree models for causal inference: regularization, confounding, and heterogeneous effects. arXiv:1706.09523 [stat.ME]
  24. Hastie T, Tibshirani R. 1987. Generalized additive models: some applications. J. Am. Stat. Assoc. 82:371–86
     [Google Scholar]
  25. Hastie T, Tibshirani R. 2000. Bayesian backfitting (with comments and a rejoinder by the authors). Stat. Sci. 15:196–223
     [Google Scholar]
  26. He J, Yalov S, Hahn PR 2019. XBART: Accelerated Bayesian additive regression trees. PMLR 89:1130–38
     [Google Scholar]
  27. Hill J. 2011. Bayesian nonparametric modeling for causal inference. J. Comput. Graph. Stat. 20:217–40
     [Google Scholar]
  28. Hill J, Su YS. 2013. Assessing lack of common support in causal inference using Bayesian nonparametrics: implications for evaluating the effect of breastfeeding on children's cognitive outcomes. Ann. Appl. Stat. 7:1386–420
     [Google Scholar]
  29. Hill J, Weiss C, Zhai F 2011. Challenges with propensity score strategies in a high-dimensional setting and a potential alternative. Multivar. Behav. Res. 46:477–513
     [Google Scholar]
  30. Ibrahim JG, Chen MH, Sinha D 2005. Bayesian survival analysis. Wiley StatsRef: Statistics Reference Online N Balakrishnan, T Colton, B Everitt, W Piegorsch, F Ruggeri, JL Teugels. https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118445112.stat06003
    [Google Scholar]
  31. Kern HL, Stuart EA, Hill JL, Green DP 2016. Assessing methods for generalizing experimental impact estimates to target samples. J. Res. Educ. Eff. 9:103–27
     [Google Scholar]
  32. Kim H, Loh WY, Shih YS, Chaudhuri P 2007. Visualizable and interpretable regression models with good prediction power. IIE Trans 39:565–79
     [Google Scholar]
  33. Lakshminarayanan B, Roy DM, Teh YW, Unit G 2015. Particle Gibbs for Bayesian additive regression trees. PMLR 38:553–61
     [Google Scholar]
  34. Lambert D. 1992. Zero-inflated Poisson regression, with an application to defects in manufacturing. Technometrics 34:1–14
     [Google Scholar]
  35. LeDell E. 2016. h2oensemble: H2o ensemble learning. R package https://github.com/h2oai/h2o-3/tree/master/h2o-r/ensemble
    [Google Scholar]
  36. Linero AR. 2018. Bayesian regression trees for high-dimensional prediction and variable selection. J. Am. Stat. Assoc. 113:626–36
     [Google Scholar]
  37. Linero AR, Sinha D, Lipsitz SR 2018. Semiparametric mixed-scale models using shared Bayesian forests. arXiv:1809.08521 [stat.ME]
  38. Linero AR, Yang Y. 2018. Bayesian regression tree ensembles that adapt to smoothness and sparsity. J. R. Stat. Soc. B 80:1087–110
     [Google Scholar]
  39. Liu Y, Ročková V, Wang Y 2019. Variable selection with ABC Bayesian forests. arXiv:1806.02304 [stat.ME]
  40. Logan BR, Sparapani R, McCulloch RE, Laud PW 2017. Decision making and uncertainty quantification for individualized treatments using Bayesian additive regression trees. Stat. Methods Med. Res. 28:1079–93
     [Google Scholar]
  41. Murray JS. 2017. Log-linear Bayesian additive regression trees for categorical and count responses. arXiv:1701.01503 [stat.ME]
  42. Polley E, LeDell E, Kennedy C, van der Laan M 2016. SuperLearner: super learner prediction. R package https://CRAN.R-project.org/package=SuperLearner
    [Google Scholar]
  43. Pratola MT. 2016. Efficient Metropolis-Hastings proposal mechanisms for Bayesian regression tree models. Bayesian Anal 11:885–911
     [Google Scholar]
  44. Pratola MT, Chipman HA, Gattiker JR, Higdon DM, McCulloch R, Rust WN 2014. Parallel Bayesian additive regression trees. J. Comput. Graph. Stat. 23:830–52
     [Google Scholar]
  45. Pratola MT, Chipman HA, George EI, McCulloch RE 2017. Heteroscedastic BART using multiplicative regression trees. arXiv:1709.07542 [stat.ME]
  46. Rockova V, van der Pas S 2017. Posterior concentration for Bayesian regression trees and their ensembles. arXiv:1708.08734 [math.ST]
  47. Rubin DB. 1978. Bayesian inference for causal effects: the role of randomization. Ann. Stat. 6:34–58
     [Google Scholar]
  48. Sivaganesan S, Müller P, Huang B 2017. Subgroup finding via Bayesian additive regression trees. Stat. Med. 36:2391–403
     [Google Scholar]
  49. Sparapani RA, Logan BR, McCulloch RE, Laud PW 2016. Nonparametric survival analysis using Bayesian additive regression trees (BART). Stat. Med. 35:162741–53
     [Google Scholar]
  50. Starling JE, Murray JS, Carvalho CM, Bukowski R, Scott JG 2019. BART with targeted smoothing: an analysis of patient-specific stillbirth risk. arXiv:1805.07656 [stat.ME]
  51. Van Der Vaart AW, Wellner JA 1996. Weak Convergence New York: Springer
  52. Wendling T, Jung K, Callahan A, Schuler A, Shah N, Gallego B 2018. Comparing methods for estimation of heterogeneous treatment effects using observational data from health care databases. Stat. Med. 37:3309–24
     [Google Scholar]
  53. Wu Y, Tjelmeland H, West M 2007. Bayesian CART: prior specification and posterior simulation. J. Comput. Graph. Stat. 16:44–66
     [Google Scholar]
  54. Yang Y, Tokdar ST. 2015. Minimax-optimal nonparametric regression in high dimensions. Ann. Stat. 43:652–74
     [Google Scholar]
  55. Zeldow B, Re VL III, Roy J 2018. A semiparametric modeling approach using Bayesian additive regression trees with an application to evaluate heterogeneous treatment effects. arXiv:1806.04200 [stat.AP]
/content/journals/10.1146/annurev-statistics-031219-041110
Loading
Bayesian Additive Regression Trees: A Review and Look Forward
Annual Review of Statistics and Its Application 7, 251 (2020); https://doi.org/10.1146/annurev-statistics-031219-041110
/content/journals/10.1146/annurev-statistics-031219-041110
/content/journals/10.1146/annurev-statistics-031219-041110
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error