Machine learning can be used to make sense of healthcare data. Probabilistic machine learning models help provide a complete picture of observed data in healthcare. In this review, we examine how probabilistic machine learning can advance healthcare. We consider challenges in the predictive model building pipeline where probabilistic models can be beneficial, including calibration and missing data. Beyond predictive models, we also investigate the utility of probabilistic machine learning models in phenotyping, in generative models for clinical use cases, and in reinforcement learning.


Article metrics loading...

Loading full text...

Full text loading...


Literature Cited

  1. 1. 
    Steinhubl SR, Muse ED, Topol EJ. 2013. Can mobile health technologies transform health care?. JAMA 310:2395–96
    [Google Scholar]
  2. 2. 
    Poplin R, Varadarajan AV, Blumer K, Liu Y, McConnell MV et al. 2018. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning. Nat. Biomed. Eng. 2:158–64
    [Google Scholar]
  3. 3. 
    Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. 2018. An interpretable machine learning model for accurate prediction of sepsis in the ICU. Crit. Care Med. 46:54753
    [Google Scholar]
  4. 4. 
    Pivovarov R, Elhadad N. 2015. Automated methods for the summarization of electronic health records. J. Am. Med. Inform. Assoc. 22:938–47
    [Google Scholar]
  5. 5. 
    Segler MH, Kogej T, Tyrchan C, Waller MP. 2018. Generating focused molecule libraries for drug discovery with recurrent neural networks. ACS Central Sci. 4:120–31
    [Google Scholar]
  6. 6. 
    Fox I, Lee J, Busui R, Wiens J. 2020. Deep reinforcement learning for closed-loop blood glucose control. arXiv:2009.09051 [cs.LG]
  7. 7. 
    Komorowski M, Gordon A, Celi L, Faisal A. 2016. A Markov decision process to suggest optimal treatment of severe infections in intensive care Paper presented at the Neural Information Processing Systems Workshop on Machine Learning for Health Barcelona, Spain: Dec. 8
  8. 8. 
    Dillon JV, Langmore I, Tran D, Brevdo E, Vasudevan S et al. 2017. Tensorflow distributions. arXiv:1711.10604 [cs.LG]
  9. 9. 
    Bingham E, Chen JP, Jankowiak M, Obermeyer F, Pradhan N et al. 2019. Pyro: deep universal probabilistic programming. J. Mach. Learn. Res. 20:973–78
    [Google Scholar]
  10. 10. 
    Miscouridou X, Perotte A, Elhadad N, Ranganath R. 2018. Deep survival analysis: nonparametrics and missingness. Proc. Mach. Learn. Res. 85:244–56
    [Google Scholar]
  11. 11. 
    Che Z, Purushotham S, Cho K, Sontag D, Liu Y. 2018. Recurrent neural networks for multivariate time series with missing values. Sci. Rep. 8:6085
    [Google Scholar]
  12. 12. 
    Ranganath R, Perotte A, Elhadad N, Blei D. 2016. Deep survival analysis. Proc. Mach. Learn. Res. 56:101–14
    [Google Scholar]
  13. 13. 
    White IR, Royston P, Wood AM. 2011. Multiple imputation using chained equations: issues and guidance for practice. Stat. Med. 30:377–99
    [Google Scholar]
  14. 14. 
    Little RJ, Rubin DB. 2019. Statistical Analysis with Missing Data New York: Wiley. , 3rd ed..
  15. 15. 
    Zhao Y, Udell M 2019. Missing value imputation for mixed data through Gaussian copula. arXiv:1910.12845 [stat.ME]
  16. 16. 
    Mohan K, Pearl J. 2018. Graphical models for processing missing data. arXiv:1801.03583 [stat.ME]
  17. 17. 
    Sadinle M, Reiter JP. 2019. Sequentially additive nonignorable missing data modelling using auxiliary marginal information. Biometrika 106:889–911
    [Google Scholar]
  18. 18. 
    Sun B, Liu L, Miao W, Wirth K, Robins J, Tchetgen EJT. 2018. Semiparametric estimation with data missing not at random using an instrumental variable. Stat. Sin. 28:1965–83
    [Google Scholar]
  19. 19. 
    Hyde J. 1977. Testing survival under right censoring and left truncation. Biometrika 64:2225–30
    [Google Scholar]
  20. 20. 
    Lee C, Zame WR, Yoon J, van der Schaar M. 2018. DeepHit: a deep learning approach to survival analysis with competing risks. Proceedings of the 32nd AAAI Conference on Artificial Intelligence2314–21 Palo Alto, CA: AAAI
    [Google Scholar]
  21. 21. 
    Katzman JL, Shaham U, Cloninger A, Bates J, Jiang T, Kluger Y. 2018. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC Med. Res. Methodol. 18:24
    [Google Scholar]
  22. 22. 
    Brier GW. 1950. Verification of forecasts expressed in terms of probability. Mon. Weather Rev. 78:1–3
    [Google Scholar]
  23. 23. 
    Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS et al. 2008. Random survival forests. Ann. Appl. Stat. 2:841–60
    [Google Scholar]
  24. 24. 
    Nelson W. 1969. Hazard plotting for incomplete failure data. J. Q. Technol. 1:27–52
    [Google Scholar]
  25. 25. 
    Leung KM, Elashoff RM, Afifi AA. 1997. Censoring issues in survival analysis. Annu. Rev. Public Health 18:83–104
    [Google Scholar]
  26. 26. 
    Cho H, Jewell NP, Kosorok MR. 2019. Interval censored recursive forests. arXiv:1912.09983 [stat.ME]
  27. 27. 
    Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB 1998. Prediction of coronary heart disease using risk factor categories. Circulation 97:1837–47
    [Google Scholar]
  28. 28. 
    Guo C, Pleiss G, Sun Y, Weinberger KQ. 2017. On calibration of modern neural networks. Proceedings of the 34th International Conference on Machine Learning D Precup, YW Teh 1321–30 New York: Assoc. Comput. Mach.
    [Google Scholar]
  29. 29. 
    Pfohl SR, Duan T, Ding DY, Shah NH. 2019. Counterfactual reasoning for fair clinical risk prediction. Proc. Mach. Learn. Res. 106:325–58
    [Google Scholar]
  30. 30. 
    Van Calster B, McLernon DJ, Van Smeden M, Wynants L, Steyerberg EW. 2019. Calibration: the Achilles heel of predictive analytics. BMC Med. 17:230
    [Google Scholar]
  31. 31. 
    Naeini MP, Cooper G, Hauskrecht M. 2015. Obtaining well calibrated probabilities using Bayesian binning. Proceedings of the 29th AAAI Conference on Artificial Intelligence2901–7 Palo Alto, CA: AAAI
    [Google Scholar]
  32. 32. 
    Platt JC 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers AJ Smola, P Bartlett, B Schölkopf, D Schuurmans 61–74 Cambridge, MA: MIT Press
    [Google Scholar]
  33. 33. 
    Chakravarti N. 1989. Isotonic median regression: a linear programming approach. Math. Oper. Res. 14:303–8
    [Google Scholar]
  34. 34. 
    Zadrozny B, Elkan C. 2002. Transforming classifier scores into accurate multiclass probability estimates. Proceedings of the 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining694–99 New York: Assoc. Comput. Mach.
    [Google Scholar]
  35. 35. 
    Zadrozny B. 2002. Reducing multiclass to binary by coupling probability estimates. Advances in Neural Information Processing Systems 14 T Dietterich, S Becker, Z Ghahramani 1041–48 https://papers.nips.cc/paper/2001/file/abdbeb4d8dbe30df8430a8394b7218ef-Paper.pdf
    [Google Scholar]
  36. 36. 
    Kumar A, Liang PS, Ma T 2019. Verified uncertainty calibration. Advances in Neural Information Processing Systems 32 H Wallach, H Larochelle, A Beygelzimer, F d'Alché-Buc, E Fox, R Garnett 3792–803 https://papers.nips.cc/paper/2019/file/f8c0c968632845cd133308b1a494967f-Paper.pdf
    [Google Scholar]
  37. 37. 
    Goldstein M, Han X, Puli A, Perotte A, Ranganath R. 2020. X-CAL: explicit calibration for survival analysis. Advances in Neural Information Processing Systems 33 H Larochelle, M Ranzato, R Hadsell, MF Balcan, H Lin 18296–307 https://papers.nips.cc/paper/2020/file/d4a93297083a23cc099f7bd6a8621131-Paper.pdf
    [Google Scholar]
  38. 38. 
    Raghu M, Blumer K, Sayres R, Obermeyer Z, Kleinberg B et al. 2019. Direct uncertainty prediction for medical second opinions. Proc. Mach. Learn. Res. 97:5281–90
    [Google Scholar]
  39. 39. 
    De Bruijn NG. 1981. Asymptotic Methods in Analysis New York: Dover
  40. 40. 
    Efron B, Gong G. 1983. A leisurely look at the bootstrap, the jackknife, and cross-validation. Am. Stat. 37:36–48
    [Google Scholar]
  41. 41. 
    Lakshminarayanan B, Pritzel A, Blundell C. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems 30 U von Luxburg, I Guyon, S Bengio, H Wallach, R Fergus et al.6402–13 https://papers.nips.cc/paper/2017/file/9ef2ed4b7fd2c810847ffa5fa85bce38-Paper.pdf
    [Google Scholar]
  42. 42. 
    Gal Y. 2016. Uncertainty in deep learning PhD Thesis, Dep. Eng., Univ. Cambridge Cambridge, UK:
  43. 43. 
    Gal Y, Ghahramani Z. 2016. Dropout as a Bayesian approximation: representing model uncertainty in deep learning. Proc. Mach. Learn. Res. 48:1050–59
    [Google Scholar]
  44. 44. 
    Grünwald P, Van Ommen T 2017. Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it. Bayesian Anal. 12:1069–103
    [Google Scholar]
  45. 45. 
    Dusenberry MW, Tran D, Choi E, Kemp J, Nixon J et al. 2020. Analyzing the role of model uncertainty for electronic health records. Proceedings of the ACM Conference on Health, Inference, and Learning204–13 New York: Assoc. Comput. Mach.
    [Google Scholar]
  46. 46. 
    Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. 2018. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLOS Med. 15:e1002683
    [Google Scholar]
  47. 47. 
    Nestor B, McDermott M, Boag W, Berner G, Naumann T et al. 2019. Feature robustness in non-stationary health records: caveats to deployable model performance in common clinical machine learning tasks. arXiv:1908.00690 [cs.LG]
  48. 48. 
    Croft JB, Wheaton AG, Liu Y, Xu F, Lu H et al. 2018. Urban-rural county and state differences in chronic obstructive pulmonary disease—United States; 2015. Morb. Mortal. Wkly. Rep. 67:205–11
    [Google Scholar]
  49. 49. 
    Yang P, Dumont G, Ansermino JM. 2006. Adaptive change detection in heart rate trend monitoring in anesthetized children. IEEE Trans. Biomed. Eng. 53:2211–19
    [Google Scholar]
  50. 50. 
    Malladi R, Kalamangalam GP, Aazhang B. 2013. Online Bayesian change point detection algorithms for segmentation of epileptic activity. 2013 Asilomar Conference on Signals, Systems and Computers1833–37 New York: IEEE
    [Google Scholar]
  51. 51. 
    Shimodaira H. 2000. Improving predictive inference under covariate shift by weighting the log-likelihood function. J. Stat. Plan. Inference 90:227–44
    [Google Scholar]
  52. 52. 
    Sugiyama M, Blankertz B, Krauledat M, Dornhege G, Müller KR 2006. Importance-weighted cross-validation for covariate shift. Pattern Recognition: Proceedings of the 28th DAGM Symposium K Franke, K-R Müller, B Nickolay, R Schäfer 354–63 Berlin: Springer-Verlag
    [Google Scholar]
  53. 53. 
    Subbaswamy A, Saria S. 2020. From development to deployment: dataset shift, causality, and shift-stable models in health AI. Biostatistics 21:345–52
    [Google Scholar]
  54. 54. 
    Thorsteinsdottir S, Dickman PW, Landgren O, Blimark C, Hultcrantz M et al. 2018. Dramatically improved survival in multiple myeloma patients in the recent decade: results from a Swedish population-based study. Haematologica 103:e412–15
    [Google Scholar]
  55. 55. 
    Caruana R, Lou Y, Gehrke J, Koch P, Sturm M, Elhadad N. 2015. Intelligible models for healthcare: predicting pneumonia risk and hospital 30-day readmission. Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining1721–30 New York: Assoc. Comput. Mach.
    [Google Scholar]
  56. 56. 
    Schulam P, Saria S. 2017. Reliable decision support using counterfactual models. Advances in Neural Information Processing Systems 30 U von Luxburg, I Guyon, S Bengio, H Wallach, R Fergus et al.1697–708 https://papers.nips.cc/paper/2017/file/299a23a2291e2126b91d54f3601ec162-Paper.pdf
    [Google Scholar]
  57. 57. 
    Perdomo JC, Zrnic T, Mendler-Dünner C, Hardt M. 2020. Performative prediction. arXiv:2002.06673 [cs.LG]
  58. 58. 
    Bottou L, Peters J, Quiñonero-Candela J, Charles DX, Chickering DM et al. 2013. Counterfactual reasoning and learning systems: the example of computational advertising. J. Mach. Learn. Res. 14:3207–60
    [Google Scholar]
  59. 59. 
    Milli S, Miller J, Dragan AD, Hardt M. 2019. The social cost of strategic classification. Proceedings of the Conference on Fairness, Accountability, and Transparency230–39 New York: Assoc. Comput. Mach.
    [Google Scholar]
  60. 60. 
    Adamson AS, Smith A. 2018. Machine learning and health care disparities in dermatology. JAMA Dermatol. 154:1247–48
    [Google Scholar]
  61. 61. 
    Obermeyer Z, Powers B, Vogeli C, Mullainathan S. 2019. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366:447–53
    [Google Scholar]
  62. 62. 
    Rajkomar A, Hardt M, Howell MD, Corrado G, Chin MH. 2018. Ensuring fairness in machine learning to advance health equity. Ann. Intern. Med. 169:866–72
    [Google Scholar]
  63. 63. 
    Hardt M, Price E, Srebro N 2016. Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems 29 D Lee, M Sugiyama, U von Luxburg, I Guyon, R Garnett 3315–23 https://papers.nips.cc/paper/2016/file/9d2682367c3935defcb1f9e247a97c0d-Paper.pdf
    [Google Scholar]
  64. 64. 
    Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R. 2012. Fairness through awareness. Proceedings of the 3rd Innovations in Theoretical Computer Science Conference214–26 New York: Assoc. Comput. Mach.
    [Google Scholar]
  65. 65. 
    Zemel R, Wu Y, Swersky K, Pitassi T, Dwork C. 2013. Learning fair representations. Proc. Mach. Learn. Conf. 28:3325–33
    [Google Scholar]
  66. 66. 
    Bolukbasi T, Chang KW, Zou JY, Saligrama V, Kalai AT. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. Advances in Neural Information Processing Systems 29 D Lee, M Sugiyama, U von Luxburg, I Guyon, R Garnett 4349–57 https://proceedings.neurips.cc/paper/2016/file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf
    [Google Scholar]
  67. 67. 
    Chouldechova A. 2017. Fair prediction with disparate impact: a study of bias in recidivism prediction instruments. Big Data 5:153–63
    [Google Scholar]
  68. 68. 
    Kleinberg J, Mullainathan S, Raghavan M. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv:1609.05807 [cs.LG]
  69. 69. 
    Singh H, Singh R, Mhasawade V, Chunara R. 2019. Fair predictors under distribution shift. arXiv:1911.00677 [cs.LG]
  70. 70. 
    Pleiss G, Raghavan M, Wu F, Kleinberg J, Weinberger KQ. 2017. On fairness and calibration. Advances in Neural Information Processing Systems 30 U von Luxburg, I Guyon, S Bengio, H Wallach, R Fergus et al.5680–89 https://papers.nips.cc/paper/2017/file/b8b9c74ac526fffbeb2d39ab038d1cd7-Paper.pdf
    [Google Scholar]
  71. 71. 
    Simoiu C, Corbett-Davies S, Goel S et al. 2017. The problem of infra-marginality in outcome tests for discrimination. Ann. Appl. Stat. 11:1193–216
    [Google Scholar]
  72. 72. 
    Hébert-Johnson Ú, Kim M, Reingold O, Rothblum G. 2018. Multicalibration: calibration for the (computationally-identifiable) masses. Proc. Mach. Learn. Res. 80:1939–48
    [Google Scholar]
  73. 73. 
    Zink A, Rose S. 2020. Fair regression for health care spending. Biometrics 76:3973–82
    [Google Scholar]
  74. 74. 
    Bechavod Y, Ligett K. 2017. Learning fair classifiers: a regularization-inspired approach. arXiv:1707.00044 [cs.LG]
  75. 75. 
    Zafar MB, Valera I, Rogriguez MG, Gummadi KP. 2017. Fairness constraints: mechanisms for fair classification. Proc. Mach. Learn. Res. 54:962–70
    [Google Scholar]
  76. 76. 
    Chen I, Johansson FD, Sontag D. 2018. Why is my classifier discriminatory?. Advances in Neural Information Processing Systems 31 S Bengio, H Wallach, H Larochelle, K Grauman, N Cesa-Bianchi, R Garnett 3539–50 https://papers.nips.cc/paper/2018/file/1f1baa5b8edac74eb4eaa329f14a0361-Paper.pdf
    [Google Scholar]
  77. 77. 
    Richesson RL, Hammond WE, Nahm M, Wixted D, Simon GE et al. 2013. Electronic health records based phenotyping in next-generation clinical trials: a perspective from the NIH health care systems collaboratory. J. Am. Med. Inform. Assoc. 20:e226–31
    [Google Scholar]
  78. 78. 
    Banda JM, Seneviratne M, Hernandez-Boussard T, Shah NH. 2018. Advances in electronic phenotyping: from rule-based definitions to machine learning models. Annu. Rev. Biomed. Data Sci. 1:53–68
    [Google Scholar]
  79. 79. 
    Alaa AM, van der Schaar M. 2019. Attentive state-space modeling of disease progression. Advances in Neural Information Processing Systems 32 H Wallach, H Larochelle, A Beygelzimer, F d'Alché-Buc, E Fox, R Garnett 11334–44 https://papers.nips.cc/paper/2019/file/1d0932d7f57ce74d9d9931a2c6db8a06-Paper.pdf
    [Google Scholar]
  80. 80. 
    Ghassemi M, Wu M, Feng M, Celi LA, Szolovits P, Doshi-Velez F. 2016. Understanding vasopressor intervention and weaning: risk prediction in a public heterogeneous clinical time series database. J. Am. Med. Inform. Assoc. 24:3488–95
    [Google Scholar]
  81. 81. 
    Doshi-Velez F, Ge Y, Kohane I. 2014. Comorbidity clusters in autism spectrum disorders: an electronic health record time-series analysis. Pediatrics 133:e54–63
    [Google Scholar]
  82. 82. 
    Halpern Y, Horng S, Choi Y, Sontag D. 2016. Electronic medical record phenotyping using the anchor and learn framework. J. Am. Med. Inform. Assoc. 23:731–40
    [Google Scholar]
  83. 83. 
    Yu S, Ma Y, Gronsbell J, Cai T, Ananthakrishnan AN et al. 2018. Enabling phenotypic big data with PheNorm. J. Am. Med. Inform. Assoc. 25:54–60
    [Google Scholar]
  84. 84. 
    Kirby JC, Speltz P, Rasmussen LV, Basford M, Gottesman O et al. 2016. PheKB: a catalog and workflow for creating electronic phenotype algorithms for transportability. J. Am. Med. Inform. Assoc. 23:1046–52
    [Google Scholar]
  85. 85. 
    Liao KP, Sun J, Cai TA, Link N, Hong C et al. 2019. High-throughput multimodal automated phenotyping (map) with application to PheWAS. J. Am. Med. Inform. Assoc. 26:1255–62
    [Google Scholar]
  86. 86. 
    Zhang Y, Cai T, Yu S, Cho K, Hong C et al. 2019. High-throughput phenotyping with electronic medical record data using a common semi-supervised approach (PheCAP). Nat. Protoc. 14:3426–44
    [Google Scholar]
  87. 87. 
    Ghassemi M, Naumann T, Schulam P, Beam AL, Chen IY, Ranganath R. 2019. Practical guidance on artificial intelligence for health-care data. Lancet Digital Health 1:e157–59
    [Google Scholar]
  88. 88. 
    Greenspan NS. 2018. Autism, evolution, and the inadequacy of ‘spectrum.’. Evol. Med. Public Health 2018:213–16
    [Google Scholar]
  89. 89. 
    Pivovarov R, Perotte AJ, Grave E, Angiolillo J, Wiggins CH, Elhadad N. 2015. Learning probabilistic phenotypes from heterogeneous EHR data. J. Biomed. Inform. 58:156–65
    [Google Scholar]
  90. 90. 
    Hirsch JS, Tanenbaum JS, Lipsky Gorman S, Liu C, Schmitz E et al. 2015. Harvest, a longitudinal patient record summarizer. J. Am. Med. Inform. Assoc. 22:263–74
    [Google Scholar]
  91. 91. 
    Kingma DP, Welling M. 2013. Auto-encoding variational Bayes. arXiv:1312.6114 [stat.ML]
  92. 92. 
    Pierson E, Koh PW, Hashimoto T, Koller D, Leskovec J et al. 2019. Inferring multidimensional rates of aging from cross-sectional data. Proc. Mach. Learn. Res. 89:97–107
    [Google Scholar]
  93. 93. 
    Henderson J, He H, Malin BA, Denny JC, Kho AN et al. 2018. Phenotyping through semi-supervised tensor factorization (PSST). AMIA Annu. Symp. Proc. 2018:564–73
    [Google Scholar]
  94. 94. 
    Belgrave DC, Custovic A, Simpson A. 2013. Characterizing wheeze phenotypes to identify endotypes of childhood asthma, and the implications for future management. Expert Rev. Clin. Immunol. 9:921–36
    [Google Scholar]
  95. 95. 
    Deliu M, Belgrave D, Sperrin M, Buchan I, Custovic A. 2017. Asthma phenotypes in childhood. Expert Rev. Clin. Immunol. 13:705–13
    [Google Scholar]
  96. 96. 
    Hripcsak G, Ryan PB, Duke JD, Shah NH, Park RW et al. 2016. Characterizing treatment pathways at scale using the OHDSI network. PNAS 113:7329–36
    [Google Scholar]
  97. 97. 
    Urteaga I, McKillop M, Elhadad N. 2020. Learning endometriosis phenotypes from patient-generated data. NPJ Digit. Med. 3:88
    [Google Scholar]
  98. 98. 
    Schulam P, Wigley F, Saria S. 2015. Clustering longitudinal clinical marker trajectories from electronic health data: applications to phenotyping and endotype discovery. Proceedings of the 29th AAAI Conference on Artificial Intelligence2956–64 Palo Alto, CA: AAAI
    [Google Scholar]
  99. 99. 
    Ghassemi M, Wu M, Hughes MC, Szolovits P, Doshi-Velez F. 2017. Predicting intervention onset in the ICU with switching state space models. AMIA Jt. Summits Transl. Sci. Proc. 2017:82–91
    [Google Scholar]
  100. 100. 
    Suresh H, Szolovits P, Ghassemi M. 2017. The use of autoencoders for discovering patient phenotypes. arXiv:1703.07004 [cs.LG]
  101. 101. 
    Suresh H, Hunt N, Johnson A, Celi LA, Szolovits P, Ghassemi M. 2017. Clinical intervention prediction and understanding with deep neural networks. Proc. Mach. Learn. Res. 68:322–37
    [Google Scholar]
  102. 102. 
    Chen GH, Weiss JC. 2017. Survival-supervised topic modeling with anchor words: characterizing pancreatitis outcomes. arXiv:1712.00535 [stat.ML]
  103. 103. 
    Rodriguez VA, Perotte A. 2019. Phenotype inference with semi-supervised mixed membership models. Proc. Mach. Learn. Res. 106:304–24
    [Google Scholar]
  104. 104. 
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D et al. 2014. Generative adversarial nets. Advances in Neural Information Processing Systems 27 Z Ghahramani, M Welling, C Cortes, N Lawrence, KQ Weinberger 2672–80 https://papers.nips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
    [Google Scholar]
  105. 105. 
    Rezende D, Mohamed S 2015. Variational inference with normalizing flows. Proc. Mach. Learn. Res. 37:1530–38
    [Google Scholar]
  106. 106. 
    Shorten C, Khoshgoftaar TM. 2019. A survey on image data augmentation for deep learning. J. Big Data 6:60
    [Google Scholar]
  107. 107. 
    Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan H. 2018. GAN-based synthetic medical image augmentation for increased CNN performance in liver lesion classification. Neurocomputing 321:321–31
    [Google Scholar]
  108. 108. 
    Gupta A, Venkatesh S, Chopra S, Ledig C. 2019. Generative image translation for data augmentation of bone lesion pathology. Proc. Mach. Learn. Res. 102:225–35
    [Google Scholar]
  109. 109. 
    Han C, Murao K, Noguchi T, Kawata Y, Uchiyama F et al. 2019. Learning more with less: conditional PGGAN-based data augmentation for brain metastases detection using highly-rough annotation on MR images. Proceedings of the 28th ACM International Conference on Information and Knowledge Management119–27 New York: Assoc. Comput. Mach.
    [Google Scholar]
  110. 110. 
    Bailo O, Ham D, Min Shin Y 2019. Red blood cell image generation for data augmentation using conditional generative adversarial networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops1039–48 Los Alamitos, CA: IEEE Comput. Soc.
    [Google Scholar]
  111. 111. 
    Beaulieu-Jones BK, Wu ZS, Williams C, Lee R, Bhavnani SP et al. 2019. Privacy-preserving generative deep neural networks support clinical data sharing. Circulation 12:e005122
    [Google Scholar]
  112. 112. 
    Kazeminia S, Baur C, Kuijper A, van Ginneken B, Navab N et al. 2018. GANs for medical image analysis. arXiv:1809.06222 [cs.CV]
  113. 113. 
    Chen X, Pawlowski N, Rajchl M, Glocker B, Konukoglu E. 2018. Deep generative models in the real-world: an open challenge from medical imaging. arXiv:1806.05452 [cs.CV]
  114. 114. 
    Baumgartner CF, Koch LM, Can Tezcan K, Xi Ang J, Konukoglu E 2018. Visual feature attribution using Wasserstein GANs. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition8309–19 Los Alamitos, CA: IEEE Comput. Soc.
    [Google Scholar]
  115. 115. 
    Armanious K, Jiang C, Fischer M, Küstner T, Hepp T et al. 2020. MedGAN: medical image translation using GANs. Comput. Med. Imaging Graph. 79:101684
    [Google Scholar]
  116. 116. 
    Brown N, Fiscato M, Segler MH, Vaucher AC. 2019. GuacaMol: benchmarking models for de novo molecular design. J. Chem. Inform. Model. 59:1096–108
    [Google Scholar]
  117. 117. 
    Putin E, Asadulaev A, Vanhaelen Q, Ivanenkov Y, Aladinskaya AV et al. 2018. Adversarial threshold neural computer for molecular de novo design. Mol. Pharm. 15:4386–97
    [Google Scholar]
  118. 118. 
    Preuer K, Renz P, Unterthiner T, Hochreiter S, Klambauer G. 2018. Fréchet ChemNet distance: a metric for generative models for molecules in drug discovery. J. Chem. Inform. Model. 58:1736–41
    [Google Scholar]
  119. 119. 
    Zhavoronkov A, Ivanenkov YA, Aliper A, Veselov MS, Aladinskiy VA et al. 2019. Deep learning enables rapid identification of potent DDR1 kinase inhibitors. Nat. Biotechnol. 37:1038–40
    [Google Scholar]
  120. 120. 
    Futoma J, Hughes MC, Doshi-Velez F. 2020. POPCORN: partially observed prediction constrained reinforcement learning. Proc. Mach. Learn. Res. 108:3578–88
    [Google Scholar]
  121. 121. 
    Shortreed SM, Laber E, Lizotte DJ, Stroup TS, Pineau J, Murphy SA. 2011. Informing sequential clinical decision-making through reinforcement learning: an empirical study. Mach. Learn. 84:109–36
    [Google Scholar]
  122. 122. 
    Chang CH, Mai M, Goldenberg A. 2019. Dynamic measurement scheduling for event forecasting using deep RL. Proc. Mach. Learn. Res. 97:951–60
    [Google Scholar]
  123. 123. 
    Prasad N, Cheng LF, Chivers C, Draugelis M, Engelhardt BE. 2017. A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. arXiv:1704.06300 [cs.AI]
  124. 124. 
    Martn-Guerrero JD, Gomez F, Soria-Olivas E, Schmidhuber J, Climente-Mart M, Jiménez-Torres NV. 2009. A reinforcement learning approach for individualizing erythropoietin dosages in hemodialysis patients. Expert Syst. Appl. 36:9737–42
    [Google Scholar]
  125. 125. 
    Li L, Komorowski M, Faisal AA. 2018. The actor search tree critic (ASTC) for off-policy POMDP learning in medical decision making. arXiv:1805.11548 [cs.AI]
  126. 126. 
    Yu C, Liu J, Nemati S. 2019. Reinforcement learning in healthcare: a survey. arXiv:1908.08796 [cs.LG]
  127. 127. 
    Gottesman O, Johansson F, Komorowski M, Faisal A, Sontag D et al. 2019. Guidelines for reinforcement learning in healthcare. Nat. Med. 25:16–18
    [Google Scholar]
  128. 128. 
    Sutton RS, Barto AG. 2018. Reinforcement Learning: An Introduction Cambridge, MA: MIT Press. , 2nd ed..
  129. 129. 
    Javad MOM, Agboola SO, Jethwani K, Zeid A, Kamarthi S. 2019. A reinforcement learning–based method for management of type 1 diabetes: exploratory study. JMIR Diabetes 4:e12905
    [Google Scholar]
  130. 130. 
    Raghu A, Komorowski M, Celi LA, Szolovits P, Ghassemi M. 2017. Continuous state-space models for optimal sepsis treatment: a deep reinforcement learning approach. Proc. Mach. Learn. Res. 68:147–63
    [Google Scholar]
  131. 131. 
    Man CD, Micheletto F, Lv D, Breton M, Kovatchev B, Cobelli C. 2014. The UVA/PADOVA type 1 diabetes simulator: new features. J. Diabetes Sci. Technol. 8:26–34
    [Google Scholar]
  132. 132. 
    Harris JA, Benedict FG. 1919. A Biometric Study of Basal Metabolism in Man Washington, DC: Carnegie Inst. Wash.
  133. 133. 
    Gottesman O, Johansson F, Komorowski M, Faisal A, Sontag D et al. 2019. Guidelines for reinforcement learning in healthcare. Nat. Med. 25:16–18
    [Google Scholar]
  134. 134. 
    Van Parys J. 2016. Variation in physician practice styles within and across emergency departments. PLOS ONE 11:e0159882
    [Google Scholar]
  135. 135. 
    Oberst M, Sontag D. 2019. Counterfactual off-policy evaluation with Gumbel-Max structural causal models. Proc. Mach. Learn. Res. 97:4881–90
    [Google Scholar]
  136. 136. 
    Sallans B 2000. Learning factored representations for partially observable Markov decision processes. Advances in Neural Information Processing Systems 12 SA Solla, TK Leen, K Müller 1050–56 http://papers.neurips.cc/paper/1754-learning-factored-representations-for-partially-observable-markov-decision-processes.pdf
    [Google Scholar]
  137. 137. 
    Tennenholtz G, Mannor S, Shalit U. 2019. Off-policy evaluation in partially observable environments. Proceedings of the 34th AAAI Conference on Artificial Intelligence10276–83 Palo Alto, CA: AAAI
    [Google Scholar]
  138. 138. 
    Hernán MA, Robins JM. 2020. Causal Inference: What If Boca Raton, FL: Chapman & Hall/CRC
  139. 139. 
    Cain LE, Robins JM, Lanoy E, Logan R, Costagliola D, Hernán MA. 2010. When to start treatment? A systematic approach to the comparison of dynamic regimes using observational data. Int. J. Biostat. 6:18
    [Google Scholar]
  140. 140. 
    Dorie V, Hill J, Shalit U, Scott M, Cervone D et al. 2019. Automated versus do-it-yourself methods for causal inference: lessons learned from a data analysis competition. Stat. Sci. 34:43–68
    [Google Scholar]
  141. 141. 
    Krishnan RG, Shalit U, Sontag D. 2017. Structured inference networks for nonlinear state space models. Proceedings of the 31st AAAI Conference on Artificial Intelligence2101–9 Palo Alto, CA: AAAI
    [Google Scholar]

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error