1932

Abstract

In the healthcare landscape, data science (DS) methods have emerged as indispensable tools to harness real-world data (RWD) from various data sources such as electronic health records, claim and registry data, and data gathered from digital health technologies. Real-world evidence (RWE) generated from RWD empowers researchers, clinicians, and policymakers with a more comprehensive understanding of real-world patient outcomes. Nevertheless, persistent challenges in RWD (e.g., messiness, voluminousness, heterogeneity, multimodality) and a growing awareness of the need for trustworthy and reliable RWE demand innovative, robust, and valid DS methods for analyzing RWD. In this article, I review some common current DS methods for extracting RWE and valuable insights from complex and diverse RWD. This article encompasses the entire RWE-generation pipeline, from study design with RWD to data preprocessing, exploratory analysis, methods for analyzing RWD, and trustworthiness and reliability guarantees, along with data ethics considerations and open-source tools. This review, tailored for an audience that may not be experts in DS, aspires to offer a systematic review of DS methods and assists readers in selecting suitable DS methods and enhancing the process of RWE generation for addressing their specific challenges.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-biodatasci-102423-113220
2024-08-23
2024-12-06
Loading full text...

Full text loading...

/deliver/fulltext/biodatasci/7/1/annurev-biodatasci-102423-113220.html?itemId=/content/journals/10.1146/annurev-biodatasci-102423-113220&mimeType=html&fmt=ahah

Literature Cited

  1. 1.
    US Food Drug Admin. 2023.. Real-world evidence. . US Food and Drug Administration. https://www.fda.gov/science-research/science-and-research-special-topics/real-world-evidence
    [Google Scholar]
  2. 2.
    US Food Drug Admin. 2021.. FDA approves abatacept for prophylaxis of acute graft versus host disease. . US Food and Drug Administration. https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-abatacept-prophylaxis-acute-graft-versus-host-disease
    [Google Scholar]
  3. 3.
    Liu F, Panagiotakos D. 2022.. Real-world data: a brief review of the methods, applications, challenges and opportunities. . BMC Med. Res. Methodol. 22:287. Correction. 2023.. BMC Med. Res. Methodol. 23:109
    [Google Scholar]
  4. 4.
    Fayyad U, Piatetsky-Shapiro G, Smyth P. 1996.. From data mining to knowledge discovery in databases. . AI Mag. 17(3):3754
    [Google Scholar]
  5. 5.
    Cios KJ, Moore GW. 2002.. Uniqueness of medical data mining. . Artif. Intel. Med. 26(1–2):124
    [Google Scholar]
  6. 6.
    Ford I, Norrie J. 2016.. Pragmatic trials. . N. Engl. J. Med. 375(5):45463
    [Google Scholar]
  7. 7.
    Hernán MA, Robins JM. 2017.. Per-protocol analyses of pragmatic trials. . N. Engl. J. Med. 377(14):139198
    [Google Scholar]
  8. 8.
    Mor V, Volandes AE, Gutman R, Gatsonis C, Mitchell SL. 2017.. Pragmatic trial of video education in nursing homes: the design and rationale for a pragmatic cluster randomized trial in the nursing home setting. . Clin. Trials 14(2):14051
    [Google Scholar]
  9. 9.
    Petersen AW, Shah AS, Simmons SF, Shotwell MS, Jacobsen JML, et al. 2018.. Shed-MEDS: pilot of a patient-centered deprescribing framework reduces medications in hospitalized older adults being transferred to inpatient postacute care. . Ther. Adv. Drug Saf. 9(9):52333
    [Google Scholar]
  10. 10.
    Hernán MA, Robins JM. 2016.. Using big data to emulate a target trial when a randomized trial is not available. . Am. J. Epidemiol. 183(8):75864
    [Google Scholar]
  11. 11.
    García-Albéniz X, Hsu J, Hernán MA. 2017.. The value of explicitly emulating a target trial when using real world evidence: an application to colorectal cancer screening. . Eur. J. Epidemiol. 32(6):495500
    [Google Scholar]
  12. 12.
    Schonberg MA, Kistler CE, Pinheiro A, Jacobson AR, Aliberti GM, et al. 2020.. Effect of a mammography screening decision aid for women 75 years and older: a cluster randomized clinical trial. . JAMA Intern. Med. 180(6):83142
    [Google Scholar]
  13. 13.
    Hinton GE, Osindero S, Teh YW. 2006.. A fast learning algorithm for deep belief nets. . Neural Comput. 18(7):152754
    [Google Scholar]
  14. 14.
    McMahan B, Moore E, Ramage D, Hampson S, Agüera y Arcas B. 2017.. Communication-efficient learning of deep networks from decentralized data. . Proc. Mach. Learn. Res. 54:127382
    [Google Scholar]
  15. 15.
    De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, et al. 2018.. Clinically applicable deep learning for diagnosis and referral in retinal disease. . Nat. Med. 24(9):134250
    [Google Scholar]
  16. 16.
    Ravizza S, Huschto T, Adamov A, Böhm L, Büsser A, et al. 2019.. Predicting the early risk of chronic kidney disease in patients with diabetes using real-world data. . Nat. Med. 25(1):5759
    [Google Scholar]
  17. 17.
    Mount Sinai. 2018.. Mount Sinai and RenalytixAI launch groundbreaking artificial intelligence solution for improved kidney disease management and patient care. Press Release. https://www.mountsinai.org/about/newsroom/2018/mount-sinai-and-renalytixai-launch-groundbreaking-artificial-intelligence-solution-for-improved-kidney-disease-management-and-patient-care
    [Google Scholar]
  18. 18.
    Fellegi IP, Sunter AB. 1969.. A theory for record linkage. . J. Am. Stat. Assoc. 64(328):1183210
    [Google Scholar]
  19. 19.
    Dempster AP, Laird NM, Rubin DB. 1977.. Maximum likelihood from incomplete data via the EM algorithm. . J. R. Stat. Soc. B 39(1):138
    [Google Scholar]
  20. 20.
    Hochreiter S, Schmidhuber J. 1997.. Long short-term memory. . Neural Comput. 9(8):173580
    [Google Scholar]
  21. 21.
    Devlin J, Chang MW, Lee K, Toutanova K. 2018.. BERT: pre-training of deep bidirectional transformers for language understanding. . arXiv:1810.04805 [cs.CL]
  22. 22.
    Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, et al. 2017.. Attention is all you need. . Adv. Neural Inform. Proc. Syst. 30:59986008
    [Google Scholar]
  23. 23.
    Zeng Z, Deng Y, Li X, Naumann T, Luo Y. 2018.. Natural language processing for EHR-based computational phenotyping. . IEEE/ACM Trans. Comput. Biol. Bioinform. 16(1):13953
    [Google Scholar]
  24. 24.
    Juhn Y, Liu H. 2020.. Artificial intelligence approaches using natural language processing to advance EHR-based clinical research. . J. Allergy Clin. Immunol. 145(2):46369
    [Google Scholar]
  25. 25.
    Chan L, Beers K, Yau AA, Chauhan K, Duffy A, et al. 2020.. Natural language processing of electronic health records is superior to billing codes to identify symptom burden in hemodialysis patients. . Kidney Int. 97(2):38392
    [Google Scholar]
  26. 26.
    Rubin DB. 1987.. Multiple Imputation for Nonresponse in Surveys. New York:: Wiley
    [Google Scholar]
  27. 27.
    Little RJ, Rubin DB. 2019.. Statistical Analysis with Missing Data. Hoboken, NJ:: Wiley
    [Google Scholar]
  28. 28.
    Andridge RR, Little RJ. 2010.. A review of hot deck imputation for survey non-response. . Int. Stat. Rev. 78(1):4064
    [Google Scholar]
  29. 29.
    Van Buuren S, Groothuis-Oudshoorn K. 2011.. mice: multivariate imputation by chained equations in R. . J. Stat. Softw. 45(3):167
    [Google Scholar]
  30. 30.
    Honaker J, King G, Blackwell M. 2011.. Amelia II: a program for missing data. . J. Stat. Softw. 45(2):147
    [Google Scholar]
  31. 31.
    Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, et al. 2014.. Generative adversarial nets. . Adv. Neural Inform. Proc. Syst. 27:267280
    [Google Scholar]
  32. 32.
    Rezende D, Mohamed S. 2015.. Variational inference with normalizing flows. . Proc. Mach. Learn Res. 37:153038
    [Google Scholar]
  33. 33.
    Kingma DP, Welling M. 2013.. Auto-encoding variational Bayes. . arXiv:1312.6114 [stat.ML]
  34. 34.
    Papamakarios G, Pavlakou T, Murray I. 2017.. Masked autoregressive flow for density estimation. . Adv. Neural Inform. Proc. Syst. 30:233847
    [Google Scholar]
  35. 35.
    Ho J, Jain A, Abbeel P. 2020.. Denoising diffusion probabilistic models. . Adv. Neural Inform. Proc. Syst. 33:684051
    [Google Scholar]
  36. 36.
    Yoon J, Jordon J, Schaar M. 2018.. Gain: missing data imputation using generative adversarial nets. . Proc. Mach. Learn. Res. 80:568998
    [Google Scholar]
  37. 37.
    Mattei PA, Frellsen J. 2019.. MIWAE: deep generative modelling and imputation of incomplete data sets. . Proc. Mach. Learn. Res. 97:441323
    [Google Scholar]
  38. 38.
    Richardson TW, Wu W, Lin L, Xu B, Bernal EA. 2020.. McFlow: Monte Carlo flow models for data imputation. . In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14193202. Piscataway, NJ:: IEEE
    [Google Scholar]
  39. 39.
    Tibshirani R. 1996.. Regression shrinkage and selection via the lasso. . J. R. Stat. Soc. B 58(1):26788
    [Google Scholar]
  40. 40.
    Zou H, Hastie T. 2005.. Regularization and variable selection via the elastic net. . J. R. Stat. Soc. B 62(2):30120
    [Google Scholar]
  41. 41.
    Zou H. 2006.. The adaptive lasso and its oracle properties. . J. Am. Stat. Assoc. 101(1):141829
    [Google Scholar]
  42. 42.
    Fan J, Li R. 2001.. Variable selection via nonconcave penalized likelihood and its oracle properties. . J. Am. Stat. Assoc. 96(456):134860
    [Google Scholar]
  43. 43.
    Yuan M, Lin Y. 2007.. Model selection and estimation in the Gaussian graphical model. . Biometrika 94:1935
    [Google Scholar]
  44. 44.
    Simon N, Friedman J, Hastie T, Tibshirani R. 2013.. SGL: fit a GLM (or Cox model) with a combination of lasso and group lasso regularization. . R package, version 1.1. https://rdrr.io/cran/SGL/
    [Google Scholar]
  45. 45.
    Zhang CH. 2010.. Nearly unbiased variable selection under minimax concave penalty. . Ann. Stat. 38(2):894942
    [Google Scholar]
  46. 46.
    Dicker L, Huang B, Lin X. 2013.. Variable selection and estimation with the seamless-L0 penalty. . Stat. Sin. 23:92962
    [Google Scholar]
  47. 47.
    Li X, Xu H, Grannis S. 2022.. The data-adaptive Fellegi-Sunter model for probabilistic record linkage: algorithm development and validation for incorporating missing data and field selection. . J. Med. Intern. Res. 24(9):e33775
    [Google Scholar]
  48. 48.
    Guyon I, Weston J, Barnhill S, Vapnik V. 2002.. Gene selection for cancer classification using support vector machines. . Mach. Learn. 46:389422
    [Google Scholar]
  49. 49.
    Granitto PM, Furlanello C, Biasioli F, Gasperi F. 2006.. Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products. . Chemom. Intel. Lab. Syst. 83(2):8390
    [Google Scholar]
  50. 50.
    Kramer MA. 1991.. Nonlinear principal component analysis using autoassociative neural networks. . AIChE J. 37(2):23343
    [Google Scholar]
  51. 51.
    Remeseiro B, Bolon-Canedo V. 2019.. A review of feature selection methods in medical applications. . Comput. Biol. Med. 112:103375
    [Google Scholar]
  52. 52.
    Payrovnaziri SN, Chen Z, Rengifo-Moreno P, Miller T, Bian J, et al. 2020.. Explainable artificial intelligence models using real-world electronic health record data: a systematic scoping review. . J. Am. Med. Inform. Assoc. 27(7):117385
    [Google Scholar]
  53. 53.
    Røislien J, Winje B. 2013.. Feature extraction across individual time series observations with spikes using wavelet principal component analysis. . Stat. Med. 32(21):366069
    [Google Scholar]
  54. 54.
    Jahankhani P, Kodogiannis V, Revett K. 2006.. EEG signal classification using wavelet feature extraction and neural networks. . In IEEE John Vincent Atanasoff 2006 International Symposium on Modern Computing, pp. 12024. Piscataway, NJ:: IEEE
    [Google Scholar]
  55. 55.
    Barandas M, Folgado D, Fernandes L, Santos S, Abreu M, et al. 2020.. TSFEL: time series feature extraction library. . SoftwareX 11:100456
    [Google Scholar]
  56. 56.
    Leskovec J, Faloutsos C. 2006.. Sampling from large graphs. . In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 63136. New York:: Assoc. Comp. Mach.
    [Google Scholar]
  57. 57.
    De Choudhury M, Lin YR, Sundaram H, Candan KS, Xie L, Kelliher A. 2010.. How does the data sampling strategy impact the discovery of information diffusion in social media?. Proc. Int. AAAI Conf. Web Soc. Media 4:3441
    [Google Scholar]
  58. 58.
    Knaus WA, Zimmerman JE, Wagner DP, Draper EA, Lawrence DE. 1981.. APACHE—acute physiology and chronic health evaluation: a physiologically based classification system. . Crit. Care Med. 9(8):59197
    [Google Scholar]
  59. 59.
    Abdia Y, Kulasekera K, Datta S, Boakye M, Kong M. 2017.. Propensity scores based methods for estimating average treatment effect and average treatment effect among treated: a comparative study. . Biometrical J. 59(5):96785
    [Google Scholar]
  60. 60.
    Desai RJ, Franklin JM. 2019.. Alternative approaches for confounding adjustment in observational studies using weighting based on the propensity score: a primer for practitioners. . BMJ 367:l5657
    [Google Scholar]
  61. 61.
    Rosenbaum PR, Rubin DB. 1983.. The central role of the propensity score in observational studies for causal effects. . Biometrika 70(1):4155
    [Google Scholar]
  62. 62.
    Brookhart MA, Schneeweiss S, Rothman KJ, Glynn RJ, Avorn J, Stürmer T. 2006.. Variable selection for propensity score models. . Am. J. Epidemiol. 163(12):114956
    [Google Scholar]
  63. 63.
    Lee BK, Lessler J, Stuart EA. 2010.. Improving propensity score weighting using machine learning. . Stat. Med. 29(3):33746
    [Google Scholar]
  64. 64.
    Choi BY, Wang CP, Gelfond J. 2020.. Machine learning outcome regression improves doubly robust estimation of average causal effects. . Pharmacoepidemiol. Drug Saf. 29(9):112033
    [Google Scholar]
  65. 65.
    Austin PC. 2014.. A comparison of 12 algorithms for matching on the propensity score. . Stat. Med. 33(6):105769
    [Google Scholar]
  66. 66.
    Hansen BB, Klopfer SO. 2006.. Optimal full matching and related designs via network flows. . J. Comput. Graph. Stat. 15(3):60927
    [Google Scholar]
  67. 67.
    Hernán MA, Robins JM. 2023.. Causal Inference: What If. Boca Raton, FL:: CRC Press
    [Google Scholar]
  68. 68.
    Robins JM, Rotnitzky A, Zhao LP. 1994.. Estimation of regression coefficients when some regressors are not always observed. . J. Am. Stat. Assoc. 89(427):84666
    [Google Scholar]
  69. 69.
    Li J, Vachani A, Epstein A, Mitra N. 2018.. A doubly robust approach for cost–effectiveness estimation from observational data. . Stat. Methods Med. Res. 27(10):312638
    [Google Scholar]
  70. 70.
    Yu J, Qin W, Huang W, Thomas K. 2023.. Oral health and mortality among older adults: a doubly robust survival analysis. . Am. J. Prev. Med. 64(1):916
    [Google Scholar]
  71. 71.
    Pearl J. 2000.. Causality: Models, Reasoning and Inference. Cambridge, UK:: Cambridge Univ. Press
    [Google Scholar]
  72. 72.
    Angrist JD, Krueger AB. 2001.. Instrumental variables and the search for identification: from supply and demand to natural experiments. . J. Econ. Perspect. 15(4):6985
    [Google Scholar]
  73. 73.
    Didelez V, Sheehan N. 2007.. Mendelian randomization as an instrumental variable approach to causal inference. . Stat. Methods Med. Res. 16(4):30930
    [Google Scholar]
  74. 74.
    Baiocchi M, Cheng J, Small DS. 2014.. Instrumental variable methods for causal inference. . Stat. Med. 33(13):2297340
    [Google Scholar]
  75. 75.
    Dimick JB, Ryan AM. 2014.. Methods for evaluating changes in health care policy: the difference-in-differences approach. . JAMA 312(22):24012
    [Google Scholar]
  76. 76.
    Wing C, Simon K, Bello-Gomez RA. 2018.. Designing difference in difference studies: best practices for public health policy research. . Annu. Rev. Public Health 39:45369
    [Google Scholar]
  77. 77.
    Kim S, Headley TY, Tozan Y. 2022.. Universal healthcare coverage and health service delivery before and during the COVID-19 pandemic: a difference-in-difference study of childhood immunization coverage from 195 countries. . PLOS Med. 19(8):e1004060
    [Google Scholar]
  78. 78.
    Li L, Shen C, Wu AC, Li X. 2011.. Propensity score-based sensitivity analysis method for uncontrolled confounding. . Am. J. Epidemiol. 174(3):34553
    [Google Scholar]
  79. 79.
    Shen C, Li X, Li L, Were MC. 2011.. Sensitivity analysis for causal inference using inverse probability weighting. . Biometrical J. 53(5):82237
    [Google Scholar]
  80. 80.
    Angrist JD, Imbens GW, Rubin DB. 1996.. Identification of causal effects using instrumental variables. . J. Am. Stat. Assoc. 91(434):44455
    [Google Scholar]
  81. 81.
    Small DS. 2007.. Sensitivity analysis for instrumental variables regression with overidentifying restrictions. . J. Am. Stat. Assoc. 102(479):104958
    [Google Scholar]
  82. 82.
    VanderWeele TJ, Ding P. 2017.. Sensitivity analysis in observational research: introducing the E-value. . Ann. Intern. Med. 167(4):26874
    [Google Scholar]
  83. 83.
    Qian M, Murphy SA. 2011.. Performance guarantees for individualized treatment rules. . Ann. Stat. 39(2):1180210
    [Google Scholar]
  84. 84.
    Robins JM, Rotnitzky A. 1992.. Recovery of information and adjustment for dependent censoring using surrogate markers. . In AIDS Epidemiology: Methodological Issues, ed. N Jewell, K Dietz, V Farewell , pp. 297331. Boston:: Birkhäuser
    [Google Scholar]
  85. 85.
    Scharfstein DO, Rotnitzky A, Robins JM. 1999.. Adjusting for nonignorable drop-out using semiparametric nonresponse models. . J. Am. Stat. Assoc. 94(448):1096120
    [Google Scholar]
  86. 86.
    Zhao Y, Zeng D, Rush J, Kosorok M. 2012.. Estimating individualized treatment rules using outcome weighted learning. . J. Am. Stat. Assoc. 107(499):110618
    [Google Scholar]
  87. 87.
    Zhou X, Mayer-Hamblett N, Khan U, Kosorok MR. 2017.. Residual weighted learning for estimating individualized treatment rules. . J. Am. Stat. Assoc. 112(517):16987
    [Google Scholar]
  88. 88.
    Wu P, Zeng D, Wang Y. 2020.. Matched learning for optimizing individualized treatment strategies using electronic health records. . J. Am. Stat. Assoc. 115(529):38092
    [Google Scholar]
  89. 89.
    Chen G, Zeng D, Kosorok M. 2016.. Personalized dose finding using outcome weighted learning. . J. Am. Stat. Assoc. 111(516):150921
    [Google Scholar]
  90. 90.
    Zhou X, Wang Y, Zeng D. 2018.. Outcome-weighted learning for personalized medicine with multiple treatment options. . In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics, ed. F Bonchi, F Provost, T Eliassi-Rad, W Wang, C Cattuto, R Ghani , pp. 56574. Piscataway, NJ:: IEEE
    [Google Scholar]
  91. 91.
    Imai K, Li ML. 2023.. Experimental evaluation of individualized treatment rules. . J. Am. Stat. Assoc. 118(541):24256
    [Google Scholar]
  92. 92.
    Eguchi S, Komori O. 2022.. Outcome weighted learning in dynamic treatment regimes. . In Minimum Divergence Methods in Statistical Machine Learning: From an Information Geometric Viewpoint, pp. 197216. Tokyo:: Springer
    [Google Scholar]
  93. 93.
    Ahsan MM, Siddique Z. 2022.. Machine learning-based heart disease diagnosis: a systematic literature review. . Artif. Intel. Med. 128:102289
    [Google Scholar]
  94. 94.
    Mei J, Desrosiers C, Frasnelli J. 2021.. Machine learning for the diagnosis of Parkinson's disease: a review of literature. . Front. Aging Neurosci. 13:633752
    [Google Scholar]
  95. 95.
    Iyer SV, Harpaz R, LePendu P, Bauer-Mehren A, Shah NH. 2014.. Mining clinical text for signals of adverse drug-drug interactions. . J. Am. Med. Inform. Assoc. 21(2):35362
    [Google Scholar]
  96. 96.
    Cheerla A, Gevaert O. 2019.. Deep learning with multimodal representation for pancancer prognosis prediction. . Bioinformatics 35(14):i44654
    [Google Scholar]
  97. 97.
    Deng Y, Xu X, Qiu Y, Xia J, Zhang W, Liu S. 2020.. A multimodal deep learning framework for predicting drug–drug interaction events. . Bioinformatics 36(15):431622
    [Google Scholar]
  98. 98.
    Rackauckas C, Ma Y, Martensen J, Warner C, Zubov K, et al. 2020.. Universal differential equations for scientific machine learning. . arXiv:2001.04385 [cs.LG]
  99. 99.
    Cuomo S, Di Cola V, Giampaolo F, Rozza G, Raissi M, Piccialli F. 2022.. Scientific machine learning through physics–informed neural networks: where we are and what's next. . J. Sci. Comput. 92(3):88
    [Google Scholar]
  100. 100.
    Laurie M, Lu J. 2023.. Explainable deep learning for tumor dynamic modeling and overall survival prediction using neural-ODE. . arXiv:2308.01362 [q-bio.QM]
  101. 101.
    Barber RF, Candès EJ. 2015.. Controlling the false discovery rate via knockoffs. . Ann. Stat. 43(5):205585
    [Google Scholar]
  102. 102.
    Candes E, Fan Y, Janson L, Lv J. 2018.. Panning for gold: ‘model-X’ knockoffs for high dimensional controlled variable selection. . J. R. Stat. Soc. B 80(3):55177
    [Google Scholar]
  103. 103.
    Wu Z, Ge R, Shi G, Zhang L, Chen Y, et al. 2020.. MD-NDNet: a multi-dimensional convolutional neural network for false-positive reduction in pulmonary nodule detection. . Phys. Med. Biol. 65(23):235053
    [Google Scholar]
  104. 104.
    Song Z, Li J. 2021.. Variable selection with false discovery rate control in deep neural networks. . Nat. Mach. Intel. 3(5):42633
    [Google Scholar]
  105. 105.
    Zhu Z, Fan Y, Kong Y, Lv J, Sun F. 2021.. DeepLINK: deep learning inference using knockoffs with applications to genomics. . PNAS 118(36):e2104683118
    [Google Scholar]
  106. 106.
    Jha A, Aicher JK, Gazzara MR, Singh D, Barash Y. 2020.. Enhanced integrated gradients: improving interpretability of deep learning models using splicing codes as a case study. . Genome Biol. 21(1):149
    [Google Scholar]
  107. 107.
    Banerjee S, Mitra S, Sharma A, Shankar BU. 2018.. A cade system for gliomas in brain MRI using convolutional neural networks. . arXiv:1806.07589 [cs.CV]
  108. 108.
    Terzi R, Azginoglu N, Terzi DS. 2022.. False positive repression: data centric pipeline for object detection in brain MRI. . Concurr. Comput. Pract. Exp. 34(20):e6821
    [Google Scholar]
  109. 109.
    Gammerman A, Vovk V, Vapnik V. 1998.. Learning by transduction. . In Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 14855. San Francisco:: Morgan Kaufman
    [Google Scholar]
  110. 110.
    Papadopoulos H, Proedrou K, Vovk V, Gammerman A. 2002.. Inductive confidence machines for regression. . In Machine Learning: ECML 2002, ed. T Elomaa, H Mannila, H Toivonen , pp. 34556. Berlin:: Springer
    [Google Scholar]
  111. 111.
    Shafer G, Vovk V. 2008.. A tutorial on conformal prediction. . J. Mach. Learn. Res. 9(3):371421
    [Google Scholar]
  112. 112.
    Pereira T, Cardoso S, Guerreiro M, Mendonça de A, Madeira SC, . 2020.. Targeting the uncertainty of predictions at patient-level using an ensemble of classifiers coupled with calibration methods, Venn-ABERS, and Conformal Predictors: a case study in AD. . J. Biomed. Inform. 101:103350
    [Google Scholar]
  113. 113.
    Olsson H, Kartasalo K, Mulliqi N, Capuccini M, Ruusuvuori P, et al. 2022.. Estimating diagnostic uncertainty in artificial intelligence assisted pathology using conformal prediction. . Nat. Commun. 13(1):7761
    [Google Scholar]
  114. 114.
    Vazquez J, Facelli JC. 2022.. Conformal prediction in clinical medical sciences. . J. Healthcare Inform. Res. 6(3):24152
    [Google Scholar]
  115. 115.
    Zhang CH, Zhang S. 2014.. Confidence intervals for low dimensional parameters in high dimensional linear models. . J. R. Stat. Soc. B 76(1):21742
    [Google Scholar]
  116. 116.
    Zhang C, Zhang SS. 2013.. Confidence intervals for low-dimensional parameters in high-dimensional linear models. . J. R. Stat. Soc. B 76(1):21742
    [Google Scholar]
  117. 117.
    Javanmard A, Montanari A. 2014.. Confidence intervals and hypothesis testing for high-dimensional regression. . J. Mach. Learn. Res. 15:2869909
    [Google Scholar]
  118. 118.
    Li Y, Liu F. 2022.. Adaptive noisy data augmentation for regularized estimation and inference of generalized linear models. . In 2022 IEEE 46th Annual Computers, Software, and Applications Conference, pp. 31120. Piscataway, NJ:: IEEE
    [Google Scholar]
  119. 119.
    Kuchibhotla AK, Kolassa JE, Kuffner TA. 2022.. Post-selection inference. . Annu. Rev. Stat. Appl. 9:50527
    [Google Scholar]
  120. 120.
    Guidotti R, Monreale A, Ruggieri S, Turini F, Giannotti F, Pedreschi D. 2018.. A survey of methods for explaining black box models. . ACM Comput. Surv. 51(5):93
    [Google Scholar]
  121. 121.
    Ahmad MA, Eckert C, Teredesai A. 2018.. Interpretable machine learning in healthcare. . In Proceedings of the 2018 ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics, pp. 55960. New York:: Assoc. Comp. Mach.
    [Google Scholar]
  122. 122.
    Stiglic G, Kocbek P, Fijacko N, Zitnik M, Verbert K, Cilar L. 2020.. Interpretability of machine learning-based prediction models in healthcare. . WIREs Data Min. Knowl. 10(5):e1379
    [Google Scholar]
  123. 123.
    Lundberg SM, Lee SI. 2017.. A unified approach to interpreting model predictions. . Adv. Neural Inform. Proc. Syst. 30:476877
    [Google Scholar]
  124. 124.
    Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. 2017.. Grad-CAM: visual explanations from deep networks via gradient-based localization. . In 2017 IEEE International Conference on Computer Vision, pp. 61826. Piscataway, NJ:: IEEE
    [Google Scholar]
  125. 125.
    Tseng PY, Chen YT, Wang CH, Chiu KM, Peng YS, et al. 2020.. Prediction of the development of acute kidney injury following cardiac surgery by machine learning. . Critical Care 24(1):478
    [Google Scholar]
  126. 126.
    Prendin F, Pavan J, Cappon G, Del Favero S, Sparacino G, Facchinetti A. 2023.. The importance of interpreting machine learning models for blood glucose prediction in diabetes: an analysis using SHAP. . Sci. Rep. 13(1):16865
    [Google Scholar]
  127. 127.
    Panwar H, Gupta P, Siddiqui MK, Morales-Menendez R, Bhardwaj P, Singh V. 2020.. A deep learning and grad-CAM based color visualization approach for fast detection of COVID-19 cases using chest X-ray and CT-Scan images. . Chaos Solitons Fractals 140:110190
    [Google Scholar]
  128. 128.
    Jahmunah V, Ng EYK, Tan RS, Oh SL, Acharya UR. 2022.. Explainable detection of myocardial infarction using deep learning models with grad-CAM technique on ECG signals. . Comput. Biol. Med. 146:105550
    [Google Scholar]
  129. 129.
    Konečn'y J, McMahan HB, Yu FX, Richtárik P, Suresh AT, Bacon D. 2016.. Federated learning: strategies for improving communication efficiency. . arXiv:1610.05492 [cs.LG]
  130. 130.
    Brisimi TS, Chen R, Mela T, Olshevsky A, Paschalidis IC, Shi W. 2018.. Federated learning of predictive models from federated electronic health records. . Int. J. Med. Inform. 112:5967
    [Google Scholar]
  131. 131.
    Guo K, Chen T, Ren S, Li N, Hu M, Kang J. 2022.. Federated learning empowered real-time medical data processing method for smart healthcare. . IEEE/ACM Trans. Comput. Biol. Bioinform. https://doi.org/10.1109/TCBB.2022.3185395
    [Google Scholar]
  132. 132.
    Rieke N, Hancox J, Li W, Milletari F, Roth HR, et al. 2020.. The future of digital health with federated learning. . npj Digital Med. 3(1):119
    [Google Scholar]
  133. 133.
    Du W, Atallah MJ. 2001.. Secure multi-party computation problems and their applications: a review and open problems. . In Proceedings of the 2001 Workshop on New Security Paradigms, pp. 1322. New York:: Assoc. Comp. Mach.
    [Google Scholar]
  134. 134.
    Rogers J, Adetoro E, Bater J, Canter T, Fu D, et al. 2022.. VaultDB: a real-world pilot of secure multi-party computation within a clinical research network. . arXiv:2203.00146 [cs.DB]
  135. 135.
    Kussel T, Brenner T, Tremper G, Schepers J, Lablans M, Hamacher K. 2022.. Record linkage based patient intersection cardinality for rare disease studies using Mainzelliste and secure multi-party computation. . J. Transl. Med. 20(1):458
    [Google Scholar]
  136. 136.
    Dwork C, McSherry F, Nissim K, Smith A. 2006.. Calibrating noise to sensitivity in private data analysis. . In Theory of Cryptography Conference: Third Theory of Cryptography Conference, TCC 2006, New York, NY, USA, March 4–7, 2006, Proceedings, ed. S Halevi, T Rabin , pp. 26584. Berlin:: Springer
    [Google Scholar]
  137. 137.
    Su B, Wang Y, Schiavazzi D, Liu F. 2023.. Privacy-preserving data synthesis via differentially private normalizing flows with application to electronic health records data. . In Proceedings of the Inaugural 2023 Summer Symposium Series 2023, pp. 16167. Washington, DC:: AAAI Press
    [Google Scholar]
  138. 138.
    Ng JH, Ye F, Ward LM, Haffer S, Scholle SH. 2017.. Data on race, ethnicity, and language largely incomplete for managed care plan members. . Health Aff. 36(3):54852
    [Google Scholar]
  139. 139.
    Adamson AS, Smith A. 2018.. Machine learning and health care disparities in dermatology. . JAMA Dermatol. 154(11):124748
    [Google Scholar]
  140. 140.
    Obermeyer Z, Powers B, Vogeli C, Mullainathan S. 2019.. Dissecting racial bias in an algorithm used to manage the health of populations. . Science 366(6464):44753
    [Google Scholar]
  141. 141.
    Hardt M, Price E, Srebro N. 2016.. Equality of opportunity in supervised learning. . Adv. Neural Inform. Proc. Syst. 29:331523
    [Google Scholar]
  142. 142.
    Xu J, Xiao Y, Wang WH, Ning Y, Shenkman EA, et al. 2022.. Algorithmic fairness in computational medicine. . eBioMedicine 84:104250
    [Google Scholar]
  143. 143.
    Kleinberg J, Mullainathan S, Raghavan M. 2016.. Inherent trade-offs in the fair determination of risk scores. . arXiv:1609.05807 [cs.LG]
  144. 144.
    Nilsson A, Bonander C, Strömberg U, Canivet C, Östergren PO, Björk J. 2021.. Reweighting a Swedish health questionnaire survey using extensive population register and self-reported data for assessing and improving the validity of longitudinal associations. . PLOS ONE 16(7):e0253969
    [Google Scholar]
  145. 145.
    Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP. 2002.. SMOTE: synthetic minority over-sampling technique. . J. Artif. Intel. Res. 16:32157
    [Google Scholar]
  146. 146.
    Li Y, Wang H, Luo Y. 2022.. Improving fairness in the prediction of heart failure length of stay and mortality by integrating social determinants of health. . Circ. Heart Fail. 15(11):e009473
    [Google Scholar]
  147. 147.
    Kamishima T, Akaho S, Sakuma J. 2011.. Fairness-aware learning through regularization approach. . In 2011 IEEE 11th International Conference on Data Mining Workshops, pp. 64350. Piscataway, NJ:: IEEE
    [Google Scholar]
  148. 148.
    Agarwal S. 2020.. Trade-offs between fairness, interpretability, and privacy in machine learning. MMath Thesis , Univ. Waterloo, Waterloo, ON, Canada:
    [Google Scholar]
  149. 149.
    Loftus JR, Russell C, Kusner MJ, Silva R. 2018.. Causal reasoning for algorithmic fairness. . arXiv:1805.05859 [cs.AI]
/content/journals/10.1146/annurev-biodatasci-102423-113220
Loading
/content/journals/10.1146/annurev-biodatasci-102423-113220
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error