1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Statistics and Its Application
  4. Volume 3, 2016
  5. Article

Annual Review of Statistics and Its Application

Volume 3, 2016

There Is Individualized Treatment. Why Not Individualized Inference?

Abstract

Doctors use statistics to advance medical knowledge; we use a medical analogy to build statistical inference “from scratch” and to highlight an improvement. A doctor, perhaps implicitly, predicts a treatment's effectiveness for an individual patient based on its performance in a clinical trial; the trial patients serve as controls for that particular patient. The same logic underpins statistical inference: To identify the best statistical procedure to use for a problem, we simulate a set of control problems and evaluate candidate procedures on the controls. Recent interest in personalized/individualized medicine stems from the recognition that some clinical trial patients are better controls for a particular patient than others. Therefore, an individual patient's treatment decisions should depend only on a subset of relevant patients. Individualized statistical inference implements this idea for control problems (rather than for patients). Its potential for improving data analysis matches that of personalized medicine for improving health care. The central issue—for both individualized medicine and individualized inference—is how to make the right relevance-robustness trade-off: If we exercise too much judgment in determining which controls are relevant, our inferences will not be robust. How much is too much? We argue that the unknown answer is the Holy Grail of statistical inference.

Keyword(s): Bayes-frequentist dichotomy, conditional inference, fiducial inference, objective inference, relevance-robustness tradeoff
Loading

Article metrics loading...

/content/journals/10.1146/annurev-statistics-010814-020310
2016-06-01
2025-04-29
Loading full text...

Full text loading...

/deliver/fulltext/statistics/3/1/annurev-statistics-010814-020310.html?itemId=/content/journals/10.1146/annurev-statistics-010814-020310&mimeType=html&fmt=ahah

Literature Cited

  1. Arlot S, Celisse A. 2010. A survey of cross-validation procedures for model selection. Stat. Surv. 4:40–79 [Google Scholar]
  2. Barnard GA, Jenkins GM, Winsten CB. 1962. Likelihood inference and time series. J. R. Stat. Soc. A 125:321–72 [Google Scholar]
  3. Basu D. 1980. Randomization analysis of experimental data: the Fisher randomization test. J. Am. Stat. Assoc. 75:575–82 [Google Scholar]
  4. Bayarri MJ, Berger JO. 2000. P values for composite null models. J. Am. Stat. Assoc. 95:1127–42 [Google Scholar]
  5. Bayarri MJ, Berger JO. 2004. The interplay of Bayesian and frequentist analysis. Stat. Sci. 19:58–80 [Google Scholar]
  6. Benjamini Y, Hochberg Y. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. R. Stat. Soc. B 57:289–300 [Google Scholar]
  7. Berger JO. 1985. The frequentist viewpoint and conditioning. Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer 1 L LeCam, R Olshen 15–44 Belmont, CA: Wadsworth [Google Scholar]
  8. Berger JO. 1988. An alternative: The estimated confidence approach. Statistical Decision Theory and Related Topics IV SS Gupta, JO Berger 85–90 New York: Springer-Verlag [Google Scholar]
  9. Berger JO. 2003. Could Fisher, Jeffreys and Neyman have agreed on testing?. Stat. Sci. 18:1–32 [Google Scholar]
  10. Berger JO. 2006. The case for objective Bayesian analysis. Bayesian Anal. 1:385–402 [Google Scholar]
  11. Berger JO, Brown LD, Wolpert RL. 1994. A unified conditional frequentist and Bayesian test for fixed and sequential simple hypothesis testing. Ann. Stat. 22:1787–807 [Google Scholar]
  12. Berger JO, Sellke T. 1987. Testing a point null hypothesis: the irreconcilability of P values and evidence. J. Am. Stat. Assoc. 82:112–22 [Google Scholar]
  13. Berger JO, Wolpert RL. 1988. The Likelihood Principle Beachwood, OH: Inst. Math. Stat. [Google Scholar]
  14. Box GE. 1980. Sampling and Bayes' inference in scientific modelling and robustness. J. R. Stat. Soc. A 143:383–430 [Google Scholar]
  15. Breiman L. 2001. Statistical modeling: the two cultures. Stat. Sci. 16:199–231 [Google Scholar]
  16. Brown LD. 1978. A contribution to Kiefer's theory of conditional confidence procedures. Ann. Stat. 6:59–71 [Google Scholar]
  17. Brown LD. 1990. An ancillarity paradox which appears in multiple linear regression. Ann. Stat. 18:471–93 [Google Scholar]
  18. Brown LD. 1994. Minimaxity: more or less. Statistical Decision Theory and Related Topics V SS Gupta, JO Berger 1–18 New York: Springer [Google Scholar]
  19. Burnham KP, Anderson DR. 2002. Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach New York: Springer [Google Scholar]
  20. Carlin BP, Louis TA. 2000. Bayes and Empirical Bayes Methods for Data Analysis Boca Raton, FL: CRC [Google Scholar]
  21. Casella G. 1985. An introduction to empirical Bayes data analysis. Am. Stat. 39:83–87 [Google Scholar]
  22. Casella G, Berger RL. 1987. Reconciling Bayesian and frequentist evidence in the one-sided testing problem. J. Am. Stat. Assoc. 82:106–11 [Google Scholar]
  23. Cesa-Bianchi N. 2006. Prediction, Learning, and Games Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  24. Cochran WG. 2007. Sampling Techniques New York: Wiley [Google Scholar]
  25. Conti S, Gosling JP, Oakley JE, O'Hagan A. 2009. Gaussian process emulation of dynamic computer codes. Biometrika 96:663–76 [Google Scholar]
  26. Cox DR. 1958. Some problems connected with statistical inference. Ann. Math. Stat. 29:357–72 [Google Scholar]
  27. Cox DR. 1982. Randomization and concomitant variables in the design of experiments. Statistics and Probability: Essays in Honor of C.R. Rao G Kallianpur, PR Krishnaiah, JK Ghosh 197–202 Amsterdam: North-Holland [Google Scholar]
  28. Cox DR. 2006. Principles of Statistical Inference Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  29. Daubechies I. 2010. Wavelets and applications. The Princeton Companion to Mathematics T Gowers, J Barrow-Green, I Leader 848–62 Princeton, NJ: Princeton Univ. Press [Google Scholar]
  30. Davison AC, Hinkley DV. 1997. Bootstrap Methods and Their Application Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  31. Dawid AP, Stone M. 1982. The functional-model basis of fiducial inference. Ann. Stat. 10:1054–67 [Google Scholar]
  32. Dempster AP. 1968. A generalization of Bayesian inference. J. R. Stat. Soc. B 30:205–47 [Google Scholar]
  33. Dempster AP. 1997. The direct use of likelihood for significance testing. Stat. Comput. 7:247–52 [Google Scholar]
  34. Donoho DL, Johnstone IM, Kerkyacharian G, Picard D. 1995. Wavelet shrinkage: asymptopia?. J. R. Stat. Soc. B 57:301–69 [Google Scholar]
  35. Efron B. 1986. Why isn't everyone a Bayesian?. Am. Stat. 40:1–5 [Google Scholar]
  36. Efron B. 1998. R.A. Fisher in the 21st century. Stat. Sci. 13:95–114 [Google Scholar]
  37. Efron B. 2007. Correlation and large-scale simultaneous significance testing. J. Am. Stat. Assoc. 102:93–103 [Google Scholar]
  38. Efron B. 2010. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  39. Efron B, Gong G. 1983. A leisurely look at the bootstrap, the jackknife, and cross-validation. Am. Stat. 37:36–48 [Google Scholar]
  40. Efron B, Morris C. 1975. Data analysis using Stein's estimator and its generalizations. J. Am. Stat. Assoc. 70:311–19 [Google Scholar]
  41. Efron B, Morris C. 1977. Stein's paradox in statistics. Sci. Am. 236:119–27 [Google Scholar]
  42. Efron B, Tibshirani RJ. 1994. An Introduction to the Bootstrap Boca Raton, FL: CRC [Google Scholar]
  43. Evans M, Moshonov H. 2006. Checking for prior-data conflict. Bayesian Anal. 1:893–914 [Google Scholar]
  44. Fisher RA. 1925. Theory of statistical estimation. Math. Proc. Camb. Philos. Soc. 22:700–25 [Google Scholar]
  45. Fisher RA. 1930. Inverse probability. Math. Proc. Camb. Philos. Soc. 26:528–35 [Google Scholar]
  46. Fisher RA. 1934. Two new properties of mathematical likelihood. Proc. R. Soc. Lond. A 144:285–307 [Google Scholar]
  47. Fisher RA. 1935. The fiducial argument in statistical inference. Ann. Eugen. 6:391–98 [Google Scholar]
  48. Fisher RA. 1959. Mathematical probability in the natural sciences. Technometrics 1:21–29 [Google Scholar]
  49. Fourdrinier D, Wells MT. 2012. On improved loss estimation for shrinkage estimators. Stat. Sci. 27:61–81 [Google Scholar]
  50. Fraser DAS. 1968. The Structure of Inference New York: Wiley [Google Scholar]
  51. Fraser DAS. 2004. Ancillaries and conditional inference. Stat. Sci. 19:333–69 [Google Scholar]
  52. Fraser DAS. 2011. Is Bayes posterior just quick and dirty confidence?. Stat. Sci. 26:299–316 [Google Scholar]
  53. Fraser DAS, Reid N, Marras E, Yi G. 2010. Default priors for Bayesian and frequentist inference. J. R. Stat. Soc. B 72:631–54 [Google Scholar]
  54. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. 2013. Bayesian Data Analysis Boca Raton, FL: CRC [Google Scholar]
  55. Gelman A, Hill J. 2006. Data Analysis Using Regression and Multilevel/Hierarchical Models Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  56. Ghosh M. 2011. Objective priors: an introduction for frequentists. Stat. Sci. 26:187–202 [Google Scholar]
  57. Ghosh M, Reid N, Fraser DAS. 2010. Ancillary statistics: a review. Stat. Sin. 20:1309–32 [Google Scholar]
  58. Godambe VP, Thompson ME. 1976. Philosophy of survey-sampling practice. Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science W Harper, C Hooker 103–23 New York: Springer [Google Scholar]
  59. Goutis C, Casella G. 1995. Frequentist post-data inference. Int. Stat. Rev. 63:325–44 [Google Scholar]
  60. Hannig J. 2009. On generalized fiducial inference. Stat. Sin. 19:491–544 [Google Scholar]
  61. Hansen MH, Yu B. 2001. Model selection and the principle of minimum description length. J. Am. Stat. Assoc. 96:746–74 [Google Scholar]
  62. Hwang JT, Casella G, Robert C, Wells MT, Farrell RH. 1992. Estimation of accuracy in testing. Ann. Stat. 20:490–509 [Google Scholar]
  63. Ioannidis JPA, Ntzani EE, Trikalinos TA, Contopoulos-Ioannidis DG. 2001. Replication validity of genetic association studies. Nat. Genet. 29:306–9 [Google Scholar]
  64. Jaynes ET. 2003. Probability Theory: The Logic of Science Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  65. Jeffreys H. 1946. An invariant form for the prior probability in estimation problems. Proc. R. Soc. Lond. A 186:453–61 [Google Scholar]
  66. Jeffreys H. 1961. Theory of Probability New York: Oxford Univ. Press, 3rd ed.. [Google Scholar]
  67. Johnson VE, Rossell D. 2010. On the use of non-local prior densities in Bayesian hypothesis tests. J. R. Stat. Soc. B 72:143–70 [Google Scholar]
  68. Kass RE. 2011. Statistical inference: the big picture. Stat. Sci. 26:1–9 [Google Scholar]
  69. Kass RE, Wasserman L. 1996. The selection of prior distributions by formal rules. J. Am. Stat. Assoc. 91:1343–70 [Google Scholar]
  70. Kennedy MC, O'Hagan A. 2001. Bayesian calibration of computer models. J. R. Stat. Soc. B 63:425–64 [Google Scholar]
  71. Kiefer J. 1976. Admissibility of conditional confidence procedures. Ann. Stat. 4:836–65 [Google Scholar]
  72. Kiefer J. 1977. Conditional confidence statements and confidence estimators. J. Am. Stat. Assoc. 72:789–808 [Google Scholar]
  73. Kruskal W. 1988. Miracles and statistics: the casual assumption of independence. J. Am. Stat. Assoc. 83:929–40 [Google Scholar]
  74. Leeb H, Pötscher BM. 2005. Model selection and inference: facts and fiction. Econ. Theory 21:21–59 [Google Scholar]
  75. Lehmann EL. 1990. Model specification: the views of Fisher and Neyman, and later developments. Stat. Sci. 5:160–68 [Google Scholar]
  76. Lehmann EL, Casella G. 1998. Theory of Point Estimation New York: Springer, 2nd ed.. [Google Scholar]
  77. Little RJ. 2006. Calibrated Bayes: a Bayes/frequentist roadmap. Am. Stat. 60:213–23 [Google Scholar]
  78. Lu KL, Berger JO. 1989. Estimation of normal means: frequentist estimation of loss. Ann. Stat. 17:890–906 [Google Scholar]
  79. Martin R, Liu C. 2013. Inferential models: a framework for prior-free posterior probabilistic inference. J. Am. Stat. Assoc. 108:301–13 [Google Scholar]
  80. Meng XL. 2014. A trio of inference problems that could win you a Nobel Prize in Statistics (if you help fund it). Past, Present and Future of Statistical Science X Lin, DL Banks, C Genest, G Molenberghs, DW Scott, JL Wang 535–60 Boca Raton, FL: CRC [Google Scholar]
  81. Morris CN. 1983. Parametric empirical Bayes inference: theory and applications. J. Am. Stat. Assoc. 78:47–55 [Google Scholar]
  82. Neyman J. 1934. On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection. J. R. Stat. Soc. 97:558–625 [Google Scholar]
  83. Reid N. 1995. The roles of conditioning in inference. Stat. Sci. 10:138–57 [Google Scholar]
  84. Robbins H. 1956. An empirical Bayes approach to statistics. Proc. 3rd Berkeley Symp. Math. Stat. Probab. 1 J Neyman 157–63 Berkeley: Univ. Calif. Press [Google Scholar]
  85. Robbins H. 1964. The empirical Bayes approach to statistical decision problems. Ann. Math. Stat. 35:1–20 [Google Scholar]
  86. Robins J, Wasserman L. 2000. Conditioning, likelihood, and coherence: a review of some foundational concepts. J. Am. Stat. Assoc. 95:1340–46 [Google Scholar]
  87. Robinson GK. 1979a. Conditional properties of statistical procedures. Ann. Stat. 7:742–55 [Google Scholar]
  88. Robinson GK. 1979b. Conditional properties of statistical procedures for location and scale parameters. Ann. Stat. 7:756–71 [Google Scholar]
  89. Rosenbaum PR. 1984. Conditional permutation tests and the propensity score in observational studies. J. Am. Stat. Assoc. 79:565–74 [Google Scholar]
  90. Rosenbaum PR, Rubin DB. 1984. Sensitivity of Bayes inference with data-dependent stopping rules. Am. Stat. 38:106–9 [Google Scholar]
  91. Royall R. 1997. Statistical Evidence: A Likelihood Paradigm Boca Raton, FL: CRC [Google Scholar]
  92. Rubin DB. 1978. Bayesian inference for causal effects: the role of randomization. Ann. Stat. 6:34–58 [Google Scholar]
  93. Rubin DB. 1984. Bayesianly justifiable and relevant frequency calculations for the applied statistician. Ann. Stat. 12:1151–72 [Google Scholar]
  94. Rukhin AL. 1988. Estimated loss and admissible loss estimators. Statistical Decision Theory and Related Topics IV SS Gupta, JO Berger 409–18 New York: Springer-Verlag [Google Scholar]
  95. Samaniego FJ, Reneau DM. 1994. Toward a reconciliation of the Bayesian and frequentist approaches to point estimation. J. Am. Stat. Assoc. 89:947–57 [Google Scholar]
  96. Shafer G. 1976. A Mathematical Theory of Evidence Princeton, NJ: Princeton Univ. Press [Google Scholar]
  97. Stigler SM. 1990. The 1988 Neyman memorial lecture: a Galtonian perspective on shrinkage estimators. Stat. Sci. 5:147–55 [Google Scholar]
  98. Strawderman WE. 2000. Minimaxity. J. Am. Stat. Assoc. 95:1364–68 [Google Scholar]
  99. Sundberg R. 2003. Conditional statistical inference and quantification of relevance. J. R. Stat. Soc. B 65:299–315 [Google Scholar]
  100. Wasserman L. 2011a. Frasian inference. Stat. Sci. 26:322–25 [Google Scholar]
  101. Wasserman L. 2011b. Low assumptions, high dimensions. Ration. Markets Morals 2:201–9 [Google Scholar]
  102. Xie Mg, Singh K. 2013. Confidence distribution, the frequentist distribution estimator of a parameter: a review. Int. Stat. Rev. 81:3–39 [Google Scholar]
  103. Zabell SL. 1992. R.A. Fisher and the fiducial argument. Stat. Sci. 7:369–87 [Google Scholar]
  104. Zhang JL, Rubin DB, Mealli F. 2009. Likelihood-based analysis of causal effects of job-training programs using principal stratification. J. Am. Stat. Assoc. 104:166–76 [Google Scholar]
/content/journals/10.1146/annurev-statistics-010814-020310
Loading
There Is Individualized Treatment. Why Not Individualized Inference?
Annual Review of Statistics and Its Application 3, 79 (2016); https://doi.org/10.1146/annurev-statistics-010814-020310
/content/journals/10.1146/annurev-statistics-010814-020310
/content/journals/10.1146/annurev-statistics-010814-020310
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error