1932

Abstract

Program evaluation methods are widely applied in economics to assess the effects of policy interventions and other treatments of interest. In this article, we describe the main methodological frameworks of the econometrics of program evaluation. In the process, we delineate some of the directions along which this literature is expanding, discuss recent developments, and highlight specific areas where new research may be particularly fruitful.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-economics-080217-053402
2018-08-02
2024-03-29
Loading full text...

Full text loading...

/deliver/fulltext/economics/10/1/annurev-economics-080217-053402.html?itemId=/content/journals/10.1146/annurev-economics-080217-053402&mimeType=html&fmt=ahah

Literature Cited

  1. Abadie A 2002. Bootstrap tests for distributional treatment effects in instrumental variable models. J. Am. Stat. Assoc. 97:457284–92
    [Google Scholar]
  2. Abadie A 2003. Semiparametric instrumental variable estimation of treatment response models. J. Econom. 113:2231–63
    [Google Scholar]
  3. Abadie A 2005.a Causal inference. Encyclopedia of Social Measurement 1 K Kempf-Leonard 259–66 Cambridge, MA: Academic
    [Google Scholar]
  4. Abadie A 2005.b Semiparametric difference-in-differences estimators. Rev. Econ. Stud. 72:11–19
    [Google Scholar]
  5. Abadie A, Angrist J, Imbens G 2002. Instrumental variables estimates of the effect of subsidized training on the quantiles of trainee earnings. Econometrica 70:191–117
    [Google Scholar]
  6. Abadie A, Athey S, Imbens GW, Wooldridge JM 2017.a Sampling-based vs. design-based uncertainty in regression analysis. Work. Pap. Mass. Inst. Technol. Cambridge, MA:
  7. Abadie A, Athey S, Imbens GW, Wooldridge JM 2017.b When should you adjust standard errors for clustering? Work. Pap. Mass. Inst. Technol. Cambridge, MA:
  8. Abadie A, Chingos MM, West MR 2017.c Endogenous stratification in randomized experiments Work. Pap. Mass. Inst. Technol. Cambridge, MA:
  9. Abadie A, Dermisi S 2008. Is terrorism eroding agglomeration economies in central business districts? Lessons from the office real estate market in downtown Chicago. J. Urban Econ. 64:2451–63
    [Google Scholar]
  10. Abadie A, Diamond A, Hainmueller J 2010. Synthetic control methods for comparative case studies: estimating the effect of California's tobacco control program. J. Am. Stat. Assoc. 105:490493–505
    [Google Scholar]
  11. Abadie A, Diamond A, Hainmueller J 2015. Comparative politics and the synthetic control method. Am. J. Political Sci. 59:2495–510
    [Google Scholar]
  12. Abadie A, Gardeazabal J 2003. The economic costs of conflict: a case study of the Basque Country. Am. Econ. Rev. 93:1113–32
    [Google Scholar]
  13. Abadie A, Imbens GW 2006. Large sample properties of matching estimators for average treatment effects. Econometrica 74:1235–67
    [Google Scholar]
  14. Abadie A, Imbens GW 2011. Bias-corrected matching estimators for average treatment effects. J. Bus. Econ. Stat. 29:11–11
    [Google Scholar]
  15. Abadie A, Imbens GW 2012. A martingale representation for matching estimators. J. Am. Stat. Assoc. 107:498833–43
    [Google Scholar]
  16. Abadie A, Imbens GW 2016. Matching on the estimated propensity score. Econometrica 84:2781–807
    [Google Scholar]
  17. Abadie A, Kasy M 2017. The risk of machine learning Work. Pap. Mass. Inst. Technol. Cambridge, MA:
  18. Abadie A, L'Hour J 2017. A penalized synthetic control estimator for disaggregated data Work. Pap. Mass. Inst. Technol. Cambridge, MA:
  19. Abbring JH, Heckman JJ 2007. Econometric evaluation of social programs, part III: distributional treatment effects, dynamic treatment effects, dynamic discrete choice, and general equilibrium policy evaluation. Handbook of Econometrics VI J Heckman, E Leamer 5145–303 Amsterdam: Elsevier
    [Google Scholar]
  20. Abbring JH, Van den Berg GJ 2003. The nonparametric identification of treatment effects in duration models. Econometrica 71:51491–517
    [Google Scholar]
  21. Altonji JG, Elder TE, Taber CR 2005. Selection on observed and unobserved variables: assessing the effectiveness of Catholic schools. J. Political Econ. 113:1151–84
    [Google Scholar]
  22. Amjad MJ, Shah D, Shen D 2017. Robust synthetic control. arXiv1711.06940 [econ.EM]
  23. Angrist JD 1990. Lifetime earnings and the Vietnam era draft lottery: evidence from social security administrative records. Am. Econ. Rev. 80:3313–36
    [Google Scholar]
  24. Angrist JD, Fernandez-Val I 2013. ExtrapoLATE-ing: external validity and overidentification in the LATE framework. Advances in Economics and Econometrics: Tenth World Congress, Vol. III: Econometrics D Acemoglu, M Arellano, E Dekel 401–34 Cambridge, UK: Cambridge Univ. Press
    [Google Scholar]
  25. Angrist JD, Graddy K, Imbens GW 2000. The interpretation of instrumental variables estimators in simultaneous equations models with an application to the demand for fish. Rev. Econ. Stud. 67:3499–527
    [Google Scholar]
  26. Angrist JD, Imbens GW, Rubin DB 1996. Identification of causal effects using instrumental variables. J. Am. Stat. Assoc. 91:434444–55
    [Google Scholar]
  27. Angrist JD, Krueger AB 1991. Does compulsory school attendance affect schooling and earnings. Q. J. Econ. 106:4979–1014
    [Google Scholar]
  28. Angrist JD, Pischke JS 2008. Mostly Harmless Econometrics: An Empiricist's Companion Princeton, NJ: Princeton Univ. Press
  29. Angrist JD, Pischke JS 2010. The credibility revolution in empirical economics: how better research design is taking the con out of econometrics. J. Econ. Perspect. 24:23–30
    [Google Scholar]
  30. Angrist JD, Pischke JS 2014. Mastering 'Metrics: The Path from Cause to Effect Princeton, NJ: Princeton Univ. Press
  31. Angrist JD, Rokkanen M 2015. Wanna get away? Regression discontinuity estimation of exam school effects away from the cutoff. J. Am. Stat. Assoc. 110:5121331–44
    [Google Scholar]
  32. Arcidiacono P, Ellickson PB 2011. Practical methods for estimation of dynamic discrete choice models. Annu. Rev. Econ. 3:363–94
    [Google Scholar]
  33. Arellano M 2003. Panel Data Econometrics Oxford, UK: Oxford Univ. Press
  34. Ashenfelter O 1978. Estimating the effect of training programs on earnings. Rev. Econ. Stat. 60:147–57
    [Google Scholar]
  35. Ashenfelter O, Card D 1985. Using the longitudinal structure of earnings to estimate the effect of training programs. Rev. Econ. Stat. 67:648–60
    [Google Scholar]
  36. Athey S, Bayatiz M, Doudchenkox N, Imbens GW, Khosravik K 2017. Matrix completion methods for causal panel data models. arXiv1710.10251 [math.ST]
  37. Athey S, Imbens GW 2006. Identification and inference in nonlinear difference-in-differences models. Econometrica 74:2431–97
    [Google Scholar]
  38. Athey S, Imbens GW 2016. Recursive partitioning for heterogeneous causal effects. PNAS 113:277353–60
    [Google Scholar]
  39. Athey S, Imbens GW 2017.a Econometrics of randomized experiments. Handbook of Economic Field Experiments 1 AV Banerjee, E Duflo 73–140 Amsterdam: North-Holland
    [Google Scholar]
  40. Athey S, Imbens GW 2017.b Machine learning methods for estimating heterogeneous causal effects Work. Pap. Stanford Univ. Stanford, CA:
  41. Athey S, Imbens GW 2017.c The state of applied econometrics: causality and policy evaluation. J. Econ. Perspect. 31:23–32
    [Google Scholar]
  42. Athey S, Imbens GW, Wager S 2016. Approximate residual balancing: de-biased inference of average treatment effects in high dimensions. arXiv1604.07125 [stat.ME]
  43. Bakshy E, Eckles D, Bernstein MS 2014. Designing and deploying online field experiments. Proceedings of the 23rd International Conference on World Wide Web283–92 New York: Assoc. Comput. Mach.
    [Google Scholar]
  44. Balke A, Pearl J 1997. Bounds on treatment effects from studies with imperfect compliance. J. Am. Stat. Assoc. 92:4391171–76
    [Google Scholar]
  45. Banerjee AV, Chassang S, Snowberg E 2017. Decision theoretic approaches to experiment design and external validity. Handbook of Economic Field Experiments 1 AV Banerjee, E Duflo 141–74 Amsterdam: North-Holland
    [Google Scholar]
  46. Bang H, Robins JM 2005. Doubly robust estimation in missing data and causal inference models. Biometrics 61:962–72
    [Google Scholar]
  47. Behrman JR, Parker SW, Todd PE 2011. Do conditional cash transfers for schooling generate lasting benefits? A five-year followup of PROGRESA/Oportunidades. J. Hum. Resourc. 46:193–122
    [Google Scholar]
  48. Belloni A, Chernozhukov V, Fernández-Val I, Hansen C 2017. Program evaluation with high-dimensional data. Econometrica 85:1233–98
    [Google Scholar]
  49. Belloni A, Chernozhukov V, Hansen C 2014. Inference on treatment effects after selection among high-dimensional controls. Rev. Econ. Stud. 81:2608–50
    [Google Scholar]
  50. Berry S, Haile P 2016. Identification in differentiated products markets. Annu. Rev. Econ. 8:27–52
    [Google Scholar]
  51. Bertanha M, Imbens GW 2017. External validity in fuzzy regression discontinuity designs NBER Work. Pap 20773
  52. Bertrand M, Duflo E, Mullainathan S 2004. How much should we trust differences-in-differences estimates?. Q. J. Econ. 119:1249–75
    [Google Scholar]
  53. Björklund A, Moffitt R 1987. The estimation of wage gains and welfare gains in self-selection models. Rev. Econ. Stat. 69:142–49
    [Google Scholar]
  54. Blundell R, Costa Dias M 2009. Alternative approaches to evaluation in empirical microeconomics. J. Hum. Resourc. 44:3565–640
    [Google Scholar]
  55. Bothwell LE, Greene JA, Podolsky SH, Jones DS 2016. Assessing the gold standard: lessons from the history of RCTs. New Engl. J. Med. 374:222175–81
    [Google Scholar]
  56. Brinch CN, Mogstad M, Wiswall M 2017. Beyond LATE with a discrete instrument. J. Political Econ. 125:4985–1039
    [Google Scholar]
  57. Bugni F, Canay I, Shaikh A 2018. Inference under covariate-adaptive randomization. J. Am. Stat. Assoc. In press
  58. Busso M, DiNardo J, McCrary J 2014. New evidence on the finite sample properties of propensity score reweighting and matching estimators. Rev. Econ. Stat. 96:5885–97
    [Google Scholar]
  59. Calonico S, Cattaneo MD, Farrell MH 2018.a Coverage error optimal confidence intervals Work. Pap. Univ. Mich. Ann Arbor:
  60. Calonico S, Cattaneo MD, Farrell MH 2018.b On the effect of bias estimation on coverage accuracy in nonparametric inference. J. Am. Stat. Assoc. In press
  61. Calonico S, Cattaneo MD, Farrell MH, Titiunik R 2018.c Regression discontinuity designs using covariates Work. Pap. Univ. Mich. Ann Arbor:
  62. Calonico S, Cattaneo MD, Titiunik R 2014. Robust nonparametric confidence intervals for regression-discontinuity designs. Econometrica 82:62295–326
    [Google Scholar]
  63. Calonico S, Cattaneo MD, Titiunik R 2015. Optimal data-driven regression discontinuity plots. J. Am. Stat. Assoc. 110:5121753–69
    [Google Scholar]
  64. Card D 1990. The impact of the Mariel Boatlift on the Miami labor market. ILR Rev 43:2245–57
    [Google Scholar]
  65. Card D, Krueger AB 1994. Minimum wages and employment: a case study of the fast-food industry in New Jersey and Pennsylvania. Am. Econ. Rev. 84:4772–93
    [Google Scholar]
  66. Card D, Lee DS, Pei Z, Weber A 2015. Inference on causal effects in a generalized regression kink design. Econometrica 83:62453–83
    [Google Scholar]
  67. Cartwright N 2007. Are RCTs the gold standard. BioSocieties 2:11–20
    [Google Scholar]
  68. Cattaneo MD 2010. Efficient semiparametric estimation of multi-valued treatment effects under ignorability. J. Econom. 155:2138–54
    [Google Scholar]
  69. Cattaneo MD, Escanciano JC 2017. Regression Discontinuity Designs: Theory and Applications Bingley, UK: Emerald Group
  70. Cattaneo MD, Farrell MH 2011. Efficient estimation of the dose-response function under ignorability using subclassification on the covariates. Missing-Data Methods: Cross-sectional Methods and Applications D Drukker 93–127 Bingley, UK: Emerald Group
    [Google Scholar]
  71. Cattaneo MD, Frandsen B, Titiunik R 2015. Randomization inference in the regression discontinuity design: an application to party advantages in the U.S. Senate. J. Causal Inference 3:11–24
    [Google Scholar]
  72. Cattaneo MD, Idrobo N, Titiunik R 2018.a A Practical Introduction to Regression Discontinuity Designs I Cambridge, UK: Cambridge Univ. Press In press
  73. Cattaneo MD, Idrobo N, Titiunik R 2018.b A Practical Introduction to Regression Discontinuity Designs II Cambridge, UK: Cambridge Univ. Press In press
  74. Cattaneo MD, Jansson M 2018. Kernel-based semiparametric estimators: small bandwidth asymptotics and bootstrap consistency. Econometrica In press
  75. Cattaneo MD, Jansson M, Ma X 2017.a Two-step estimation and inference with possibly many included covariates Work. Pap. Univ. Mich. Ann Arbor:
  76. Cattaneo MD, Jansson M, Newey WK 2018.c Alternative asymptotics and the partially linear model with many regressors. Econom. Theory 34:2277–301
    [Google Scholar]
  77. Cattaneo MD, Jansson M, Newey WK 2018.d Inference in linear regression models with many covariates and heteroskedasticity. J. Am. Stat. Assoc. In press
  78. Cattaneo MD, Keele L, Titiunik R, Vazquez-Bare G 2016. Interpreting regression discontinuity designs with multiple cutoffs. J. Politics 78:41229–48
    [Google Scholar]
  79. Cattaneo MD, Keele L, Titiunik R, Vazquez-Bare G 2017.b Extrapolating treatment effects in multi-cutoff regression discontinuity designs Work. Pap. Univ. Mich. Ann Arbor:
  80. Cattaneo MD, Titiunik R, Vazquez-Bare G 2017.c Comparing inference approaches for RD designs: a reexamination of the effect of Head Start on child mortality. J. Policy Anal. Manag. 36:3643–81
    [Google Scholar]
  81. Cattaneo MD, Vazquez-Bare G 2016. The choice of neighborhood in regression discontinuity designs. Obs. Stud. 2:134–46
    [Google Scholar]
  82. Cellini SR, Ferreira F, Rothstein J 2010. The value of school facility investments: evidence from a dynamic regression discontinuity design. Q. J. Econ. 125:1215–61
    [Google Scholar]
  83. Chalak K, White H 2011. An extended class of instrumental variables for the estimation of causal effects. Can. J. Econ./Rev. Can. Écon. 44:11–51
    [Google Scholar]
  84. Chernozhukov V, Escanciano JC, Ichimura H, Newey WK 2016. Locally robust semiparametric estimation. arXiv1608.00033 [math.ST]
  85. Chernozhukov V, Fernandez-Val I, Melly B 2013. Inference on counterfactual distributions. Econometrica 81:62205–68
    [Google Scholar]
  86. Chernozhukov V, Hansen C 2005. An IV model of quantile treatment effects. Econometrica 73:1245–61
    [Google Scholar]
  87. Chernozhukov V, Hansen C 2013. Quantile models with endogeneity. Annu. Rev. Econ. 5:57–81
    [Google Scholar]
  88. Cochran WG 1968. The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics 24:2295–313
    [Google Scholar]
  89. Deaton A 2010. Instruments, randomization, and learning about development. J. Econ. Lit. 48:424–55
    [Google Scholar]
  90. Dehejia RH, Wahba S 1999. Causal effects in nonexperimental studies: reevaluating the evaluation of training programs. J. Am. Stat. Assoc. 94:4481053–62
    [Google Scholar]
  91. DiNardo J, Fortin NM, Lemieux T 1996. Labor market institutions and the distribution of wages, 1973–1992: a semiparametric approach. Econometrica 64:51001–44
    [Google Scholar]
  92. DiNardo J, Lee DS 2011. Program evaluation and research designs. Handbook of Labor Economics 4A A Ashenfelter, D Card 463–536 Amsterdam: Elsevier
    [Google Scholar]
  93. Ding P 2017. A paradox from randomization-based causal inference. Stat. Sci. 32:3331–45
    [Google Scholar]
  94. Dong Y, Lewbel A 2015. Identifying the effect of changing the policy threshold in regression discontinuity models. Rev. Econ. Stat. 97:51081–92
    [Google Scholar]
  95. Doudchenko N, Imbens GW 2016. Balancing, regression, difference-in-differences and synthetic control methods: a synthesis NBER Work. Pap 22791
  96. Efron B 2012. Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction 1 Cambridge, UK: Cambridge Univ. Press
  97. Ernst MD 2004. Permutation methods: a basis for exact inference. Stat. Sci. 19:4676–85
    [Google Scholar]
  98. Fan Y, Park SS 2010. Sharp bounds on the distribution of treatment effects and their statistical inference. Econom. Theory 26:3931–51
    [Google Scholar]
  99. Farrell MH 2015. Robust inference on average treatment effects with possibly more covariates than observations. J. Econom. 189:11–23
    [Google Scholar]
  100. Finkelstein A, Taubman S, Wright B, Bernstein M, Gruber J et al. 2012. The Oregon Health Insurance Experiment: evidence from the first year. Q. J. Econ. 127:31057–106
    [Google Scholar]
  101. Firpo S 2007. Efficient semiparametric estimation of quantile treatment effects. Econometrica 75:1259–76
    [Google Scholar]
  102. Firpo S, Pinto C 2016. Identification and estimation of distributional impacts of interventions using changes in inequality measures. J. Appl. Econom. 31:3457–86
    [Google Scholar]
  103. Firpo S, Ridder G 2008. Bounds on functionals of the distribution of treatment effects Work. Pap. Univ. South. Calif. Los Angeles:
  104. Fisher RA 1935. The Design of Experiments Edinburgh: Oliver & Boyd
  105. Foster J, Shorrocks AF 1988. Poverty orderings. Econometrica 56:1173–77
    [Google Scholar]
  106. Frölich M 2007. Nonparametric IV estimation of local average treatment effects with covariates. J. Econom. 139:135–75
    [Google Scholar]
  107. Frölich M, Melly B 2013. Unconditional quantile treatment effects under endogeneity. J. Bus. Econ. Stat. 31:3346–57
    [Google Scholar]
  108. Graham BS 2015. Methods of identification in social networks. Annu. Rev. Econ. 7:465–85
    [Google Scholar]
  109. Graham BS, de Paula A 2018. The Econometrics of Social and Economic Networks Amsterdam: Elsevier
  110. Gruber J 1994. The incidence of mandated maternity benefits. Am. Econ. Rev. 84:3622–41
    [Google Scholar]
  111. Hahn J, Shi R 2017. Synthetic control and inference Work. Pap Univ. Calif. Los Angeles:
  112. Hahn J, Todd P, van der Klaauw W 2001. Identification and estimation of treatment effects with a regression-discontinuity design. Econometrica 69:1201–9
    [Google Scholar]
  113. Heckman J 1990. Varieties of selection bias. Am. Econ. Rev. 80:2313–18
    [Google Scholar]
  114. Heckman JJ, Ichimura H, Todd P 1998. Matching as an econometric evaluation estimator. Rev. Econ. Stud. 65:2261–94
    [Google Scholar]
  115. Heckman JJ, Robb R 1985. Alternative methods for evaluating the impact of interventions. Longitudinal Analysis of Labor Market Data JJ Heckman, BS Singer 156–246 Cambridge, UK: Cambridge Univ. Press
    [Google Scholar]
  116. Heckman JJ, Urzua S, Vytlacil EJ 2006. Understanding instrumental variables in models with essential heterogeneity. Rev. Econ. Stat. 88:3389–432
    [Google Scholar]
  117. Heckman JJ, Vytlacil EJ 1999. Local instrumental variables and latent variable models for identifying and bounding treatment effects. PNAS 96:84730–34
    [Google Scholar]
  118. Heckman JJ, Vytlacil EJ 2005. Structural equations, treatment effects and econometric policy evaluation. Econometrica 73:3669–738
    [Google Scholar]
  119. Heckman JJ, Vytlacil EJ 2007. Econometric evaluation of social programs, part I: causal models, structural models and econometric policy evaluation. Handbook of Econometrics VI J Heckman, E Leamer 4780–874 Amsterdam: Elsevier
    [Google Scholar]
  120. Hernán MA, Robins JM 2018. Causal Inference Boca Raton, FL: CRC
  121. Hirano K, Imbens GW 2004. The propensity score with continuous treatments. Applied Bayesian Modeling and Causal Inference from Incomplete-Data Perspectives A Gelman, X-L Meng 73–84 New York: Wiley
    [Google Scholar]
  122. Hirano K, Imbens GW, Ridder G 2003. Efficient estimation of average treatment effects using the estimated propensity score. Econometrica 71:41161–89
    [Google Scholar]
  123. Hirano K, Porter JR 2009. Asymptotics for statistical treatment rules. Econometrica 77:51683–701
    [Google Scholar]
  124. Holland PW 1986. Statistics and causal inference. J. Am. Stat. Assoc. 81:396945–60
    [Google Scholar]
  125. Hong H, Nekipelov D 2010. Semiparametric efficiency in nonlinear LATE models. Quant. Econ. 1:2279–304
    [Google Scholar]
  126. Hsiao C, Ching HS, Wan SK 2012. A panel data approach for program evaluation: measuring the benefits of political and economic integration of Hong Kong with mainland China. J. Appl. Econom. 27:5705–40
    [Google Scholar]
  127. Imai K, van Dyk DA 2004. Causal inference with general treatment regimes: generalizing the propensity score. J. Am. Stat. Assoc. 99:467854–66
    [Google Scholar]
  128. Imbens GW 2000. The role of the propensity score in estimating dose-response functions. Biometrika 87:3706–10
    [Google Scholar]
  129. Imbens GW 2003. Sensitivity to exogeneity assumptions in program evaluation. Am. Econ. Rev. 93:2126–32
    [Google Scholar]
  130. Imbens GW 2004. Nonparametric estimation of average treatment effects under exogeneity: a review. Rev. Econ. Stat. 86:14–29
    [Google Scholar]
  131. Imbens GW 2010. Better LATE than nothing: some comments on Deaton (2009) and Heckman and Urzua (2009). J. Econ. Lit. 48:399–423
    [Google Scholar]
  132. Imbens GW 2014. Instrumental variables: an econometrician's perspective. Stat. Sci. 29:3323–58
    [Google Scholar]
  133. Imbens GW, Angrist JD 1994. Identification and estimation of local average treatment effects. Econometrica 62:2467–75
    [Google Scholar]
  134. Imbens GW, Kalyanaraman K 2012. Optimal bandwidth choice for the regression discontinuity estimator. Rev. Econ. Stud. 79:3933–59
    [Google Scholar]
  135. Imbens GW, Kolesár M 2016. Robust standard errors in small samples: some practical advice. Rev. Econ. Stat. 98:4701–12
    [Google Scholar]
  136. Imbens GW, Lemieux T 2008. Regression discontinuity designs: a guide to practice. J. Econom. 142:2615–35
    [Google Scholar]
  137. Imbens GW, Newey WK, Ridder G 2006. Mean-squared-error calculations for average treatment effects Work. Pap. 954748 Soc. Sci. Res. Netw. New York:
  138. Imbens GW, Rubin DB 1997. Estimating outcome distributions for compliers in instrumental variables models. Rev. Econ. Stud. 64:4555–74
    [Google Scholar]
  139. Imbens GW, Rubin DB 2015. Causal Inference in Statistics, Social, and Biomedical Sciences Cambridge, UK: Cambridge Univ. Press
  140. Imbens GW, Wooldridge JM 2009. Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47:15–86
    [Google Scholar]
  141. Jales H, Yu Z 2017. Identification and estimation using a density discontinuity approach. Regression Discontinuity Designs: Theory and Applications MD Cattaneo, JC Escanciano 29–72 Bingley, UK: Emerald Group
    [Google Scholar]
  142. Kang JD, Schafer JL 2007. Demystifying double robustness: a comparison of alternative strategies for estimating a population mean from incomplete data. Stat. Sci. 22:523–39
    [Google Scholar]
  143. Keele LJ, Titiunik R 2015. Geographic boundaries as regression discontinuities. Political Anal 23:1127–55
    [Google Scholar]
  144. Kitagawa T 2015. A test for instrument validity. Econometrica 83:52043–63
    [Google Scholar]
  145. Kitagawa T, Tetenov A 2018. Who should be treated? Empirical welfare maximization methods for treatment choice. Econometrica 86:2591–616
    [Google Scholar]
  146. Kleven HJ 2016. Bunching. Annu. Rev. Econ. 8:435–64
    [Google Scholar]
  147. Kline P, Walters C 2016. Evaluating public programs with close substitutes: the case of Head Start. Q. J. Econ. 131:41795–848
    [Google Scholar]
  148. Koenker RW, Bassett G 1978. Regression quantiles. Econometrica 46:133–50
    [Google Scholar]
  149. Lechner M 2001. Identification and estimation of causal effects of multiple treatments under the conditional independence assumption. Econometric Evaluation of Labour Market Policies M Lechner, F Pfeiffer 43–58 Berlin: Springer
    [Google Scholar]
  150. Lee DS 2008. Randomized experiments from non-random selection in U.S. House elections. J. Econom. 142:2675–97
    [Google Scholar]
  151. Lee DS, Card D 2008. Regression discontinuity inference with specification error. J. Econom. 142:2655–74
    [Google Scholar]
  152. Lee DS, Lemieux T 2010. Regression discontinuity designs in economics. J. Econ. Lit. 48:2281–355
    [Google Scholar]
  153. Lee M-J 2016. Matching, Regression Discontinuity, Difference in Differences, and Beyond Oxford, UK: Oxford Univ. Press
  154. Little RJA, Rubin DB 2002. Statistical Analysis with Missing Data New York: Wiley
  155. Lok JJ 2016. Defining and estimating causal direct and indirect effects when setting the mediator to specific values is not feasible. Stat. Med. 35:224008–20
    [Google Scholar]
  156. Lok JJ, Gill R, Van Der Vaart A, Robins J 2004. Estimating the causal effect of a time-varying treatment on time-to-event using structural nested failure time models. Stat. Neerl. 58:3271–95
    [Google Scholar]
  157. Manski CF 1988. Analog Estimation Methods in Econometrics New York: Chapman & Hall
  158. Manski CF 1990. Nonparametric bounds on treatment effects. Am. Econ. Rev. 80:2319–23
    [Google Scholar]
  159. Manski CF 2008. Identification for Prediction and Decision Cambridge, MA: Harvard Univ. Press
  160. McCrary J 2008. Manipulation of the running variable in the regression discontinuity design: a density test. J. Econom. 142:2698–714
    [Google Scholar]
  161. Morgan SL, Winship C 2015. Counterfactuals and Causal Inference: Methods and Principles for Social Research Cambridge, UK: Cambridge Univ. Press. , 2nd ed..
  162. Mullainathan S, Spiess J 2017. Machine learning: an applied econometric approach. J. Econ. Perspect. 31:287–106
    [Google Scholar]
  163. Murphy SA 2003. Optimal dynamic treatment regimes. J. R. Stat. Soc. Ser. B 65:2331–66
    [Google Scholar]
  164. Newhouse JP 1996. Free for All? Lessons from the RAND Health Insurance Experiment Cambridge, MA: Harvard Univ. Press
  165. Neyman J 1923. On the application of probability theory to agricultural experiments. Essay on principles. Section 9. Stat. Sci. 5:465–80
    [Google Scholar]
  166. Ogburn EL, Rotnitzky A, Robins JM 2015. Doubly robust estimation of the local average treatment effect curve. J. R. Stat. Soc. Ser. B 77:2373–96
    [Google Scholar]
  167. Papay JP, Willett JB, Murnane RJ 2011. Extending the regression-discontinuity approach to multiple assignment variables. J. Econom. 161:2203–7
    [Google Scholar]
  168. Pearl J 1993. Comment: graphical models, causality and intervention. Stat. Sci. 8:3266–69
    [Google Scholar]
  169. Pearl J 2009. Causality: Models, Reasoning and Inference Cambridge, UK: Cambridge Univ. Press
  170. Richardson TS, Robins JM 2013. Single world intervention graphs: a primer Work. Pap Univ. Wash. Seattle:
  171. Robins JM 2000. Marginal Structural Models Versus Structural Nested Models as Tools for Causal Inference Berlin: Springer
  172. Robins JM, Rotnitzky A, Zhao LP 1994. Estimation of regression coefficients when some regressors are not always observed. J. Am. Stat. Assoc. 89:427846–66
    [Google Scholar]
  173. Rosenbaum PR 2002. Observational Studies Berlin: Springer
  174. Rosenbaum PR 2010. Design of Observational Studies Berlin: Springer
  175. Rosenbaum PR, Rubin DB 1983. The central role of the propensity score in observational studies for causal effects. Biometrika 70:141–55
    [Google Scholar]
  176. Rubin DB 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educ. Psychol. 66:5688–701
    [Google Scholar]
  177. Rubin DB 1977. Assignment to treatment group on the basis of a covariate. J. Educ. Stat. 2:11–26
    [Google Scholar]
  178. Rubin DB 1978. Bayesian inference for causal effects: the role of randomization. Ann. Stat. 6:134–58
    [Google Scholar]
  179. Rubin DB 1980. Discussion of “Randomization analysis of experimental data in the Fisher randomization test” by Basu. J. Am. Stat. Assoc. 75:371591–93
    [Google Scholar]
  180. Rubin DB 1990. Formal modes of statistical inference for causal effects. J. Stat. Plan. Inference 25:279–92
    [Google Scholar]
  181. Rubin DB 2005. Causal inference using potential outcomes. J. Am. Stat. Assoc. 100:469322–31
    [Google Scholar]
  182. Rubin DB 2007. The design versus the analysis of observational studies for causal effects: parallels with the design of randomized trials. Stat. Med. 26:20–36
    [Google Scholar]
  183. Sekhon JS, Titiunik R 2016. Understanding regression discontinuity designs as observational studies. Obs. Stud. 2:174–82
    [Google Scholar]
  184. Sekhon JS, Titiunik R 2017. On interpreting the regression discontinuity design as a local experiment. Regression Discontinuity Designs: Theory and Applications MD Cattaneo, JC Escanciano 1–28 Bingley, UK: Emerald Group
    [Google Scholar]
  185. Sloczynski T, Wooldridge JM 2017. A general double robustness result for estimating average treatment effects. Econom. Theory 34:112–33
    [Google Scholar]
  186. Smith JA, Todd PE 2005. Does matching overcome LaLonde's critique of nonexperimental estimators. J. Econom. 125:1305–53
    [Google Scholar]
  187. Spence M 1973. Job market signaling. Q. J. Econ. 87:3355–74
    [Google Scholar]
  188. Stock J, Watson MW 2003. Introduction to Econometrics New York: Prentice Hall
  189. Taddy M, Gardner M, Chen L, Draper D 2016. A nonparametric Bayesian analysis of heterogenous treatment effects in digital experimentation. J. Bus. Econ. Stat. 34:4661–72
    [Google Scholar]
  190. Tamer E 2010. Partial identification in econometrics. Annu. Rev. Econ. 2:167–95
    [Google Scholar]
  191. Thistlethwaite DL, Campbell DT 1960. Regression-discontinuity analysis: an alternative to the ex-post facto experiment. J. Educ. Psychol. 51:6309–17
    [Google Scholar]
  192. Todd PE, Wolpin KI 2010. Ex ante evaluations of social programs. Ann. Écon. Stat. 91:259–86
    [Google Scholar]
  193. Van der Laan MJ, Robins JM 2003. Unified Methods for Censored Longitudinal Data and Causality Berlin: Springer
  194. VanderWeele T 2015. Explanation in Causal Inference: Methods for Mediation and Interaction Oxford, UK: Oxford Univ. Press
  195. Wager S, Athey S 2018. Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. In press
  196. Wooldridge JM 2010. Econometric Analysis of Cross Section and Panel Data Cambridge, MA: MIT Press
  197. Wooldridge JM 2015. Control function methods in applied econometrics. J. Hum. Resourc. 50:2420–45
    [Google Scholar]
  198. Yang S, Imbens GW, Cui Z, Faries DE, Kadziola Z 2016. Propensity score matching and subclassification in observational studies with multi-level treatments. Biometrics 72:41055–65
    [Google Scholar]
  199. Young A 2017. Channelling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results Work. Pap., London School Econ.
/content/journals/10.1146/annurev-economics-080217-053402
Loading
/content/journals/10.1146/annurev-economics-080217-053402
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error