1932

Abstract

Randomized experiments have become a popular tool in development economics research and have been the subject of a number of criticisms. This paper reviews the recent literature and discusses the strengths and limitations of this approach in theory and in practice. We argue that the main virtue of randomized experiments is that, owing to the close collaboration between researchers and implementers, they allow the estimation of parameters that would not otherwise be possible to evaluate. We discuss the concerns that have been raised regarding experiments and generally conclude that, although real, they are often not specific to experiments. We conclude by discussing the relationship between theory and experiments.

Loading

Article metrics loading...

/content/journals/10.1146/annurev.economics.050708.143235
2009-09-01
2024-05-28
Loading full text...

Full text loading...

/deliver/fulltext/arec/1/1/annurev.economics.050708.143235.html?itemId=/content/journals/10.1146/annurev.economics.050708.143235&mimeType=html&fmt=ahah

Literature Cited

  1. Abadie A. 2002. Bootstrap tests for distributional treatment effects in instrumental variables models. J. Am. Stat. Assoc. 97:(457)28492 [Google Scholar]
  2. Abbring JH, Heckman JJ. 2008. Econometrics evaluation of social programs part III: distributional treatment effects, dynamic treatment effects, dynamic discrete choice and general equilibrium policy evaluation Heckman & Leamers 200851455303
  3. Abdul Latif Jameel Poverty Action Lab (ALJ-PAL) 2005. Fighting Poverty: What Works?, Fall, Issue 1 Cambridge, MA: MIT
  4. Acemoglu D, Angrist J. 2001.How large are human-capital externalities? Evidence from compulsory schooling laws NBER Macroeconomics Annual 2000 BS Bernanke, K Rogoff 15974 Cambridge, MA: NBER
  5. Angrist J, Bettinger E, Bloom E, Kremer M, King E. 2002. Vouchers for private schooling in Colombia: evidence from randomized natural experiments. Am. Econ. Rev. 92:(5)153558 [Google Scholar]
  6. Angrist J, Bettinger E, Kremer M. 2006. Long-term educational consequences of secondary school vouchers: evidence from administrative records in Colombia. Am. Econ. Rev. 96:(3)84762 [Google Scholar]
  7. Angrist J, Lang D, Oreopoulos P. 2009. Incentives and services for college achievement: evidence from a randomized trial. Am. Econ. J. Appl. Econ. 1:13663 [Google Scholar]
  8. Angrist J, Lavy V. 2009. The effect of high school matriculation awards: evidence from group-level randomized trials. Am. Econ. Rev. In press (see also NBER Work. Pap. 9389). [Google Scholar]
  9. Ashraf N, Berry J, Shapiro JM. 2007. Can higher prices stimulate product use? Evidence from a field experiment in Zambia Work. Pap.13247 NBER:
  10. Ashraf N, Karlan D, Yin W. 2006. Tying Odysseus to the mast: evidence from a commitment savings product in the Philippines. Q. J. Econ. 121:(2)63572 [Google Scholar]
  11. Attanasio O, Barr A, Camillo J, Genicot G, Meghir C. 2008a. Group formation and risk pooling in a field experiment Mimeogr.: Georgetown Univ.
  12. Attanasio O, Kugler A, Meghir C. 2008b. Training disadvantaged youth in Latin America: evidence from a randomized trial Work. Pap. Inst. Fisc. Stud.:
  13. Attanasio O, Meghir C, Santiago A. 2001. Education choices in Mexico: using a structural model and a randomized experiment to evaluate Progresa Mimeogr.: Univ. Coll. Lond.
  14. Banaji M. 2001.Implicit attitudes can be measured The Nature of Remembering: Essays in Honor of Robert G. Crowder HL Roediger III, JS Nairne, I Neath, A Surprenant 11750 Washington, DC: Am. Psychol. Assoc.
  15. Banerjee A. 2002. The uses of economic theory: against a purely positive interpretation of theoretical results Work. Pap.007 Dep. Econ., MIT:
  16. Banerjee A. 2005. New development economics and the challenge to theory. Econ. Polit. Wkly. 40:(40)434044 [Google Scholar]
  17. Banerjee A. 2007. Making Aid Work Cambridge, MA: MIT Press
  18. Banerjee A. 2008. Big answers for big questions: the presumption of growth policy Mimeogr.: Dep. Econ., MIT
  19. Banerjee A, Banerji R, Duflo E, Glennerster R, Khemani S. 2009. Pitfalls of participatory programs: evidence from a randomized evaluation in education in India Work. Pap. 14311, NBER; Am. Econ. J: Econ. Policy Forthcoming
  20. Banerjee A, Cole S, Duflo E, Linden L. 2007. Remedying education: evidence from two randomized experiments in India. Q. J. Econ. 122:(3)123564 [Google Scholar]
  21. Banerjee A, Duflo E. 2005.Growth theory through the lens of development economics Handbook of Economic Growth S Durlauf, P Aghion 1A473552 Amsterdam Elsevier Sci. Ltd. North Holl.:
  22. Banerjee A, Duflo E, Glennerster R, Kothari D. 2008. Improving immunization coverage in rural India: a clustered randomized controlled evaluation of immunization campaigns with and without incentives Mimeogr.: Dep. Econ., MIT
  23. Banerjee A, Jacob S, Kremer M, Lanjouw J, Lanjouw P. 2005. Moving to universal education! Costs and trade offs Mimeogr.: Dep. Econ., MIT
  24. Beaman L, Chattopadhyay R, Duflo E, Pande R, Topalova P. 2009. Powerful women: does exposure reduce bias? BREAD Work. Pap. 181; Work. Pap. 14198, NBER; Q. J. Econ. Forthcoming
  25. Berry J. 2008. Child control in education decisions: an evaluation of targeted incentives to learn in India Mimeogr.: Dep. Econ., MIT
  26. Bertrand M, Chugh D, Mullainathan S. 2005. Implicit discrimination. Am. Econ. Rev. 95:(2)948 [Google Scholar]
  27. Bertrand M, Djankov S, Hanna R, Mullainathan S. 2007. Corruption in driving licensing process in Delhi. Q. J. Econ. 122:(4)163976 [Google Scholar]
  28. Bertrand M, Karlan D, Mulainathan S, Zinman J. 2009. What's advertising content worth? Evidence from a consumer credit marketing. Q. J. Econ. Forthcoming [Google Scholar]
  29. Bjorkman M, Svensson J. 2007. Power to the people: evidence from a randomized field experiment of a community-based monitoring project in Uganda. Work. Pap. 6344, CEPR; Q. J. Econ. Forthcoming
  30. Bleakley H. 2007. Disease and development: evidence from hookworm eradication in the American south. Q. J. Econ. 122:(1)73117 [Google Scholar]
  31. Bobonis G, Miguel E, Sharma CP. 2006. Anemia and school participation. J. Hum. Resour. 41:(4)692721 [Google Scholar]
  32. Cohen J, Dupas P. 2007. Free distribution or cost-sharing? Evidence from a randomized malaria prevention experiment Glob.: Work. Pap. 14, Brookings Inst.
  33. Crump R, Hotz J, Imbens G, Mitnik O. 2009. Nonparametric tests for treatment effect heterogeneity. Rev. Econ. Stat. In press [Google Scholar]
  34. Dehejia R. 2005. Program evaluation as a decision problem. J. Econom. 125:14173 [Google Scholar]
  35. de Mel S, McKenzie D, Woodruff C. 2008. Returns to capital in microenterprises: evidence from a field experiment. Q. J. Econ. 123:(4)132972 [Google Scholar]
  36. de Mel S, McKenzie D, Woodruff C. 2009. Are women more credit constrained? Experimental evidence on gender and microenterprise returns. Am. Econ. J. Appl. Econ. Forthcoming [Google Scholar]
  37. Duflo E. 2004a. The medium run consequences of educational expansion: evidence from a large school construction program in Indonesia. J. Dev. Econ. 74:(1)16397 [Google Scholar]
  38. Duflo E. 2004b.Scaling up and evaluation Accelerating Development F Bourguignon, B Pleskovic 34267 Washington, DC: World Bank/Oxford Univ. Press
  39. Duflo E. 2007.Field experiments in development economics Advances in Economic Theory and Econometrics R Blundell, W Newey, T Persson; Econ. Soc. Monogr. 42, chpt. 13 Cambridge, UK Cambridge Univ. Press:
  40. Duflo E, Chattopadhyay R. 2004. Women as policy makers: evidence from a randomized policy experiment in India. Econometrica 72:(5)140943 [Google Scholar]
  41. Duflo E, Dupas P, Kremer M. 2008a. Peer effects, pupil teacher ratios, and teacher incentives: evidence from a randomized evaluation in Kenya Mimeogr.: Dep. Econ., MIT
  42. Duflo E, Dupas P, Kremer M, Sinei S. 2006. Education and HIV/AIDS prevention: evidence from a randomized evaluation in western Kenya Work. Pap. 402, World Bank Policy Res.
  43. Duflo E, Hanna R, Ryan S. 2007. Monitoring works: getting teachers to come to school BREAD: Work. Pap. 103 (Work. Pap. 11880, NBER)
  44. Duflo E, Kremer M. 2004.Use of randomization in the evaluation of development effectiveness Evaluating Development Effectiveness, World Bank Ser. Eval. Dev. Vol. 7 OFeinstein, GK Ingram, GK Pitman 20532 New Brunswick, NJ: Transactions
  45. Duflo E, Kremer M, Glennerster R. 2008b.Using randomization in development economics research: a toolkit Handbook of Development Economics Vol. 4 T Schultz, J Strauss, chpt. 15 Amsterdam Elsevier Sci. Ltd. North Holl.:
  46. Duflo E, Kremer M, Robinson J. 2008c. How high are rates of return to fertilizer? Evidence from field experiments in Kenya. Am. Econ. Rev. Pap. Proc. 98:(2)48288 [Google Scholar]
  47. Duflo E, Kremer M, Robinson J. 2008d. Why are farmers not using fertilizer? Procrastination and learning in technology adoption Mimeogr.: Dep. Econ., MIT
  48. Dupas P. 2007. Relative risks and the market for sex: teenage pregnancy, HIV, and partner selection in Kenya Mimeogr.: Univ. Calif. Los Angeles
  49. Fiszbein A, Schady N. 2009. Conditional Cash Transfers: Reducing Present and Future Poverty Washington, DC World Bank:
  50. Gine X, Karlan D, Zinman J. 2008. Put your money where your butt is: a commitment savings account for smoking cessation Mimeogr.: Dep. Econ., Yale Univ.
  51. Glewwe P, Ilias N, Kremer M. 2003. Teacher incentives Work. Pap., Dep. Econ. Harvard Univ.:
  52. Glewwe P, Kremer M, Moulin S. 2009. Many children left behind? Textbooks and test scores in Kenya. Am. Econ. J. Appl. Econ. 1:11235 [Google Scholar]
  53. Glewwe P, Kremer M, Moulin S, Zitzewitz E. 2004. Retrospective vs. prospective analyses of school inputs: the case of flip charts in Kenya. J. Dev. Econ. 74:(1)25168 [Google Scholar]
  54. Heckman JJ. 1992.Randomization and social policy evaluation Evaluating Welfare and Training Programs C Manski, I Garfinkel 20130 Cambridge, MA: Harvard Univ. Press
  55. Heckman JJ, Ichimura H, Smith J, Todd P. 1998. Characterizing selection bias using experimental data. Econometrica 66:101798 [Google Scholar]
  56. Heckman JJ, Ichimura H, Todd P. 1997a. Matching as an econometric evaluation estimator: evidence from evaluating a job training program. Rev. Econ. Stud. 64:60554 [Google Scholar]
  57. Heckman J, Leamers E. 2008. Handbook of Econometrics Vol. 6B Amsterdam Elsevier Sci. Ltd. North Holl.:1054
  58. Heckman JJ, Lochner L, Taber C. 1999. Human capital formation and general equilibrium treatment effects: a study of tax and tuition policy. Fisc. Stud. 20:(1)2540 [Google Scholar]
  59. Heckman JJ, Smith J, Clements N. 1997b. Making the most out of programme evaluations and social experiments: accounting for heterogeneity in programme impacts. Rev. Econ. Stud. 64:487535 [Google Scholar]
  60. Heckman JJ, Schmierer D, Urzua S. 2010. Testing the correlated random coefficient model. J. Econom. Forthcoming [Google Scholar]
  61. Heckman JJ, Urzua S, Vytlacil EJ. 2006. Understanding instrumental variables in models with essential heterogeneity. Rev. Econ. Stat. 88:(3)389432 [Google Scholar]
  62. Heckman JJ, Vytlacil EJ. 2008a. Econometrics evaluation of social program part I: using the marginal treatment effect to organize alternative economic estimators to evaluate social programs and to forecast their effect in new environment See Heckman & Leamers 20084779874
  63. Heckman JJ, Vytlacil EJ. 2008b. Econometrics evaluation of social program part II: using the marginal treatment effect to organize alternative economic estimators to evaluate social programs and to forecast their effect in new environment Heckman & Leamers 200848755144
  64. Hirano K, Porter J. 2005. Asymptotics for statistical decision rules. Econometrica 71:(5)130738 [Google Scholar]
  65. Hsieh C-T, Urquiola M. 2006. The effects of generalized school choice on achievement and stratification: evidence from Chile's voucher program. J. Public Econ. 90:1477503 [Google Scholar]
  66. Imbens G, Angrist J. 1994. Identification and estimation of local average treatment effects. Econometrica 61:(2)46776 [Google Scholar]
  67. Imbens G, Wooldridge JM. 2008. Recent developments in the econometrics of program evaluation. Mimeogr., Dep. Econ., Harvard Univ.. J. Econ. Lit. Forthcoming [Google Scholar]
  68. Karlan D, Zinman J. 2005. Observing unobservables: identifying information asymmetries with a consumer credit field experiment Work. Pap.94 Dep. Econ., Yale Univ.:
  69. Karlan D, Zinman J. 2007. Expanding credit access: using randomized supply decisions to estimate the impacts Mimeogr.: Dep. Econ., Yale Univ.
  70. Karlan D, Zinman J. 2008. Credit elasticities in less developed countries: implications for microfinance. Am. Econ. Rev. 98:(3)104068 [Google Scholar]
  71. Kremer M, Holla A. 2008. Pricing and access: lessons from randomized evaluation in education and health Mimeogr.: Dep. Econ., Harvard Univ.
  72. Kremer M, Miguel E. 2007. The illusion of sustainability. Q. J. Econ. 122:(3)100765 [Google Scholar]
  73. Kremer M, Miguel E, Thornton R. 2007. Incentives to learn Work. Pap. 10971, NBER; Rev. Econ. Stat. Forthcoming
  74. Manski C. 2000. Identification problems and decisions under ambiguity: empirical analysis of treatment response and normative analysis of treatment choice. J. Econom. 95:41542 [Google Scholar]
  75. Manski C. 2002. Treatment choice under ambiguity induced by inferential problems. J. Stat. Plan. Inference. 105:6782 [Google Scholar]
  76. Manski C. 2004. Statistical treatment rules for heterogeneous populations. Econometrica. 2:(4)122146 [Google Scholar]
  77. McKenzie D, Woodruff C. 2008. Experimental evidence on returns to capital and access to finance in Mexico. World Bank Econ. Rev. 22:(3)45782 [Google Scholar]
  78. Miguel E, Kremer M. 2004. Worms: identifying impacts on education and health in the presence of treatment externalities. Econometrica. 72:(1)159217 [Google Scholar]
  79. Olken B. 2007. Monitoring corruption: evidence from a field experiment in Indonesia. J. Polit. Econ. 115:(2)20049 [Google Scholar]
  80. Rodrik D. 2008. The new development economics: we shall experiment, but how shall we learn? Mimeogr.: Dep. Econ., Harvard Univ.
  81. Rubin D. 2006. Matched Sampling for Causal Effects Cambridge, UK: Cambridge Univ. Press
  82. Topalova P, Duflo E. 2003. Unappreciated service: performance, perceptions, and women leaders in India Mimeogr. MIT:
  83. Thornton R. 2007. The demand for and impact of HIV testing. Evidence from a field experiment. Am. Econ. Rev. 98:(5)182963 [Google Scholar]
  84. Todd P, Wolpin KI. 2006. Using experimental data to validate a dynamic behavioral model of child schooling: assessing the impact of a school subsidy program in Mexico. Am. Econ. Rev. 96:(5)1384417 [Google Scholar]
  85. World Bank 2008. De nouveaux modes de gestion pour accroitre les performances de l'enseignement primaire malgache Work. Pap. World Bank:
/content/journals/10.1146/annurev.economics.050708.143235
Loading
  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error