1932

Abstract

Using field experiments, scholars can identify causal effects via randomization while studying people and groups in their naturally occurring contexts. In light of renewed interest in field experimental methods, this review covers a wide range of field experiments from across the social sciences, with an eye to those that adopt virtuous practices, including unobtrusive measurement, naturalistic interventions, attention to realistic outcomes and consequential behaviors, and application to diverse samples and settings. The review covers four broad research areas of substantive and policy interest: first, randomized controlled trials, with a focus on policy interventions in economic development, poverty reduction, and education; second, experiments on the role that norms, motivations, and incentives play in shaping behavior; third, experiments on political mobilization, social influence, and institutional effects; and fourth, experiments on prejudice and discrimination. We discuss methodological issues concerning generalizability and scalability as well as ethical issues related to field experimental methods. We conclude by arguing that field experiments are well equipped to advance the kind of middle-range theorizing that sociologists value.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-soc-073014-112445
2017-07-31
2024-03-29
Loading full text...

Full text loading...

/deliver/fulltext/soc/43/1/annurev-soc-073014-112445.html?itemId=/content/journals/10.1146/annurev-soc-073014-112445&mimeType=html&fmt=ahah

Literature Cited

  1. Abascal M. 2015. Us and them: black–white relations in the wake of Hispanic population growth. Am. Sociol. Rev. 80:789–813 [Google Scholar]
  2. Adida CL, Laitin DD, Valfort MA. 2016. Why Muslim Integration Fails in Christian-Heritage Societies Cambridge, MA: Harvard Univ. Press
  3. Ahmed AM, Hammarstedt M. 2008. Discrimination in the rental housing market: a field experiment on the Internet. J. Urban Econ. 64:362–72 [Google Scholar]
  4. Ahmed AM, Hammarstedt M. 2009. Detecting discrimination against homosexuals: evidence from a field experiment on the Internet. Economica 76:599–97 [Google Scholar]
  5. Arceneaux K, Nickerson DW. 2009. Who is mobilized to vote? A re-analysis of 11 field experiments. Am. J. Political Sci. 53:1–16 [Google Scholar]
  6. Attanasio O, Augsburg B, De Haas R, Fitzsimons E, Harmgart H. 2012. Group lending or individual lending? Evidence from a randomised field experiment in Mongolia. Work. Pap. No. 136, Eur. Bank Reconstr. Dev.
  7. Attanasio O, Pellerano L, Reyes SP. 2009. Building trust? Conditional cash transfer programmes and social capital. Fiscal Stud. 30:139–77 [Google Scholar]
  8. Avdeenko A, Gilligan MG. 2015. International interventions to build social capital: evidence from a field experiment in Sudan. Am. Political Sci. Rev. 109:427–49 [Google Scholar]
  9. Ayres I, Siegelman P. 1995. Race and gender discrimination in bargaining for a new car. Am. Econ. Rev. 85:304–21 [Google Scholar]
  10. Baldassarri D. 2015. Cooperative networks: altruism, group solidarity, and reciprocity in Ugandan farmer organizations. Am. J. Sociol. 121:355–95 [Google Scholar]
  11. Baldassarri D. 2016. Prosocial behavior across communities: evidence from a nationwide lost-letter experiment Presented at Advances with Field Experiments Conf., Sept. 16, Univ Chicago:
  12. Banerjee A, Bertrand M, Datta S, Mullainathan S. 2009. Labor market discrimination in Delhi: evidence from a field experiment. J. Comp. Econ. 37:14–27 [Google Scholar]
  13. Banerjee A, Duflo E. 2009. The experimental approach to development economics. Annu. Rev. Econ. 1:151–78 [Google Scholar]
  14. Banerjee A, Duflo E. 2011. Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty. New York: Public Affairs
  15. Banerjee A, Duflo E, Glennerster R, Kothari D. 2010a. Improving immunization coverage in rural India: Clustered randomized controlled immunisation campaigns with and without incentives. Br. Med. J.340:c2220
  16. Banerjee A, Duflo E, Glennerster R, Kinnan C. 2010b. The miracle of microfinance? Evidence from a randomized evaluation. Work. Pap. No. 13-09, Dep. Econ., MIT
  17. Barr A. 2003. Trust and expected trustworthiness: experimental evidence from Zimbabwean villages. Econ. J. 113:614–30 [Google Scholar]
  18. Bauchet J, Marshall C, Starita L, Thomas J, Yalouris A. 2011. Latest findings from randomized evaluations of microfinance. Access Finance Forum Rep. 2:1–27 [Google Scholar]
  19. Beath A, Christia F, Enikolopov R. 2013. Empowering women: evidence from a field experiment in Afghanistan. Am. Political Sci. Rev. 107:540–57 [Google Scholar]
  20. Benson PL, Karabenick SA, Lerner RM. 1976. Pretty pleases: the effects of physical attractiveness, race, and sex on receiving help. J. Exp. Soc. Psychol. 12:409–15 [Google Scholar]
  21. Benz M, Meier S. 2008. Do people behave in experiments as in the field? Evidence from donations. Exp. Econ. 11:278–81 [Google Scholar]
  22. Bertrand M, Karlan D, Mullainathan S, Shafir E, Zinman J. 2010. What's advertising content worth? Evidence from a consumer credit marketing field experiment. Q. J. Econ. 125:263–306 [Google Scholar]
  23. Bertrand M, Mullainathan S. 2004. Are Emily and Greg more employable than Lakisha and Jamal? A field experiment on labor market discrimination. Am. Econ. Rev. 94:991–1013 [Google Scholar]
  24. Besbris M, Faber JW, Rich P, Sharkey P. 2015. Effect of neighborhood stigma on economic transitions. PNAS 112:4994–98 [Google Scholar]
  25. Bettinger EP. 2012. Paying to learn: the effect of financial incentives on elementary school test scores. Rev. Econ. Stat. 94:686–98 [Google Scholar]
  26. Bigoni M, Bortolotti S, Casari M, Gambetta D, Pancotto F. 2016. Amoral familism, social capital, or trust? The behavioural foundations of the Italian north–south divide. Econ. J. 126:1318–41 [Google Scholar]
  27. Blommaert L, Coenders M, van Tubergen F. 2014. Discrimination of Arabic-named applicants in the Netherlands: an Internet-based field experiment examining different phases in online recruitment procedures. Soc. Forces 92:957–82 [Google Scholar]
  28. Bond RM, Fariss CJ, Jones JJ, Kramer AD, Marlow C. et al. 2012. A 61-million-person experiment in social influence and political mobilization. Nature 489:295–98 [Google Scholar]
  29. Bosch M, Carnero MA, Farré L. 2010. Information and discrimination in the rental housing market: evidence from a field experiment. Reg. Sci. Urban Econ. 40:11–19 [Google Scholar]
  30. Brearley HC. 1931. Experimental sociology in the United States. Soc. Forces 10:196–99 [Google Scholar]
  31. Butler DM, Broockman DE. 2011. Do politicians racially discriminate against constituents? A field experiment on state legislators. Am. J. Political Sci. 55:463–77 [Google Scholar]
  32. Butler DM, Nickerson DW. 2011. Can learning constituency opinion affect how legislators vote? Results from a field experiment. Q. J. Political Sci. 6:55–83 [Google Scholar]
  33. Camerer C. 2003. Behavioral Game Theory: Experiments in Strategic Interaction New York, NY: Russell Sage Found.
  34. Cardenas J, Carpenter J. 2008. Behavioural development economics: lessons from field labs in the developing world. J. Dev. Stud. 44:337–64 [Google Scholar]
  35. Casey K, Glennerster R, Miguel E. 2012. Reshaping institutions: evidence on external aid and local collective action. Q. J. Econ. 127:1755–812 [Google Scholar]
  36. Castilla EJ, Benard S. 2010. The paradox of meritocracy in organizations. Adm. Sci. Q. 55:543–676 [Google Scholar]
  37. Centola D. 2010. The spread of behavior in an online social network experiment. Science 329:1194–97 [Google Scholar]
  38. Charness G, Gneezy U. 2009. Incentives to exercise. Econometrica 77:909–31 [Google Scholar]
  39. Chetty R, Hendren N, Katz LF. 2015. The effects of exposure to better neighborhoods on children: new evidence from the moving to opportunity experiment. Work. Pap. 21156, NBER, Cambridge, MA
  40. Chong D, Junn J. 2011. Politics from the perspective of minority populations. Cambridge Handbook of Experimental Political Science JN Druckman, DP Green, JH Kuklinski, A Lupia, 602–33 Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  41. Cialdini RB, Ascani K. 1976. Test of a concession procedure for inducing verbal, behavioral, and further compliance with a request to give blood. J. Pers. Soc. Psychol. 61:295–300 [Google Scholar]
  42. Cialdini RB, Vincent JE, Lewis SK, Catalan J, Wheeler D, Darby BL. 1975. Reciprocal concessions procedure for inducing compliance: the door-in-the-face technique. J. Pers. Soc. Psychol. 31:206–15 [Google Scholar]
  43. Clampet-Lundquist S, Massey DS. 2008. Neighborhood effects on economic self-sufficiency: a reconsideration of the Moving to Opportunity experiment. Am. J. Sociol. 114:107–43 [Google Scholar]
  44. Cohen J, Dupas P. 2010. Free distribution or cost-sharing? Evidence from a randomized malaria prevention experiment. Q. J. Econ. 125:1–40 [Google Scholar]
  45. Cole S, Giné X, Tobacman J, Topalova P, Townsend R, Vickery J. 2013. Barriers to household risk management: evidence from India. Am. Econ. J. Appl. Econ. 5:104–35 [Google Scholar]
  46. Cook TD, Shadish WR. 1994. Social experiments: some developments over the past fifteen years. Annu. Rev. Psychol. 45:545–80 [Google Scholar]
  47. Correll SJ, Benard S, Paik I. 2007. Getting a job: is there a motherhood penalty?. Am. J. Sociol. 112:1297–339 [Google Scholar]
  48. Cox D. 1958. Planning of Experiments New York: Wiley
  49. Crépon B, Devoto F, Duflo E, Parienté W. 2011. Impact of microcredit in rural areas of Morocco: evidence from a randomized evaluation. Work. Pap., Dep. Econ., MIT
  50. Cross H, Kenney GM, Mell J, Zimmerman W. 1990. Employer hiring practices: differential treatment of Hispanic and Anglo job seekers. Tech. rep., Urban Inst., Washington, DC
  51. Deaton A. 2010. Instruments, randomization, and learning about development. J. Econ. Lit. 48:424–55 [Google Scholar]
  52. Dehejia R, Pop-Eleches C, Samii C. 2015. From local to global: external validity in a fertility natural experiment. Work. Pap. 21459, NBER, Cambridge, MA
  53. Doob AN, Gross AE. 1968. Status as an inhibitor of horn-honking responses. J. Soc. Psychol. 76:213–18 [Google Scholar]
  54. Druckman JN, Green DP, Kuklinski JH, Lupia A. 2011. Cambridge Handbook of Experimental Political Science Cambridge, UK: Cambridge Univ. Press
  55. Duflo E, Kremer M, Robinson J. 2008. How high are rates of return to fertilizer? Evidence from field experiments in Kenya. Am. Econ. Rev. 98:482–88 [Google Scholar]
  56. Duflo E, Kremer M, Robinson J. 2011. Nudging farmers to use fertilizer: theory and experimental evidence from Kenya. Am. Econ. Rev. 101:2350–90 [Google Scholar]
  57. Dunn EW, Aknin LB, Norton MI. 2008. Spending money on others promotes happiness. Science 319:1687–88 [Google Scholar]
  58. Dunning T. 2012. Natural Experiments in the Social Sciences: A Design-Based Approach Cambridge, UK: Cambridge Univ. Press
  59. Dupas P. 2009. What matters (and what does not) in households’ decision to invest in malaria prevention?. Am. Econ. Rev. 99:224–30 [Google Scholar]
  60. Dupas P. 2011. Do teenagers respond to HIV risk information? Evidence from a field experiment in Kenya. Am. Econ. J. Appl. Econ. 3:1–34 [Google Scholar]
  61. Dupas P. 2014. Short-run subsidies and long-run adoption of new health products: evidence from a field experiment. Econometrica 82:197–228 [Google Scholar]
  62. Dupas P, Robinson J. 2011. Savings constraints and microenterprise development: evidence from a field experiment in Kenya. Work. Pap. 14693, NBER, Cambridge, MA
  63. Emswiller T, Deaux K, Willits JE. 1971. Similarity, sex, and requests for small favors. J. Appl. Soc. Psychol. 1:284–91 [Google Scholar]
  64. Enos RD. 2014. Causal effect of intergroup contact on exclusionary attitudes. PNAS 111:3699–704 [Google Scholar]
  65. Enos RD, Fowler A, Vavreck L. 2014. Increasing inequality: the effect of GOTV mobilization on the composition of the electorate. J. Polit. 76:273–88 [Google Scholar]
  66. Fearon JD, Humphreys M, Weinstein JM. 2009. Can development aid contribute to social cohesion after civil war? Evidence from a field experiment in post-conflict Liberia. Am. Econ. Rev. 99:287–91 [Google Scholar]
  67. Fearon JD, Humphreys M, Weinstein JM. 2015. How does development assistance affect collective action capacity? Results from a field experiment in post-conflict Liberia. Am. J. Political Sci. 109:450–69 [Google Scholar]
  68. Fershtman C, Gneezy U. 2001. Discrimination in a segmented society: an experimental approach. Q. J. Econ. 116:351–77 [Google Scholar]
  69. Fisher RA. 1935. The Design of Experiments New York: Hafner
  70. Fiszbein A, Schady N. 2009. Conditional cash transfers: reducing present and future poverty. World Bank Policy Res. Rep., World Bank Washington, DC:
  71. Forbes GB, Gromoll HF. 1971. The lost letter technique as a measure of social variables: some exploratory findings. Soc. Forces 50:113–15 [Google Scholar]
  72. Freedman JL, Fraser SC. 1966. Compliance without pressure: the foot-in-the-door technique. J. Pers. Soc. Psychol. 4:195–202 [Google Scholar]
  73. Freese J, Peterson D. 2017. Replication in social science. Annu. Rev. Sociol. 43. In press
  74. Fryer R. 2011. Financial incentives and student achievement: evidence from randomized trials. Q. J. Econ. 126:1755–98 [Google Scholar]
  75. Gaddis SM. 2015. Discrimination in the credential society: an audit study of race and college selectivity in the labor market. Soc. Forces 93:1451–79 [Google Scholar]
  76. Gaddis SM, Ghoshal R. 2015. Arab American housing discrimination, ethnic competition, and the contact hypothesis. Ann. Am. Acad. Political Soc. Sci. 660:282–99 [Google Scholar]
  77. Galster G, Constantine P. 1991. Discrimination against female-headed households in rental housing: theory and exploratory evidence. Rev. Soc. Econ. 49:76–100 [Google Scholar]
  78. Gantner L. 2007. PROGRESA: An integrated approach to poverty alleviation in Mexico. Case Studies in Food Policy for Developing Countries: Policies for Health, Nutrition, Food Consumption, and Poverty P Pinstrup-Andersen, F Cheng, Vol 1211–20 Ithaca, NY: Cornell Univ. Press [Google Scholar]
  79. Garfinkel H. 1967. Studies in Ethnomethodology Englewood Cliffs, NJ: Prentice-Hall
  80. Gelman A. 2014. Experimental reasoning in social science. Field Experiments and Their Critics: Essays on the Uses and Abuses of Experimentation in the Social Sciences DL Teele 185–95 New Haven, CT: Yale Univ. Press [Google Scholar]
  81. Gerber AS. 2011. Field experiments in political science. Cambridge Handbook of Experimental Political Science JN Druckman, DP Green, JH Kuklinski, A Lupia 115–38 Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  82. Gerber AS, Green DP. 2000. The effects of canvassing, telephone calls, and direct mail on voter turnout: a field experiment. Am. Political Sci. Rev. 94:653–63 [Google Scholar]
  83. Gerber AS, Green DP. 2012. Field Experiments New York: Norton
  84. Gerber AS, Green DP, Larimer CW. 2008. Social pressure and voter turnout: evidence from a large scale field experiment. Am. Political Sci. Rev. 102:33–48 [Google Scholar]
  85. Gerber AS, Green DP, Shachar R. 2003. Voting may be habit-forming: evidence from a randomized field experiment. Am. J. Political Sci. 47:540–50 [Google Scholar]
  86. Gil-White F. 2004. Ultimatum game with an ethnicity manipulation: results from Kohvdiin Bulgan Sum, Mongolia. Foundations of Human Sociality: Economic Experiments and Ethnographic Evidence from Fifteen Small-Scale Societies J Henrich, R Boyd, S Bowles, C Camerer, E Fehr, H Gintis, 260–304 Oxford, UK: Oxford Univ. Press [Google Scholar]
  87. Gilligan MJ, Pasquale BJ, Samii C. 2014. Civil war and social cohesion: lab-in-the-field evidence from Nepal. Am. J. Political Sci. 58:604–19 [Google Scholar]
  88. Giné X, Karlan D. 2014. Group versus individual liability: short and long term evidence from Philippine-microcredit lending groups. J. Dev. Econ. 107:65–83 [Google Scholar]
  89. Giné X, Karlan D, Zinman J. 2010. Put your money where your butt is: a commitment contract for smoking cessation. Am. Econ. J. Appl. Econ.213–35
  90. Gneezy U, List J, Price MK. 2012. Toward an understanding of why people discriminate: evidence from a series of natural field experiments. Work. Pap. 17855, NBER, Cambridge, MA
  91. Gneezy U, Meier S, Rey-Biel P. 2011. When and why incentives (don't) work to modify behavior. J. Econ. Perspect. 25:191–210 [Google Scholar]
  92. Gneezy U, Rey-Biel P. 2014. On the relative efficiency of performance pay and noncontingent incentives. J. Eur. Econ. Assoc. 12:62–72 [Google Scholar]
  93. Gneezy U, Rustichini A. 2000. A fine is a price. J. Legal Stud. 29:1–17 [Google Scholar]
  94. Goel V. 2014. Facebook tinkers with users’ emotions in news feed experiment, stirring outcry. New York Times, June 30 B1
  95. Gosnell HF. 1927. Getting Out the Vote: An Experiment in the Stimulation of Voting Chicago: Chicago Univ. Press
  96. Green DP, Gerber A. 2008. Get Out the Vote: How to Increase Voter Turnout Washington, DC: Brookings Inst. Press. 2nd ed.
  97. Green DP, Wong J. 2009. Tolerance and the contact hypothesis: a field experiment. The Political Psychology of Democratic Citizenship228–46 Oxford, UK: Oxford Univ. Press [Google Scholar]
  98. Greenberg D, Shroder M. 2004. The Digest of Social Experiments. Washington, DC: Urban Inst. Press
  99. Grose CR. 2014. Field experimental work on political institutions. Annu. Rev. Political Sci. 17:355–70 [Google Scholar]
  100. Grossman G, Baldassarri D. 2012. The impact of elections on cooperation: evidence from a lab in the field experiment in Uganda. Am. J. Political Sci. 56:964–85 [Google Scholar]
  101. Grossman G, Paler L. 2015. Using experiments to study political institutions. Handbook of Comparative Political Institutions J Gandhi, R Ruiz-Rufino 84–97 London: Routledge [Google Scholar]
  102. Habyarimana J, Humphreys M, Posner DN, Weinstein JM. 2009. Coethnicity: Diversity and the Dilemmas of Collective Action New York: Russell Sage Found.
  103. Harrison GW. 2013. Field experiments and methodological intolerance. J. Econ. Methodol. 20:103–17 [Google Scholar]
  104. Harrison GW, List JA. 2004. Field experiments. J. Econ. Lit. 42:1009–55 [Google Scholar]
  105. Hausman JA, Wise DA. 1985. Social Experimentation Chicago: Chicago Univ. Press
  106. Heckman JJ. 1992. Randomization and social policy evaluation. Evaluating Welfare and Training Programs CF Manski, I Garfinkel 201–30 Cambridge, MA: Harvard Univ. Press [Google Scholar]
  107. Heckman JJ. 1998. Detecting discrimination. J. Econ. Perspect. 12:101–16 [Google Scholar]
  108. Heckman JJ, Siegelman P. 1993. The Urban Institute audit studies: their methods and findings. Clear and Convincing Evidence: Measurement of Discrimination in America M Fix, RJ Struyk 187–258 Washington, DC: Urban Inst. Press [Google Scholar]
  109. Henrich J, Boyd R, Bowles S, Camerer C, Fehr E. et al. 2001. In search of homo economicus: behavioral experiments in 15 small-scale societies. Am. Econ. Rev. 91:73–78 [Google Scholar]
  110. Henrich J, Ensminger J, McElreath R, Barr A, Barrett C. et al. 2010. Markets, religion, community size, and the evolution of fairness and punishment. Science 327:1480–84 [Google Scholar]
  111. Henrich J, McElreath R, Barr A, Ensminger J, Barrett C. et al. 2006. Costly punishment across human societies. Science 312:1767–70 [Google Scholar]
  112. Henry PJ. 2008. College sophomores in the laboratory redux: influences of a narrow data base on social psychology's view of the nature of prejudice. Psychol. Inq. 19:49–71 [Google Scholar]
  113. Herberich DH, List JA, Price MK. 2011. How many economists does it take to change a light bulb? A natural field experiment on technology adoption Work. Pap., Univ. Chicago
  114. Heyman J, Ariely D. 2004. Effort for payment: a tale of two markets. Psychol. Sci. 15:787–93 [Google Scholar]
  115. Holland J, Silva AS, Mace R. 2012. Lost letter measure of variation in altruistic behaviour in 20 neighbourhoods. PLOS ONE 7:e43294 [Google Scholar]
  116. Houlette MA, Gaertner SL, Johnson KM, Banker BS, Riek BM, Dovidio JF. 2004. Developing a more inclusive social identity: an elementary school intervention. J. Soc. Issues 60:35–55 [Google Scholar]
  117. Humphreys M, Sanchez de la Sierra R, van der Windt P. 2013. Fishing, commitment, and communication: a proposal for comprehensive nonbinding research registration. Polit. Anal. 21:1–20 [Google Scholar]
  118. Imbens G, Wooldridge J. 2009. Recent developments in the econometrics of program evaluation. J. Econ. Lit. 47:5–86 [Google Scholar]
  119. Isen AM, Levin PF. 1972. Effect of feeling good on helping: cookies and kindness. J. Pers. Soc. Psychol. 21:384–88 [Google Scholar]
  120. Jackson M, Cox DR. 2013. The principles of experimental design and their application in sociology. Annu. Rev. Sociol. 39:27–49 [Google Scholar]
  121. Jensen R, Miller N. 2008. Giffen behavior and subsistence consumption. Am. Econ. Rev. 98:1553–77 [Google Scholar]
  122. Kamenica E. 2012. Behavioral economics and psychology of incentives. Annu. Rev. Econ. 4:427–52 [Google Scholar]
  123. Karlan D. 2005. Using experimental economics to measure social capital and predict financial decisions. Am. Econ. Rev. 95:1688–99 [Google Scholar]
  124. Karlan D, Appel J. 2011. More Than Good Intentions: Improving the Ways the World's Poor Borrow, Save, Farm, Learn, and Stay Healthy New York: Penguin
  125. Karlan D, Goldberg N. 2011. Microfinance evaluation strategies: notes on methodology and findings. The Handbook of Microfinance B Armendáriz, M Labie 17–58 London: World Scientific [Google Scholar]
  126. Karlan D, McConnell M, Mullainathan S, Zinman J. 2014. Getting to the top of mind: how reminders increase saving. Manag. Sci. 62:3393–3411 [Google Scholar]
  127. Karlan D, Osei-Akoto I, Osei R, Udry C. 2010. Examining underinvestment in agriculture: measuring returns to capital and insurance. Work. Pap., Abdul Latif Jameel Poverty Action Lab. https://www.poverty-action.org/sites/default/files/Panel3-3-Farmers-Returns-Capital.pdf
  128. Karlan D, Zinman J. 2011. Microcredit in theory and practice: using randomized credit scoring for impact. Science 332:1278–84 [Google Scholar]
  129. Keizer K, Lindenberg S, Steg L. 2008. The spreading of disorder. Science 322:1681–85 [Google Scholar]
  130. Kelly E, Moena P, Oakes J, Fan W, Okechukwu C. et al. 2014. Changing work and work-family conflict: evidence from the work, family, and health network. Am. Sociol. Rev. 79:485–516 [Google Scholar]
  131. Kling JR, Liebman JB, Katz LF. 2007. Experimental analysis of neighborhood effects. Econometrica 75:83–119 [Google Scholar]
  132. Kotran A. 2015. Opower and utility partners save over eight terawatt-hours of energy power and utility partners save over eight terawatt-hours of energy. News release, May 21
  133. Kramer ADI, Guillory JE, Hancock JT. 2014. Experimental evidence of massive-scale emotional contagion through social networks. PNAS 111:8788–90 [Google Scholar]
  134. Kremer M. 2003. Randomized evaluations of educational programs in developing countries: some lessons. Am. Econ. Rev. 93:102–6 [Google Scholar]
  135. Kremer M, Brannen C, Glennerster R. 2013. The challenge of education and learning in the developing world. Science 340:297–300 [Google Scholar]
  136. Kremer M, Leino J, Miguel E, Zwane AP. 2011. Spring cleaning: rural water impacts, valuation, and property rights institutions. Q. J. Econ. 126:145–205 [Google Scholar]
  137. Kugelmass H. 2016. “Sorry, I'm not accepting new patients”: an audit study of access to mental health care. J. Health Soc. Behav. 57:168–83 [Google Scholar]
  138. Lacetera N, Macis M. 2010. Do all material incentives for pro-social activities backfire? The response to cash and non-cash incentives for blood donations. J. Econ. Psychol. 31:738–48 [Google Scholar]
  139. Lacetera N, Macis M, Slonim R. 2013. Economic rewards to motivate blood donations. Science 340:927–28 [Google Scholar]
  140. Landry CE, Lange A, List JA, Price MK, Rupp NG. 2010. Is a donor in hand better than two in the bush? Evidence from a natural field experiment. Am. Econ. Rev. 100:958–83 [Google Scholar]
  141. Langer EJ, Rodin J. 1976. The effects of choice and enhanced responsibility for the aged: a field experiment in an institutional setting. J. Pers. Soc. Psychol. 34:191–98 [Google Scholar]
  142. Lauster N, Easterbrook A. 2011. No room for new families? A field experiment measuring rental discrimination against same-sex couples and single parents. Soc. Probl. 58:389–409 [Google Scholar]
  143. Leuven E, Oosterbeek H, van der Klaauw B. 2010. The effect of financial rewards on students’ achievement: evidence from a randomized experiment. J. Eur. Econ. Assoc. 8:1243–65 [Google Scholar]
  144. Levine M, Prosser A, Evans D, Reicher S. 2005. Identity and emergency intervention: how social group membership and inclusiveness of group boundaries shape helping behavior. Pers. Soc. Psychol. Bull. 31:443–53 [Google Scholar]
  145. Levitt SD, List JA. 2009. Field experiments in economics: the past, the present, and the future. Eur. Econ. Rev. 53:1–18 [Google Scholar]
  146. Levitt SD, List JA, Neckerman S, Sadoff S. 2012. The behavioralist goes to school: leveraging behavioral economics to improve educational performance. Work. Pap. 18165, NBER Cambridge, MA:
  147. List JA. 2007. Field experiments: a bridge between lab and naturally occurring data. B.E. J. Econ. Anal. Policy 5:2 [Google Scholar]
  148. Lucas JW. 2003. Theory-testing, generalization, and the problem of external validity. Sociol. Theory 21:236–53 [Google Scholar]
  149. Ludwig J, Duncan GJ, Gennetian LA, Katz LF, Kessler RC. et al. 2013. Long-term neighborhood effects on low-income families: evidence from moving to opportunity. Am. Econ. Rev. 103:226–31 [Google Scholar]
  150. Ludwig J, Liebman JB, Kling JR, Duncan GJ, Katz LF. et al. 2008. What can we learn about neighborhood effects from the moving to opportunity experiment?. Am. J. Sociol. 114:144–88 [Google Scholar]
  151. Marwell G, Ames RE. 1979. Experiments on the provision of public goods: resources, interest, group size, and the free-rider problem. Am. J. Sociol. 84:1335–60 [Google Scholar]
  152. Massey DS, Lundy G. 2001. Use of Black English and racial discrimination in urban housing markets: new methods and findings. Urban Aff. Rev. 36:452–69 [Google Scholar]
  153. McDermott R. 2011. Internal and external validity. Cambridge Handbook of Experimental Political Science JN Druckman, DP Green, JH Kuklinski, A Lupia, 27–40 Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  154. McEwan PJ. 2015. Improving learning in primary schools of developing countries: a meta-analysis of randomized experiments. Rev. Educ. Res. 85:353–94 [Google Scholar]
  155. McNutt M. 2015. Editorial retraction of Lacour & Green. Science 346:1366–69 Science 348:1100 [Google Scholar]
  156. Merton RK. 1945. Sociological theory. Am. J. Sociol. 50:462–73 [Google Scholar]
  157. Michelson M, Nickerson DW. 2011. Voter Mobilization Cambridge, UK: Cambridge Univ. Press
  158. Miguel E, Kremer M. 2004. Worms: identifying impacts on education and health in the presence of treatment externalities. Econometrica 72:159–217 [Google Scholar]
  159. Milgram S, Liberty HJ, Toledo R, Wackenhut J. 1986. Response to intrusion into waiting lines. J. Pers. Soc. Psychol. 51:683–89 [Google Scholar]
  160. Milgram S, Mann L, Hartner S. 1965. The lost letter technique: a tool of social research. Public Opin. Q. 29:437–38 [Google Scholar]
  161. Milkman KL, Akinola M, Chugh D. 2015. What happens before? A field experiment exploring how pay and representation differentially shape bias on the pathway into organizations. J. Appl. Psychol. 100:1678–712 [Google Scholar]
  162. Milkman KL, Beshears J, Choi JJ, Laibson D, Madrian BC. 2011. Using implementation intentions prompts to enhance influenza vaccination rates. PNAS 108:10415–20 [Google Scholar]
  163. Morgan S, Winship C. 2007. Counterfactuals and Causal Inference Cambridge, UK: Cambridge Univ. Press
  164. Morton R, Williams K. 2010. Experimental Political Science and the Study of Causality Cambridge, UK: Cambridge Univ. Press
  165. Moss-Racusin CA, Dovidio JF, Brescoll V, Graham MJ, Handelsman J. 2012. Science faculty's subtle gender biases favor male students. PNAS 109:16474–79 [Google Scholar]
  166. Munnell AH. 1986. Lessons from the Income Maintenance Experiments Boston: Fed. Res. Bank of Boston
  167. Mutz DC. 2011. Population-Based Survey Experiments Princeton, NJ: Princeton Univ. Press
  168. Nagda BRA, Tropp LR, Paluck EL. 2006. Looking back as we look ahead: integrating research, theory, and practice on intergroup relations. J. Soc. Issues 62:439–51 [Google Scholar]
  169. Neumark D, Bank RJ, Nort KDV. 1996. Sex discrimination in restaurant hiring: an audit study. Q. J. Econ. 111:915–41 [Google Scholar]
  170. Nickerson DW. 2008. Is voting contagious? Evidence from two field experiments. Am. Political Sci. Rev. 102:49–57 [Google Scholar]
  171. Nolan JM, Kenefick J, Schultz PW. 2011. Normative messages promoting energy conservation will be underestimated by experts unless you show them the data. Soc. Influence 6:169–80 [Google Scholar]
  172. Nolan JM, Schultz PW, Cialdini RB, Goldstein NJ, Griskevicius V. 2008. Normative social influence is underdetected. Pers. Soc. Psychol. Bull. 34:913–23 [Google Scholar]
  173. Nosek B, Aarts A, Anderson J, Anderson C, Attridge P. et al. 2015a. Estimating the reproducibility of psychological science. Science 349:943–51 [Google Scholar]
  174. Nosek B, Alter G, Banks G, Borsboom D, Bowman S. et al. 2015b. Promoting an open research culture. Science 348:1422–25 [Google Scholar]
  175. Olken B. 2007. Monitoring corruption: evidence from a field experiment in Indonesia. J. Political Econ. 115:200–49 [Google Scholar]
  176. Olken B. 2010. Direct democracy and local public goods: evidence from a field experiment in Indonesia. Am. Political Sci. Rev. 104:243–67 [Google Scholar]
  177. Pager D. 2003. The mark of a criminal record. Am. J. Sociol. 108:937–75 [Google Scholar]
  178. Pager D. 2007. The use of field experiments for studies of employment discrimination: contributions, critiques, and directions for the future. Ann. Am. Acad. Political Soc. Sci. 609:104–33 [Google Scholar]
  179. Pager D, Quillian L. 2005. Walking the talk: what employers say versus what they do. Am. Sociol. Rev. 70:355–80 [Google Scholar]
  180. Pager D, Western B, Bonikowski B. 2009. Discrimination in a low-wage labor market: a field experiment. Am. Sociol. Rev. 74:777–99 [Google Scholar]
  181. Paluck EL. 2009. Reducing intergroup prejudice and conflict using the media: a field experiment in Rwanda. Interpers. Relat. Group Process. 96:574–87 [Google Scholar]
  182. Paluck EL, Cialdini RB. 2014. Field research methods. Handbook of Research Methods in Social and Personality Psychology HT Reis, CM Judd 81–97 New York: Cambridge Univ. Press, 2nd ed.. [Google Scholar]
  183. Paluck EL, Green DP. 2009. Prejudice reduction: what works? A review and assessment of research and practice. Annu. Rev. Psychol. 60:339–67 [Google Scholar]
  184. Paluck EL, Shepherd H. 2012. The salience of social referents: a field experiment on collective norms and harassment behavior in a school social network. J. Pers. Soc. Psychol. 103:899–915 [Google Scholar]
  185. Paluck EL, Shepherd H, Aronow PM. 2016. Changing climates of conflict: a social network driven experiment in 56 schools. PNAS 113:566–71 [Google Scholar]
  186. Pedulla DS. 2016. Penalized or protected? Gender and the consequences of non-standard and mismatched employment histories. Am. Sociol. Rev. 81:262–89 [Google Scholar]
  187. Pettigrew TF. 1998. Intergroup contact theory. Annu. Rev. Psychol. 49:65–85 [Google Scholar]
  188. Riach PA, Rich J. 2002. Field experiments of discrimination in the market place. Econ. J. 112:480–518 [Google Scholar]
  189. Rodríguez-Planas N. 2012. Longer-term impacts of mentoring, educational services, and learning incentives: evidence from a randomized trial in the United States. Am. Econ. J. Appl. Econ. 4:121–39 [Google Scholar]
  190. Rondeau D, List JA. 2008. Matching and challenge gifts to charity: evidence from laboratory and natural field experiments. Exp. Econ. 11:253–67 [Google Scholar]
  191. Ross SL, Turner MA. 2005. Housing discrimination in metropolitan America: explaining changes between 1989 and 2000. Soc. Probl. 52:152–80 [Google Scholar]
  192. Rossi PH, Berk RA, Lenihan KJ. 1980. Money, Work, and Crime: Experimental Evidence New York: Academic Press
  193. Rossi PH, Berk RA, Lenihan KJ. 1982. Saying it wrong with figures: a comment on Zeisel. Am. J. Sociol. 88:390–93 [Google Scholar]
  194. Rossi PH, Lyall KC. 1978. An overview evaluation of the NIT experiment. Eval. Stud. Rev. 3:412–28 [Google Scholar]
  195. Sabin N. 2015. Modern microfinance: a field in flux. Social Finance Nicholls A, Paton R, Emerson J Oxford, UK: Oxford Univ. Press [Google Scholar]
  196. Salganik MJ, Dodds PS, Watts DJ. 2006. Experimental study of inequality and unpredictability in an artificial cultural market. Science 311:854–56 [Google Scholar]
  197. Sampson RJ. 2008. Moving to inequality: neighborhood effects and experiments meet social structure. Am. J. Sociol. 114:189–231 [Google Scholar]
  198. Sampson RJ. 2012. Great American City: Chicago and the Enduring Neighborhood Effect Chicago, IL: Chicago Univ. Press
  199. Schuler SR, Hashemi SM, Badal SH. 1998. Men's violence against women in rural Bangladesh: undermined or exacerbated by microcredit programmes?. Dev. Pract. 8:148–57 [Google Scholar]
  200. Schultz P. 2004. School subsidies for the poor: evaluating the Mexican Progresa poverty program. J. Dev. Econ. 74:199–250 [Google Scholar]
  201. Shadish WR, Cook TD. 2009. The renaissance of field experimentation in evaluating interventions. Annu. Rev. Psychol.607–29
  202. Shadish WR, Cook TD, Campbell DT. 2002. Experimental and Quasi-experimental Designs for Generalized Causal Inference. New York: Houghton, Mifflin and Company
  203. Simpson BT, McGrimmon T, Irwin K. 2007. Are blacks really less trusting than whites? Revisiting the race and trust question. Soc. Forces 86:525–52 [Google Scholar]
  204. Sniderman PM, Grob DB. 1996. Innovations in experimental design in attitude surveys. Annu. Rev. Sociol. 22:377–99 [Google Scholar]
  205. Steinpreis RE, Anders KA, Ritzke D. 1999. The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: a national empirical study. Sex Roles 41:509–28 [Google Scholar]
  206. Stutzer A, Goette L, Zehnder M. 2011. Active decisions and prosocial behaviour: a field experiment on blood donations. Econ. J. 121:476–93 [Google Scholar]
  207. Teele DL. 2014. Reflections on the ethics of field experiments. Field Experiments and Their Critics: Essays on the Uses and Abuses of Experimentation in the Social Sciences DL Teele 115–40 New Haven, CT: Yale Univ. Press [Google Scholar]
  208. Thornton RL. 2008. The demand for, and impact of, learning HIV status. Am. Econ. Rev. 98:1829–63 [Google Scholar]
  209. Tilcsik A. 2011. Pride and prejudice: employment discrimination against openly gay men in the United States. Am. J. Sociol. 117:586–626 [Google Scholar]
  210. Travers J, Milgram S. 1969. An experimental study of the small world problem. Sociometry 32:425–43 [Google Scholar]
  211. Turner MA, Bednarz BA, Herbig C, Lee SJ. 2003. Discrimination in metropolitan housing markets phase 2: Asians and Pacific Islanders Tech. rep., Urban Inst., Washington, DC
  212. Turner MA, Fix M, Struyk RJ. 1991. Opportunities Denied, Opportunities Diminished: Racial Discrimination in Hiring Washington, DC: Urban Inst. Press
  213. Turner MA, Ross SL, Galster GC, Yinger J. 2002. Discrimination in metropolitan housing markets: national results from phase 1 of the Housing Discrimination Study (HDS) Tech. rep., Urban Inst Washington, DC:
  214. Van Bavel JJ, Mende-Siedlecki P, Brady WJ, Reinero DA. 2016. Contextual sensitivity in scientific reproducibility. PNAS 113:6454–59 [Google Scholar]
  215. Van de Rijt A, Kang SM, Restivo M, Patil A. 2014. Field experiments of success-breeds-success dynamics. PNAS 111:6934–39 [Google Scholar]
  216. Van Der Merwe WG, Burns J. 2008. What's in a name? Racial identity and altruism in post-apartheid South Africa. South Afr. J. Econ. 76:266–75 [Google Scholar]
  217. Vermeersch C, Kremer M. 2005. School Meals, Educational Achievement, and School Competition: Evidence from a Randomized Evaluation. New York: World Bank
  218. Volpp KG, Troxel AB, Pauly MV, Glick HA, Puig A. et al. 2009. A randomized, controlled trial of financial incentives for smoking cessation. N. Engl. J. Med. 360:699–709 [Google Scholar]
  219. Whitt S, Wilson RK. 2007. The dictator game, fairness and ethnicity in postwar Bosnia. Am. J. Political Sci. 51:655–68 [Google Scholar]
  220. Wienk RE, Reid CE, Simonson JC, Eggers FJ. 1979. Measuring racial discrimination in American housing markets: the housing market practices survey. Tech. Rep. HUD-PDR-444(2), Dep. Hous. Urban Dev Washington, DC:
  221. Williams WM, Ceci SJ. 2015. National hiring experiments reveal 2:1 faculty preference for women on STEM tenure track. PNAS 112:5360–65 [Google Scholar]
  222. Yamagishi T. 2011. Trust: The Evolutionary Game of Mind and Society New York: Springer
  223. Yamagishi T, Cook KS, Watabe M. 1998. Uncertainty, trust, and commitment formation in the United States and Japan. Am. J. Sociol. 104:165–94 [Google Scholar]
  224. Zeisel H. 1982. Disagreement over the evaluation of a controlled experiment. Am. J. Sociol. 88:378–89 [Google Scholar]
  225. Zelditch M. 2007. External validity of experiments that test theories. Laboratory Experiments in the Social Sciences M Webster, J Sell 87–112 New York: Elsevier [Google Scholar]
/content/journals/10.1146/annurev-soc-073014-112445
Loading
/content/journals/10.1146/annurev-soc-073014-112445
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error