1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Political Science
  4. Volume 22, 2019
  5. Article

Annual Review of Political Science

Volume 22, 2019

Better Government, Better Science: The Promise of and Challenges Facing the Evidence-Informed Policy Movement

Abstract

Collaborations between the academy and governments promise to improve the lives of people, the operations of government, and our understanding of human behavior and public policy. This review shows that the evidence-informed policy movement consists of two main threads: () an effort to invent new policies using insights from the social and behavioral science consensus about human behavior and institutions and () an effort to evaluate the success of governmental policies using transparent and high-integrity research designs such as randomized controlled trials. We argue that the problems of each approach may be solved or at least well addressed by teams that combine the two. We also suggest that governmental actors ought to want to learn about a new policy works as much as they want to know the policy works. We envision a future evidence-informed public policy practice that () involves cross-sector collaborations using the latest theory plus deep contextual knowledge to design new policies, () applies the latest insights in research design and statistical inference for causal questions, and () is focused on assessing explanations as much as on discovering what works. The evidence-informed public policy movement is a way that new data, new questions, and new collaborators can help political scientists improve our theoretical understanding of politics and also help our policy partners to improve the practice of government itself.

Keyword(s): behavioral insights, evidence-based policy, evidence-informed policy, randomized field experiments
Loading

Article metrics loading...

/content/journals/10.1146/annurev-polisci-050517-124041
2019-05-11
2024-04-25
Loading full text...

Full text loading...

/deliver/fulltext/polisci/22/1/annurev-polisci-050517-124041.html?itemId=/content/journals/10.1146/annurev-polisci-050517-124041&mimeType=html&fmt=ahah

Literature Cited

  1. Angrist J, Autor D, Hudson S, Pallais A 2016. Evaluating post-secondary aid: enrollment, persistence, and projected completion effects Tech. rep., Natl. Bur. Econ. Res. Cambridge, MA:
  2. Banerjee A, Banerji R, Berry J, Duflo E, Kannan H et al. 2017. From proof of concept to scalable policies: challenges and solutions, with an application. J. Econ. Perspect. 31:73–102
     [Google Scholar]
  3. Baumgartner FR, Jones BD 1991. Agenda dynamics and policy subsystems. J. Politics 53:1044–74
     [Google Scholar]
  4. Benartzi S, Beshears J, Milkman KL, Sunstein CR, Thaler RH et al. 2017. Should governments invest more in nudging?. Psychol. Sci. 28:1041–55
     [Google Scholar]
  5. Benartzi S, Thaler RH 2013. Behavioral economics and the retirement savings crisis. Science 339:1152–53
     [Google Scholar]
  6. Beshears J, Choi JJ, Laibson D, Madrian BC 2008. The importance of default options for retirement saving outcomes: evidence from the USA. Lessons from Pension Reform in the Americas 1 SJ Kay, T Sinha 271–307 Oxford, UK: Oxford Univ. Press
     [Google Scholar]
  7. Bettinger EP, Long BT, Oreopoulos P, Sanbonmatsu L 2012. The role of application assistance and information in college decisions: results from the H&R Block FAFSA experiment. Q. J. Econ. 127:1205–42
     [Google Scholar]
  8. Bhatti Y, Dahlgaard JO, Hansen JH, Hansen KM 2015. Getting out the vote with evaluative thinking. Am. J. Eval. 36:389–400
     [Google Scholar]
  9. Bhatti Y, Dahlgaard JO, Hansen JH, Hansen KM 2017. Moving the campaign from the front door to the front pocket: field experimental evidence on the effect of phrasing and timing of text messages on voter turnout. J. Elect. Public Opin. Parties 27:291–310
     [Google Scholar]
  10. Bluhm R, Borgerson K 2011. Evidence-based medicine. Philosophy of Medicine203–38 Amsterdam: Elsevier
     [Google Scholar]
  11. Brady HE 2019. The challenge of big data and data science. Annu. Rev. Political Sci. 22:297–323:
     [Google Scholar]
  12. Brocas I, Carrillo JD 2014. Dual-process theories of decision-making: a selective survey. J. Econ. Psychol. 41:45–54
     [Google Scholar]
  13. Brown CA, Lilford RJ 2006. The stepped wedge trial design: a systematic review. BMC Med. Res. Methodol. 6:54
     [Google Scholar]
  14. Cairney P 2016. The Politics of Evidence-Based Policy Making New York: Palgrave Macmillan
  15. Campbell AL 2003. How Policies Make Citizens: Senior Political Activism and the American Welfare State Princeton, NJ: Princeton Univ. Press
  16. Carter EC, Kofler LM, Forster DE, McCullough ME 2015. A series of meta-analytic tests of the depletion effect: self-control does not seem to rely on a limited resource. J. Exp. Psychol. Gen. 144:796–815
     [Google Scholar]
  17. Cartwright N, Hardie J 2012. Evidence-Based Policy: A Practical Guide to Doing It Better Oxford, UK: Oxford Univ. Press
  18. CASBS. 2018. CASBS in the history of behavioral economics. Center for Advanced Study in Behavioral Science. https://casbs.stanford.edu/news/casbs-history-behavioral-economics
  19. Choi JJ, Laibson D, Madrian BC, Metrick A 2003. Optimal defaults. Am. Econ. Rev. 93:180–85
     [Google Scholar]
  20. Choi Laibson D, Madrian BC, Metrick A 2004. For better or for worse: default effects and 401(k) savings behavior. Perspectives on the Economics of Aging81–126 Chicago: Univ. Chicago Press
     [Google Scholar]
  21. Chong D, Druckman JN 2007. Framing theory. Annu. Rev. Political Sci. 10:103–26
     [Google Scholar]
  22. Comm. Evidence-Based Policymaking. 2017. The promise of evidence-based policymaking Rep. Comm. Evidence-Based Policymaking Washington, DC:
  23. Congdon WJ, Shankar M 2015. The White House social and behavioral sciences team: lessons learned from year one. Behav. Sci. Policy 1:77–86
     [Google Scholar]
  24. Congdon WJ, Shankar M 2018. The role of behavioral economics in evidence-based policymaking. Ann. Am. Acad. Political Soc. Sci. 678:81–92
     [Google Scholar]
  25. Coryn CL, Noakes LA, Westine CD, Schröter DC 2011. A systematic review of theory-driven evaluation practice from 1990 to 2009. Am. J. Eval. 32:199–226
     [Google Scholar]
  26. Damelio R 2016. The Basics of Process Mapping New York: CRC/Productivity Press
  27. De Stefano V 2015. The rise of the just-in-time workforce: on-demand work, crowdwork, and labor protection in the gig-economy. Comp. Labor Law Policy J. 37:461–71
     [Google Scholar]
  28. Deaton A, Cartwright N 2017. Understanding and misunderstanding randomized controlled trials. Soc. Sci. Med. 210:2–21
     [Google Scholar]
  29. Deming D, Dynarski S 2010. College aid. Targeting Investments in Children: Fighting Poverty When Resources Are Limited283–302 Chicago: Univ. Chicago Press
     [Google Scholar]
  30. Dinner I, Johnson EJ, Goldstein DG, Liu K 2011. Partitioning default effects: why people choose not to choose. J. Exp. Psychol. Appl. 17:332–41
     [Google Scholar]
  31. Djulbegovic B, Guyatt GH 2017. Progress in evidence-based medicine: a quarter century on. Lancet 390:415–23
     [Google Scholar]
  32. Dolan P, Hallsworth M, Halpern D, King D, Vlaev I 2010. MINDSPACE: influencing behaviour through public policy Rep., Institute for Government and Cabinet Office https://www.instituteforgovernment.org.uk/sites/default/files/publications/MINDSPACE.pdf
  33. Druckman JN, Green DP, Kuklinski JH, Lupia A 2006. The growth and development of experimental research in political science. Am. Political Sci. Rev. 100:627–35
     [Google Scholar]
  34. Dunning T, Grossman G, Humphreys M, Hyde S, McIntosh C, Nellis G 2018. Metaketa I: information, accountability, and cumulative learning. Evidence in Governance and Politics. http://egap.org/metaketa/metaketa-information-and-accountability
  35. Evans JSBT 2008. Dual-processing accounts of reasoning, judgment, and social cognition. Annu. Rev. Psychol. 59:255–78
     [Google Scholar]
  36. Evans JSBT, Stanovich KE 2013. Dual-process theories of higher cognition: advancing the debate. Perspect. Psychol. Sci. 8:3223–41
     [Google Scholar]
  37. Fisher RA 1925. Statistical Methods for Research Workers Edinburgh/London: Oliver & Boyd
  38. Fisher RA 1935. The Design of Experiments Edinburgh/London: Oliver & Boyd
  39. Freedman DA 1991. Statistical models and shoe leather. Sociol. Methodol. 18:2291–313
     [Google Scholar]
  40. Friese M, Loschelder DD, Gieseler K, Frankenbach J, Inzlicht M 2018. Is ego depletion real? An analysis of arguments. Personal. Soc. Psychol. Rev. https://doi.org/10.1177/1088868318762183
    [Crossref]
  41. Gaines BJ, Kuklinski JH, Quirk PJ 2007. The logic of the survey experiment reexamined. Political Anal 15:1–20
     [Google Scholar]
  42. Gale WG, Iwry JM, Orszag PR, Lucas L 2005. The automatic 401(k): a simple way. Tax Policy Center 401:1207–14
     [Google Scholar]
  43. Garner S, Docherty M, Somner J, Sharma T, Choudhury M et al. 2013. Reducing ineffective practice: challenges in identifying low-value health care using Cochrane systematic reviews. J. Health Services Res. Policy 18:6–12
     [Google Scholar]
  44. Gelman A 2018. The failure of null hypothesis significance testing when studying incremental changes, and what to do about it. Personal. Soc. Psychol. Bull. 44:16–23
     [Google Scholar]
  45. Gerber A, Green DP 2017. Field experiments on voter mobilization: an overview of a burgeoning literature. Handbook of Economic Field Experiments, Vol 1395–438 Amsterdam: North Holland
     [Google Scholar]
  46. Gerber AS, Green DP 2000. The effects of canvassing, telephone calls, and direct mail on voter turnout: a field experiment. Am. Political Sci. Rev. 94:3653–63
     [Google Scholar]
  47. Gerber AS, Green DP 2012. Field Experiments: Design, Analysis, and Interpretation New York: W.W. Norton
  48. Giacomini M 2009. Theory-based medicine and the role of evidence: why the emperor needs new clothes, again. Perspect. Biol. Med. 52:234–51
     [Google Scholar]
  49. Gigerenzer G 2015. On the supposed evidence for libertarian paternalism. Rev. Philos. Psychol. 6:361–83
     [Google Scholar]
  50. Gigerenzer G, Gaissmaier W 2011. Heuristic decision making. Annu. Rev. Psychol. 62:451–82
     [Google Scholar]
  51. Gowdy JM 2008. Behavioral economics and climate change policy. J. Econ. Behav. Organ. 68:632–44
     [Google Scholar]
  52. Gray K 2017. How to map theory: reliable methods are fruitless without rigorous theory. Perspect. Psychol. Sci. 12:731–41
     [Google Scholar]
  53. Guba EG, Lincoln YS 1994. Competing paradigms in qualitative research. Handb. Qual. Res. 2:105
     [Google Scholar]
  54. Gueron JM, Rolston H 2013. Fighting for Reliable Evidence New York: Russell Sage Found.
  55. Hagger MS, Wood C, Stiff C, Chatzisarantis NLD 2010. Ego depletion and the strength model of self-control: a meta-analysis. Psychol. Bull. 136:495–525
     [Google Scholar]
  56. Harris DN, Farmer-Hinton R, Kim D, Diamond J, Reavis TB et al. 2018. The promise of free college (and its potential pitfalls) Rep., Brown Center on Education Policy at Brookings https://www.brookings.edu/research/the-promise-of-free-college-and-its-potential-pitfalls/
  57. Hausman DM, Welch B 2010. Debate: to nudge or not to nudge. J. Political Philos. 18:123–36
     [Google Scholar]
  58. Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ 2015. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. BMJ 350:h391
     [Google Scholar]
  59. Hirschman D 2016. Stylized facts in the social sciences. Sociol. Sci. 3:604–26
     [Google Scholar]
  60. Hu F, Rosenberger WF 2006. The Theory of Response-Adaptive Randomization in Clinical Trials 525 Hoboken, NJ: Wiley
  61. Humphreys M, de la Sierra RS, van der Windt P 2013. Fishing, commitment, and communication: a proposal for comprehensive nonbinding research registration. Political Anal 21:11–20
     [Google Scholar]
  62. Imai K, Ratkovic M 2013. Estimating treatment effect heterogeneity in randomized program evaluation. Ann. Appl. Stat. 7:443–70
     [Google Scholar]
  63. Iyengar S 2011. Laboratory experiments in political science. Cambridge Handbook of Experimental Political Science73–88 Cambridge, UK: Cambridge Univ. Press
     [Google Scholar]
  64. Johnson EJ, Goldstein D 2003. Do defaults save lives?. Science 302:1338–39
     [Google Scholar]
  65. Johnson EJ, Goldstein DG 2013. Decisions by default. The Behavioral Foundations of Public Policy E Shafir 417–27 Princeton, NJ: Princeton Univ. Press
     [Google Scholar]
  66. Kahneman D 2011. Thinking, Fast and Slow New York: Farrar, Straus & Giroux
  67. Kahneman D, Knetsch JL, Thaler RH 1990. Experimental tests of the endowment effect and the Coase Theorem. J. Political Econ. 98:1325–48
     [Google Scholar]
  68. Kahneman D, Tversky A 1979. Prospect theory: an analysis of decision under risk. Econometrica 47:2263–291
     [Google Scholar]
  69. Keren G 2013. A tale of two systems. Perspect. Psychol. Sci. 8:257–62
     [Google Scholar]
  70. Kinder D, Palfrey T 1993. On behalf of an experimental political science. Experimental Foundations of Political Science1–39 Ann Arbor: Univ. Mich. Press
     [Google Scholar]
  71. Kingdon JW 1984. Agendas, Alternatives, and Public Policies New York: Longman, 2nd ed..
  72. Kuklinski JH, Quirk PJ 2000. Reconsidering the rational public: cognition, heuristics, and mass opinion. Elements of Reason: Cognition, Choice, and the Bounds of Rationality A Lupia, MD McCubbins, SL Popkin 153–82 Cambridge, UK: Cambridge Univ. Press
     [Google Scholar]
  73. Kuleshov V, Precup D 2014. Algorithms for multi-armed bandit problems. arXiv:1402.6028
  74. Lin W, Green DP 2016. Standard operating procedures: a safety net for pre-analysis plans. PS Political Sci. Politics 49:3495–500
     [Google Scholar]
  75. Lodge M, Taber CS 2013. The Rationalizing Voter Cambridge, UK: Cambridge Univ. Press
  76. Löfgren Å, Martinsson P, Hennlock M, Sterner T 2012. Are experienced people affected by a pre-set default option—results from a field experiment. J. Environ. Econ. Manag. 63:66–72
     [Google Scholar]
  77. Madrian BC 2014. Applying insights from behavioral economics to policy design. Annu. Rev. Econ. 6:663–88
     [Google Scholar]
  78. Madrian BC, Shea DF 2001. The power of suggestion: inertia in 401(k) participation and savings behavior. Q. J. Econ. 116:1149–87
     [Google Scholar]
  79. Manning WG, Newhouse JP, Duan N, Keeler EB, Leibowitz A 1987. Health insurance and the demand for medical care: evidence from a randomized experiment. Am. Econ. Rev. 77:251–77
     [Google Scholar]
  80. McKenzie CR, Liersch MJ, Finkelstein SR 2006. Recommendations implicit in policy defaults. Psychol. Sci. 17:414–20
     [Google Scholar]
  81. McKiernan EC, Bourne PE, Brown CT, Buck S, Kenall A et al. 2016. How open science helps researchers succeed. eLife 5:e16800
     [Google Scholar]
  82. Mettler S, Soss J 2004. The consequences of public policy for democratic citizenship: bridging policy studies and mass politics. Perspect. Politics 2:55–73
     [Google Scholar]
  83. Morrissey M 2016. The State of American Retirement Washington, DC: Econ. Policy Inst.
  84. Morton RB, Williams KC 2008. Experimentation in political science. The Oxford Handbook of Political Methodology339–56 Oxford, UK: Oxford Univ. Press
     [Google Scholar]
  85. Morton RB, Williams KC 2010. Experimental Political Science and the Study of Causality: From Nature to the Lab New York: Cambridge Univ. Press
  86. Munnell AH, Webb A, Golub-Sass F 2012. The national retirement risk index: an update Rep., Center for Retirement Research, Boston College http://crr.bc.edu/wp-content/uploads/2012/11/IB_12-20-508.pdf
  87. Murphy SA 2003. Optimal dynamic treatment regimes. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 65:331–55
     [Google Scholar]
  88. Newhouse JP, Ins. Exp. Group. 1993. Free for All? Lessons from the RAND Health Insurance Experiment Cambridge, MA: Harvard Univ. Press
  89. Neyman JS 1990 (1923). On the application of probability theory to agricultural experiments. essay on principles, Section 9. Stat. Sci 5:4465–80 Transl. and ed. DM Dabrowska, TP Speed, from Polish
    [Google Scholar]
  90. Nguyen T, Kramer J, Evans B 2018. The effects of grant aid on student persistence and degree attainment: a systematic review and meta-analysis of the causal evidence Work. Pap 18–04 Stanford: Cent. Educ. Policy Anal. Stanford, CA:
  91. OECD. 2017. Behavioural Insights and Public Policy Paris: OECD
  92. Osman M 2004. An evaluation of dual-process theories of reasoning. Psychon. Bull. Rev. 11:988–1010
     [Google Scholar]
  93. Peirce CS, Jastrow J 1885. On small differences in sensation. Memoirs Natl. Acad. Sci. 3:73–83
     [Google Scholar]
  94. Pichert D, Katsikopoulos KV 2008. Green defaults: information presentation and pro-environmental behaviour. J. Environ. Psychol. 28:63–73
     [Google Scholar]
  95. Prell M 2013. Participation in the Supplemental Nutrition Assistance Program (SNAP) and unemployment insurance: How tight are the strands of the recessionary safety net? USDA-ERS Econ. Res. Rep. 157. https://permanent.access.gpo.gov/gpo47447/err157.pdf
  96. Pronin E, Olivola CY, Kennedy KA 2008. Doing unto future selves as you would do unto others: psychological distance and decision making. Personal. Soc. Psychol. Bull. 34:224–36
     [Google Scholar]
  97. Raudenbush SW 1997. Statistical analysis and optimal design for cluster randomized trials. Psychol. Methods 2:173
     [Google Scholar]
  98. Reiss J, Sprenger J 2014. Scientific objectivity. The Stanford Encyclopedia of Philosophy (Winter 2017 Edition) https://plato.stanford.edu/archives/win2017/entries/scientific-objectivity/
    [Google Scholar]
  99. Richburg-Hayes L, Anzelone C, Dechausay N, Landers P 2017. Nudging change in human services: final report on the Behavioral Interventions to Advance Self-Sufficiency (BIAS) Project Tech. Rep. May, Off. Plan. Res. Eval., US Dep. Health Hum. Serv. Washington, DC:
  100. Roberts L, Ahmed I, Davison A 2009. Intercessory prayer for the alleviation of ill health. Cochrane Database Syst. Rev 1:CD000368
     [Google Scholar]
  101. Sackett DL 1997. Evidence-based medicine. Sem. Perinatol 21:13–5
     [Google Scholar]
  102. Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS 1996. Evidence based medicine: What it is and what it isn't. BMJ 312:702371
     [Google Scholar]
  103. Samuelson W, Zeckhauser R 1988. Status quo bias in decision making. J. Risk Uncertainty 1:7–59
     [Google Scholar]
  104. Shafir E 2013. The Behavioral Foundations of Public Policy Princeton, NJ: Princeton Univ. Press
  105. Skocpol T 1995. Protecting Soldiers and Mothers: The Political Origins of Social Policy in United States Cambridge, MA: Harvard Univ. Press
  106. Stanovich KE, West RF 2000. Individual differences in reasoning: implications for the rationality debate?. Behav. Brain Sci. 23:645–65
     [Google Scholar]
  107. Sudman S, Bradburn NM, Schwarz N 1996. Thinking About Answers: The Application of Cognitive Processes to Survey Methodology San Francisco: Jossey-Bass
  108. Sunstein CR 2015. Nudges, agency, and abstraction: a reply to critics. Rev. Philos. Psychol. 6:511–29
     [Google Scholar]
  109. Sunstein CR, Reisch LA 2014. Automatically green: behavioral economics and environmental protection. Harvard Environ. Law Rev. 38:127–58
     [Google Scholar]
  110. Sunstein CR, Thaler RH 2003. Libertarian paternalism is not an oxymoron. Univ. Chicago Law Rev. 70:1159
     [Google Scholar]
  111. Sweller J 1994. Cognitive load theory, learning difficulty, and instructional design. Learn. Instr. 4:295–312
     [Google Scholar]
  112. Thaler RH 2016. Behavioral economics: past, present, and future. Am. Econ. Rev. 106:1577–600
     [Google Scholar]
  113. Thaler RH, Benartzi S 2004. Save More Tomorrow: using behavioral economics to increase employee saving. J. Political Econ. 112:S164–87
     [Google Scholar]
  114. Thaler RH, Ganser L 2015. Misbehaving: The Making of Behavioral Economics New York: W.W. Norton
  115. Thaler RH, Sunstein CR 2003. Libertarian paternalism. Am. Econ. Rev. 93:175–79
     [Google Scholar]
  116. Thaler RH, Sunstein CR 2008. Nudge: Improving Decisions About Health, Wealth, and Happiness New Haven, CT: Yale Univ. Press
  117. Tourangeau R, Rips LJ, Rasinski K 2000. The Psychology of Survey Response Cambridge, UK: Cambridge Univ. Press
  118. Tversky A, Kahneman D 1991. Loss aversion in riskless choice: a reference-dependent model. Q. J. Econ. 106:1039–61
     [Google Scholar]
  119. Wager S, Athey S 2017. Estimation and inference of heterogeneous treatment effects using random forests. J. Am. Stat. Assoc. 113:5231228–42
     [Google Scholar]
  120. Watts DJ 2017. Should social science be more solution-oriented?. Nat. Hum. Behav. 1:0015
     [Google Scholar]
  121. Weiss CH 1997. How can theory-based evaluation make greater headway?. Eval. Rev. 21:501–24
     [Google Scholar]
  122. Weiss CH, Whiting B, Murphy-Graham E, Petrosino A, Allison W, Gandhi G 2008. The fairy godmother—and her warts making the dream of evidence-based policy come true. Am. J. Eval. 29:29–47
     [Google Scholar]
/content/journals/10.1146/annurev-polisci-050517-124041
Loading
  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error