1932

Abstract

This article calls attention to some designs in survey experiments that give new leverage in hypothesis testing and validation. The premise of this review is the modesty of survey experiments—modesty of treatment, modesty of scale, modesty of measurement. The focus of this review, accordingly, is the compensating virtues of modesty. With respect to hypothesis testing, I spotlight () cross-category comparisons, () null-by-design experiments, () explication, () conjoint designs, and () sequential factorials. With respect to validation regimes, I discuss () parallel studies, () paired designs, and () splicing. Throughout, the emphasis is on moving from experiment in the singular to experiments in the plural, learning as you go.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-polisci-042716-115726
2018-05-11
2024-12-13
Loading full text...

Full text loading...

/deliver/fulltext/polisci/21/1/annurev-polisci-042716-115726.html?itemId=/content/journals/10.1146/annurev-polisci-042716-115726&mimeType=html&fmt=ahah

Literature Cited

  1. Alves WM, Rossi PH 1978. Who should get what? Fairness judgments of the distribution of earnings. Am. J. Sociol. 84:541–64
    [Google Scholar]
  2. Bansak K, Hainmueller J, Hangartner D 2016. How economic, humanitarian, and religious concerns shape European attitudes toward asylum seekers. Science 354:217–22
    [Google Scholar]
  3. Bansak K, Hainmueller J, Hopkins DJ, Yamamoto T 2017. Beyond the breaking point? Survey satisficing in conjoint experiments Res. Pap. 17-33, Grad. Sch. Bus. Stanford Univ. Stanford, CA:
    [Google Scholar]
  4. Bansak K, Hainmueller J, Hopkins DJ, Yamamoto T 2018. The number of choice tasks and survey satisficing in conjoint experiments. Political Anal 26:112–19
    [Google Scholar]
  5. Bentzen LE, Bjanesoy L, Ivarsflaten E 2017. Patterns of legitimacy on the far right DIGSSCORE Work. Pap. Ser., Univ. Bergen, Nor.
    [Google Scholar]
  6. Chong D, Druckman JN 2007. Framing public opinion. Am. Political Sci. Rev. 101:637–55
    [Google Scholar]
  7. Chong D, Druckman JN 2010. Dynamic public opinion. Am. Political Sci. Rev. 104:663–80
    [Google Scholar]
  8. Citrin J, Green DP, Muste C, Wong C 1997. Public opinion toward immigration reform: the role of economic motivations. J. Politics 59:858–81
    [Google Scholar]
  9. Crowder G 2013. Theories of Multiculturalism Cambridge, UK: Polity
    [Google Scholar]
  10. Druckman JN 2001. On the limits of framing effects: Who can frame?. J. Politics 63:1041–66
    [Google Scholar]
  11. Druckman JN, Peterson E, Slothuus R 2013. How elite partisan polarization affects public opinion formation. Am. Political Sci. Rev. 107:57–79
    [Google Scholar]
  12. Ford R, Morrell G, Heath A 2012. “Fewer but better”? Public attitudes to immigration. British Social Attitudes: The 29th Report A Park, E Clery, J Curtice, M Phillips, D Utting 26–44 London: Nat. Cent. Soc. Res.
    [Google Scholar]
  13. Gamson WA, Modigliani A 1989. Media discourse and public opinion on nuclear power: a constructionist approach. Am. J. Sociol. 95:1–37
    [Google Scholar]
  14. Gilbert DT, King G, Pettigrew S, Wilson TD 2016. Comment on “Estimating the reproducibility of psychological science. .” Science 351:1037
    [Google Scholar]
  15. Green PE, Srinivasan V 1990. Conjoint analysis in marketing: new developments with implications for research and practice. J. Market. 54:3–19
    [Google Scholar]
  16. Grimmer J, Westwood SJ, Messing S 2015. The Impression of Influence Princeton, NJ: Princeton Univ. Press
    [Google Scholar]
  17. Hainmueller J, Hangartners D, Yamamoto T 2015. Validating vignette and conjoint survey experiments against real-world behavior. PNAS 112:2395–400
    [Google Scholar]
  18. Hainmueller J, Hiscox MJ 2010. Attitudes toward highly skilled and low-skilled immigration: evidence from a survey experiment. Am. Political Sci. Rev. 104:61–84
    [Google Scholar]
  19. Hainmueller J, Hopkins DJ 2014. Public attitudes toward immigration. Annu. Rev. Political Sci. 17:225–49
    [Google Scholar]
  20. Hainmueller J, Hopkins DJ, Yamamoto T 2014. Causal inference in conjoint analysis: understanding multidimensional choices via stated preference experiments. Political Anal 22:1–30
    [Google Scholar]
  21. Ivarsflaten E, Sniderman PM 2017. The challenge of inclusion: Muslims in Western Europe DIGGSCORE Work. Pap., Univ. Bergen, Nor.
    [Google Scholar]
  22. Leeper TJ, Slothuus R 2016. If only citizens had a cue: the process of opinion formation over time Work. Pap., London Sch. Econ. Political Sci./Aarhus Univ., London/Aarhus, Den.
    [Google Scholar]
  23. Maniadis Z, Tufano F, List JA 2015. How to make experimental economics research more reproducible: lessons from other disciplines and a new proposal. Replication Exp. Econ. 18:215–30
    [Google Scholar]
  24. Nelson TE, Clawson RA, Oxley ZM 1997. Media framing of a civil liberties conflict and its effect on tolerance. Am. Political Sci. Rev. 91:567–83
    [Google Scholar]
  25. Nelson TE, Kinder DR 1996. Issue frames and group-centrism in American public opinion. J. Politics 58:1055–78
    [Google Scholar]
  26. Open Sci. Collab. 2015. Estimating the reproducibility of psychological science. Science 349:6251aac4716
    [Google Scholar]
  27. Petersen MB, Slothuus R, Stubager R, Togeby L 2011. Freedom for all? The strength and pliability of political tolerance. Br. J. Political Sci. 41:581–97
    [Google Scholar]
  28. Rossi P, Sampson W, Bose CE, Jasso G, Passel J 1974. Measuring household social standing. Soc. Sci. Res. 3:169–90
    [Google Scholar]
  29. Scheve KF, Slaughter MJ 2001. Labor market competition and individual preferences over immigration policy. Rev. Econ. Stat. 83:133–45
    [Google Scholar]
  30. Skocpol T 1992. Protecting Soldiers and Mothers: The Political Origins of Social Policy in the United States Cambridge, MA: Harvard Univ. Press
    [Google Scholar]
  31. Slothuus R 2010. When can political parties lead public opinion? Evidence from a natural experiment. Political Commun 27:158–77
    [Google Scholar]
  32. Sniderman PM, Brody RA, Tetlock PE 1991. Reasoning and Choice: Explorations in Political Psychology New York: Cambridge Univ. Press
    [Google Scholar]
  33. Sniderman PM, Carmines EG 1997. Reaching Beyond Race Cambridge, MA: Harvard Univ. Press
    [Google Scholar]
  34. Sniderman PM, Hagendoorn L 2007. When Ways of Life Collide Princeton, NJ: Princeton Univ. Press
    [Google Scholar]
  35. Sniderman PM, Piazza T 1993. Black Pride and Black Prejudice Princeton, NJ: Princeton Univ. Press
    [Google Scholar]
  36. Sniderman PM, Stiglitz EJ 2012. The Reputational Premium: A Theory of Party Identification and Spatial Reasoning Princeton, NJ: Princeton Univ. Press
    [Google Scholar]
  37. Tomz MR, Weeks JLP 2013. Public opinion and the democratic peace. Am. Political Sci. Rev. 107:849–65
    [Google Scholar]
  38. Waddington CH 1970. Behind Appearances Cambridge, MA: MIT Press
    [Google Scholar]
/content/journals/10.1146/annurev-polisci-042716-115726
Loading
/content/journals/10.1146/annurev-polisci-042716-115726
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error