1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Political Science
  4. Volume 20, 2017
  5. Article

Annual Review of Political Science

Volume 20, 2017

Measuring Public Opinion with Surveys

Abstract

How can we best gauge the political opinions of the citizenry? Since their emergence in the 1930s, opinion polls—or surveys—have become the dominant way to assess the public will. But even given the long history of polling, there is no agreement among political scientists on how to best measure public opinion through polls. This article is a call for political scientists to be more self-conscious about the choices we make when we attempt to measure public opinion with surveys in two realms. I first take up the question of whom to interview, discussing the major challenges survey researchers face when sampling respondents from the population of interest. I then discuss the level of specificity with which we can properly collect information about the political preferences of individuals. I focus on the types of question wording and item aggregation strategies researchers can use to accurately measure public opinion.

Keyword(s): mass behavior, polls, survey research
Loading

Article metrics loading...

/content/journals/10.1146/annurev-polisci-101513-113724
2017-05-11
2025-04-26
Loading full text...

Full text loading...

/deliver/fulltext/polisci/20/1/annurev-polisci-101513-113724.html?itemId=/content/journals/10.1146/annurev-polisci-101513-113724&mimeType=html&fmt=ahah

Literature Cited

  1. Achen CH. 1975. Mass political attitudes and the survey response. Am. Polit. Sci. Rev. 69:41218–31 [Google Scholar]
  2. Allport FH, Hartman DA. 1925. The measurement and motivation of atypical opinion in a certain group. Am. Polit. Sci. Rev. 19:4735–60 [Google Scholar]
  3. Ansolabehere S, Rodden J, Snyder JM Jr. 2008. The strength of issues: using multiple measures to gauge preference stability, ideological constraint, and issue voting. Am. Polit. Sci. Rev. 102:2215–32 [Google Scholar]
  4. Atkinson M, Baumgartner F, Coggins K, Stimson J. 2011. Mood and agendas: developing policy-specific conceptions of mood. Presented at Annu. Meet. Midwest Polit. Sci. Assoc., 69th, Chicago, Mar. 31–Apr. 3
  5. Bafumi J, Herron MC. 2010. Leapfrog representation and extremism: a study of American voters and their members in Congress. Am. Polit. Sci. Rev. 104:3519–42 [Google Scholar]
  6. Baker R, Blumberg SJ, Brick SM, Couper MP, Courtright M. et al. 2010. AAPOR report on online panels. Public Opin. Q. 74:4711–81 [Google Scholar]
  7. Barber MJ, Mann CB, Monson JQ, Patterson KD. 2014. Online polls and registration-based sampling: a new method for pre-election polling. Polit. Anal. 22:31–15 [Google Scholar]
  8. Baron RS, Hoppe SI, Kao CF, Brunsman B, Linneweh B, Rogers D. 1996. Social corroboration and opinion extremity. J. Exp. Soc. Psychol. 32:6537–60 [Google Scholar]
  9. Bartels LM. 2003. Democracy with attitudes. Electoral Democracy G Rabinowitz, M MacKuen 48–82 Ann Arbor: Univ. Mich. Press [Google Scholar]
  10. Berinsky AJ. 2002. Silent voices: social welfare policy opinions and political equality in America. Am. J. Polit. Sci. 46:2276–87 [Google Scholar]
  11. Berinsky AJ. 2004. Silent Voices: Public Opinion and Political Participation in America. Princeton, NJ: Princeton Univ. Press [Google Scholar]
  12. Bishop GF. 2008. Rational public opinion or its manufacture? Reply to Page. Crit. Rev. 20:1–2141–57 [Google Scholar]
  13. Blumberg SJ, Luke JV. 2009. Reevaluating the need for concern regarding noncoverage bias in landline surveys. Am. J. Public Health 99:101806–10 [Google Scholar]
  14. Blumberg SJ, Luke JV. 2016. Wireless substitution: early release of estimates from the National Health Interview Survey, July–December 2015. Natl. Health Interview Surv. Early Release Program 1–13, Natl. Cent. Health Stat., Hyattsville, MD. https://www.cdc.gov/nchs/data/nhis/earlyrelease/wireless201605.pdf
  15. Blumer H. 1954. What is wrong with social theory. ? Am. Soc. Rev. 19:13–10 [Google Scholar]
  16. Brians CL, Wattenberg MP. 1996. Campaign issue knowledge and salience: comparing reception from TV commercials, TV news and newspapers. Am. J. Polit. Sci. 40:1172–93 [Google Scholar]
  17. Brick JM. 2011. The future of survey sampling. Public Opin. Q. 75:5872–88 [Google Scholar]
  18. Brick JM, Dipko S, Presser S, Tucker C, Yuan Y. 2006. Nonresponse bias in a dual frame sample of cell and landline numbers. Public Opin. Q. 70:5780–93 [Google Scholar]
  19. Broockman DE. 2016. Approaches to studying policy representation. Legis. Stud. Q. 41:1181–215 [Google Scholar]
  20. Brulle RJ, Carmichael J, Jenkins JC. 2012. Shifting public opinion on climate change: an empirical assessment of factors influencing concern over climate change in the US, 2002–2010. Clim. Change 114:2169–88 [Google Scholar]
  21. Carmines EG, D'Amico NJ. 2015. The new look in political ideology research. Annu. Rev. Polit. Sci. 18:205–16 [Google Scholar]
  22. Carmines EG, Ensley MJ, Wagner MW. 2012. Political ideology in American politics: one, two, or none. ? Forum 10:34 [Google Scholar]
  23. Clark T. 2015. New research suggests why general election polls were so inaccurate. Guardian 12 Nov: http://www.theguardian.com/politics/2015/nov/13/new-research-general-election-polls-inaccurate [Google Scholar]
  24. Clinton JD, Jackman S, Rivers D. 2004. The statistical analysis of roll call data. Am. Polit. Sci. Rev. 98:2355–70 [Google Scholar]
  25. Converse PE. 1987. Changing conceptions of public opinion in the political process. Public Opin. Q. 51:S12–24 [Google Scholar]
  26. Converse PE. 1990. Popular representation and the distribution of information. Information and Democratic Processes J Ferejohn, J Kuklinski 369–88 Champaign: Univ. Illinois Press [Google Scholar]
  27. De Leeuw E, Callegaro M, Hox J, Korendijk E, Lensvelt-Mulders G. 2007. The influence of advance letters on response in telephone surveys: a meta-analysis. Public Opin. Q. 71:3413–43 [Google Scholar]
  28. Druckman JN, Jacobs LR. 2006. Lumpers and splitters the public opinion information that politicians collect and use. Public Opin. Q. 70:4453–76 [Google Scholar]
  29. Durr RH. 1993. What moves policy sentiment. ? Am. Polit. Sci. Rev. 87:1158–70 [Google Scholar]
  30. Ellis C, Faricy C. 2013. Social policy and public opinion: how the ideological direction of spending influences public mood. J. Polit. 73:41095–110 [Google Scholar]
  31. Ellis C, Stimson JA. 2012. Ideology in America New York: Cambridge Univ. Press [Google Scholar]
  32. Enns PK. 2014. The public's increasing punitiveness and its influence on mass incarceration in the United States. Am. J. Polit. Sci. 58:4857–72 [Google Scholar]
  33. Enns PK, Kellstedt PM. 2008. Policy mood and political sophistication: why everybody moves mood. Br. J. Polit. Sci. 38:3433–54 [Google Scholar]
  34. Enns PK, Koch J. 2013. Public opinion in the U.S. states: 1956 to 2010. State Polit. Policy Q 13:3349–72 [Google Scholar]
  35. Erikson RS. 1978. Analyzing one-variable three-wave panel data: a comparison of two models. Polit. Methodol. 5:2151–67 [Google Scholar]
  36. Erikson RS. 1979. The SRC panel data and mass political attitudes. Br. J. Polit. Sci. 9:189–114 [Google Scholar]
  37. Erikson RS, MacKuen MB, Stimson JA. 2002. The Macro Polity Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  38. Galea S, Tracy M. 2007. Participation rates in epidemiologic studies. Ann. Epidemiol. 17:9643–53 [Google Scholar]
  39. Geer JG. 1996. From Tea Leaves to Opinion Polls: A Theory of Democratic Leadership New York: Columbia Univ. Press [Google Scholar]
  40. Gelman A, Goel S, Rivers D, Rothschild D. 2016. The mythical swing voter. Q. J. Polit. Sci. 11:1103–30 [Google Scholar]
  41. Ginsberg B. 1986. The Captive Public: How Mass Opinion Promotes State Power New York: Basic Books [Google Scholar]
  42. Groves RM. 2006. Nonresponse rates and nonresponse bias in household surveys. Public Opin. Q. 70:5646–75 [Google Scholar]
  43. Groves RM. 2011. Three eras of survey research. Public Opin. Q. 75:5861–71 [Google Scholar]
  44. Groves RM, Fowler FJ Jr., Couper MP, Lepkowski JM, Singer E, Tourangeau R. 2009. Survey Methodology. Hoboken, NJ: Wiley [Google Scholar]
  45. Guterbock TM, Lavrakas PJ, Tompson TN, ZuWallack R. 2011. Cost and productivity ratios in dual-frame RDD telephone surveys. Surv. Pract. 4:21–7 [Google Scholar]
  46. Hacker JS, Pierson P. 2005. Abandoning the middle: the Bush tax cuts and the limits of Democratic control. Perspect. Polit. 3:0133–53 [Google Scholar]
  47. Herbst S. 1993. Numbered Voices: How Opinion Polling Has Shaped American Politics Chicago: Univ. Chicago Press [Google Scholar]
  48. Hillygus DS. 2016. The practice of survey research: changes and challenges. New Directions in Public Opinion A Berinsky 34–53 New York: Routledge, 2nd ed.. [Google Scholar]
  49. Hillygus DS, Jackson N, Young M. 2014. Professional respondents in non-probability online panels. Online Panel Research: A Data Quality Perspective M Callegaro, R Baker, J Bethlehem, AS Göritz, JA Krosnick, PJ Lavrakas 219–37 New York: Wiley [Google Scholar]
  50. Hu SS, Balluz L, Battaglia MP, Frankel MR. 2011. Improving public health surveillance using a dual-frame survey of landline and cell phone numbers. Am. J. Epidemiol. 173:6703–11 [Google Scholar]
  51. Jacoby WG. 2000. Issue framing and public opinion on government spending. Am. J. Polit. Sci. 44:4750–67 [Google Scholar]
  52. Jessee SA. 2009. Spatial voting in the 2004 presidential election. Am. Polit. Sci. Rev. 103:159–81 [Google Scholar]
  53. Jost JT, Federico CM, Napier JL. 2009. Political ideology: its structure, functions, and elective affinities. Annu. Rev. Psychol. 60:307–37 [Google Scholar]
  54. Keeter S. 2006. The impact of cell phone noncoverage bias on polling in the 2004 presidential election. Public Opin. Q. 70:188–98 [Google Scholar]
  55. Keeter S, Dimock M, Christian L. 2008. Calling cell phones in '08 pre-election polls. News release, Pew Res. Cent. People Press Washington, DC: http://www.people-press.org/files/legacy-pdf/cell-phone-commentary.pdf [Google Scholar]
  56. Kellstedt PM. 2000. Media framing and the dynamics of racial policy preferences. Am. J. Polit. Sci. 44:2245–60 [Google Scholar]
  57. Kelly KD, Chambliss WJ. 1966. Status consistency and political attitudes. Am. Sociol. Rev. 31:3375–82 [Google Scholar]
  58. Kennedy C. 2007. Evaluating the effects of screening for telephone service in dual frame RDD surveys. Public Opin. Q. 71:5750–71 [Google Scholar]
  59. Key VO. 1961. Public Opinion and American Democracy New York: Knopf [Google Scholar]
  60. Kieffer KM. 1999. An introductory primer on the appropriate use of exploratory and confirmatory factor analysis. Res. Schools 6:275–92 [Google Scholar]
  61. Kim J, Gershenon C, Glaser P, Smith TW. 2011. The polls—trends: trends in surveys on surveys. Public Opin. Q. 71:5165–91 [Google Scholar]
  62. Kish L. 1965. Survey Sampling. New York: Wiley [Google Scholar]
  63. Klar S. 2014. A multidimensional study of ideological preferences and priorities among the American public. Public Opin. Q. 78:S344–59 [Google Scholar]
  64. Knoke D. 1979. Stratification and the dimensions of American political orientations. Am. J. Polit. Sci. 23:4772–91 [Google Scholar]
  65. Krosnick JA, Boninger DS, Chuang YC, Berent MK, Carnot CG. 1993. Attitude strength: one construct or many related constructs. ? J. Pers. Soc. Psychol. 65:61132–51 [Google Scholar]
  66. Lavrakas PJ, Shuttles CD, Steeh C, Fienberg H. 2007. The state of surveying cell phone numbers in the United States 2007 and beyond. Public Opin. Q. 71:5840–54 [Google Scholar]
  67. Lavrakas PJ, Tompson TN, Benford R, Fleury C. 2010. Investigating data quality in cell phone surveying. Presented at Annu. Conf. Am. Assoc. Public Opin. Res., 65th, Chicago, May 13–16 [Google Scholar]
  68. Lax JR, Phillips JH. 2009. Gay rights in the states: public opinion and policy responsiveness. Am. Polit. Sci. Rev. 103:3367–86 [Google Scholar]
  69. Layman GC, Carsey TM. 2002. Party polarization and “conflict extension” in the American electorate. Am. J. Polit. Sci. 46:4786–802 [Google Scholar]
  70. Levine B, Harter R. 2015. Optimal allocation of cell-phone and landline respondents in dual-frame surveys. Public Opin. Q. 79:191–104 [Google Scholar]
  71. Lohr SL. 2010. Dual frame surveys: recent developments and challenges. Presented at Meet. Italian Stat. Soc., 45th, Padua, Jun. 29-Jul. 1 [Google Scholar]
  72. Lumley T. 2010. Complex Surveys: A Guide to Analysis Using R Hoboken, NJ: Wiley [Google Scholar]
  73. Luttig M. 2013. The structure of inequality and Americans’ attitudes towards redistribution. Public Opin. Q. 77:3811–21 [Google Scholar]
  74. Massey DS, Tourangeau R. 2013. Where do we go from here? Nonresponse and social measurement. Ann. Am. Acad. Polit. Soc. Sci. 645:1222–36 [Google Scholar]
  75. Moore DW. 2008. The Opinion Makers: An Insider Exposes the Truth Behind the Polls. Boston: Beacon [Google Scholar]
  76. Moskowitz AN, Jenkins JC. 2004. Structuring political opinions. Sociol. Q. 45:3395–419 [Google Scholar]
  77. Mueller J. 1973. Wars, Presidents and Public Opinion New York: Wiley [Google Scholar]
  78. Mulligan K, Grant T, Bennett D. 2013. The dynamics of public opinion on cultural policy issues in the US, 1972–2010. Polit. Behav. 35:4807–29 [Google Scholar]
  79. Murphy J, Link MW, Childs JH, Tesfaye CL, Dean E. et al. 2014. Social media in public opinion research: executive summary of the AAPOR Task Force on emerging technologies in public opinion research. Public Opin. Q. 78:4788–94 [Google Scholar]
  80. Newman B. 2003. Integrity and presidential approval, 1980–2000. Public Opin. Q. 67:3335–67 [Google Scholar]
  81. Nicholson-Crotty S, Peterson DA, Ramirez MD. 2009. Dynamic representation(s): federal criminal justice policy and an alternative dimension of public mood. Polit. Behav. 31:4629–55 [Google Scholar]
  82. O'Connor B, Balasubramanyan R, Routledge BR, Smith NA. 2010. From tweets to polls: linking text sentiment to public opinion time series. Presented at Int. Conf. Weblogs and Social Media, 4th, Washington, May 23–26 [Google Scholar]
  83. Page BI, Shapiro RY. 1992. The Rational Public: Fifty Years of Trends in Americans’ Policy Preferences Chicago: Univ. Chicago Press [Google Scholar]
  84. Pan J, Xu Y. 2015. China's ideological spectrum Res. pap., Dep. Polit. Sci., Mass. Inst. Technol. [Google Scholar]
  85. Pew Res. Cent 2012. Assessing the representativeness of public opinion surveys Washington, DC: Pew Res. Cent http://www.people-press.org/2012/05/15/assessing-the-representativeness-of-public-opinion-surveys/ [Google Scholar]
  86. Prior M. 2014. Visual political knowledge: a different road to competence?. J. Polit. 76:0141–57 [Google Scholar]
  87. Schwarz N. 1996. Cognition and Communication: Judgmental Biases, Research Methods, and the Logic of Conversation. Mahwah, NJ: Erlbaum [Google Scholar]
  88. Shapiro RY, Bloch-Elkon Y. 2006. Political polarization and the rational public Presented at Annu. Conf. Am. Assoc. Public Opin. Res., Montreal, 61st, May 18–21 [Google Scholar]
  89. Squire P. 1988. Why the 1936 literary digest poll failed. Public Opin. Q 521125–33 [Google Scholar]
  90. Stimson JA. 1999 (1991). Public Opinion in America: Moods, Cycles, and Swings Boulder, CO: Westview, 2nd ed.. [Google Scholar]
  91. Stimson JA. 2012. On the meaning and measurement of mood. Daedalus 141:423–34 [Google Scholar]
  92. Stimson JA. 2015 (2004). Tides of Consent: How Public Opinion Shapes American Politics Chapel Hill: UNC Press, 2nd ed.. [Google Scholar]
  93. Tabachnick BG, Fidell LS. 1996. Using Multiple Statistics New York: HarperCollins [Google Scholar]
  94. Tausanovitch C, Warshaw C. 2013. Measuring constituent policy preferences in Congress, state legislatures and cities. J. Polit. 75:2330–42 [Google Scholar]
  95. Tourangeau R. 2004. Survey research and societal change. Annu. Rev. Psychol. 55:775–801 [Google Scholar]
  96. Tourangeau R, Rasinski KA. 1988. Cognitive processes underlying context effects in attitude measurement. Psychol. Bull. 103:3299–314 [Google Scholar]
  97. Tourangeau R, Rips LJ, Rasinski K. 2000. The Psychology of Survey Response. Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  98. Treier SD, Hillygus DS. 2009. The nature of political ideology in the contemporary electorate. Public Opin. Q. 73:4679–703 [Google Scholar]
  99. Ura JD. 2014. Backlash and legitimation: macro political responses to Supreme Court decisions. Am. J. Polit. Sci. 58:1110–26 [Google Scholar]
  100. Verba S. 1996. The citizen as respondent: sample surveys and American democracy. Am. Polit. Sci. Rev. 90:11–7 [Google Scholar]
  101. Wilcox C, Norrander B. 2001. Of mood and morals: the dynamics of opinion on abortion and gay rights. Understanding Public Opinion B Norrander, C Wilcox 121–48 Washington, DC: CQ Press [Google Scholar]
  102. Wlezien C. 1995. The public as thermostat: dynamics of preferences for spending. Am. J. Polit. Sci. 39:4981–1000 [Google Scholar]
  103. Zaller J. 1992. The Nature and Origins of Mass Opinion. Cambridge, UK: Cambridge Univ. Press [Google Scholar]
/content/journals/10.1146/annurev-polisci-101513-113724
Loading
Measuring Public Opinion with Surveys
Annual Review of Political Science 20, 309 (2017); https://doi.org/10.1146/annurev-polisci-101513-113724
/content/journals/10.1146/annurev-polisci-101513-113724
/content/journals/10.1146/annurev-polisci-101513-113724
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error