1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Economics
  4. Volume 6, 2014
  5. Article

Annual Review of Economics

Volume 6, 2014

Belief Elicitation in the Laboratory

Abstract

One constraint we face as economists is not being able to observe all the relevant variables required to test our theories or make policy prescriptions. Laboratory techniques allow us to convert many variables (such as beliefs) that are unobservable in the field into observables. This article presents a survey of the literature on belief elicitation in laboratory experimental economics. We discuss several techniques available to elicit beliefs in an incentive-compatible manner and the problems involved in their use. We then look at how successful these techniques have been when employed in laboratory studies. We find that despite some problems, beliefs elicited in the laboratory are meaningful (i.e., they are generally used as the basis for behavior), and the process of eliciting beliefs seems not to be too intrusive. One hope for the future is that by eliciting beliefs, we may be able to develop better theories of belief formation.

Keyword(s): decision theory, experiments
Loading

Article metrics loading...

/content/journals/10.1146/annurev-economics-080213-040927
2014-08-02
2024-04-20
Loading full text...

Full text loading...

/deliver/fulltext/economics/6/1/annurev-economics-080213-040927.html?itemId=/content/journals/10.1146/annurev-economics-080213-040927&mimeType=html&fmt=ahah

Literature Cited

  1. Allen F. 1987. Discovering personal probabilities when utility functions are unknown. Manag. Sci. 33:542–44 [Google Scholar]
  2. Andersen S, Fountain J, Harrison G, Rutstrom E. 2013. Estimating subjective probabilities. J. Risk Uncertain. In press
  3. Armantier O, Bruine de Bruin W, Topa G, van der Klaauw W, Zafar B. 2013. Inflation expectations and behavior: Do survey respondents act on their beliefs? Work. Pap., Fed. Reserve Bank N.Y .
  4. Armantier O, Treich N. 2013. Eliciting beliefs: proper scoring rules, incentives, stakes and hedging. Eur. Econ. Rev. 62::17–40 [Google Scholar]
  5. Battigalli P, Dufwenberg M. 2007. Guilt in games. Am. Econ. Rev. 97:170–76 [Google Scholar]
  6. Battigalli P, Dufwenberg M. 2009. Dynamic psychological games. J. Econ. Theory 144:1–35 [Google Scholar]
  7. Becker GM, DeGroot MH, Marschak J. 1964. Measuring utility by a single-response sequential method. Behav. Sci. 9:226–32 [Google Scholar]
  8. Bellemare C, Sebald A, Strobel M. 2011. Measuring the willingness to pay to avoid guilt: estimation using equilibrium and stated belief models. J. Appl. Econ. 26:437–53 [Google Scholar]
  9. Berg JE, Daley LA, Dickhaut JW, O’Brien JR. 1986. Controlling preferences for lotteries on units of experimental exchange. Q. J. Econ. 101:281–306 [Google Scholar]
  10. Bhatt M, Camerer C. 2005. Self-referential thinking and equilibrium as states of mind in games: fMRI evidence. Games Econ. Behav. 52:424–59 [Google Scholar]
  11. Blanco M, Engelmann D, Koch AK, Normann HT. 2010. Belief elicitation in experiments: Is there a hedging problem?. Exp. Econ. 13:412–38 [Google Scholar]
  12. Blanco M, Engelmann D, Koch AK, Normann HT. 2011. Preferences and beliefs in a sequential social dilemma: a within-subjects analysis. Work. Pap., Univ. Mannheim
  13. Brier GW. 1950. Verification of forecasts expressed in terms of probability. Mon. Weather Rev. 78:1–3 [Google Scholar]
  14. Camerer CF, Ho TH. 1999. Experienced-weighted attraction learning in normal form games. Econometrica 67:827–74 [Google Scholar]
  15. Caplin A, Schotter A. 2008. The Foundations of Positive and Normative Economics New York: Oxford Univ. Press
  16. Charness G, Dufwenberg M. 2006. Promises and partnership. Econometrica 74:1579–601 [Google Scholar]
  17. Cheung YW, Friedman D. 1997. Individual learning in normal form games: some laboratory results. Games Econ. Behav. 19:46–76 [Google Scholar]
  18. Corazzini L, Kube S, Marechal MA. 2007. Towards a behavioral public choice: guilt-aversion and accountability in the lab. Work. Pap. 27, ISLA, Univ. Bocconi, Milan
  19. Costa-Gomes M, Weizsacker G. 2008. Stated beliefs and play in normal-form games. Rev. Econ. Stud. 75:729–62 [Google Scholar]
  20. Croson R. 2000. Thinking like a game theorist: factors affecting the frequency of equilibrium play. J. Econ. Behav. Organ. 41:299–314 [Google Scholar]
  21. Danz DN, Fehr D, Kubler D. 2012. Information and beliefs in a repeated normal-form game. Exp. Econ. 15:622–40 [Google Scholar]
  22. De Finetti B. 1974. Theory of Probability, Vol. 1. New York: Wiley
  23. Delavande A. 2008. Measuring revisions to subjective expectations. J. Risk Uncertain. 36:43–82 [Google Scholar]
  24. Dominitz J, Manski C. 2011. Measuring and interpreting expectations of equity returns. J. Appl. Econ. 26:352–70 [Google Scholar]
  25. Dufwenberg M, Gneezy U. 2000. Measuring beliefs in an experimental lost wallet game. Games Econ. Behav. 30:163–82 [Google Scholar]
  26. El-Gamal MA, Grether DM. 1995. Are people Bayesian? Uncovering behavioral strategies. J. Am. Stat. Assoc. 90:1137–45 [Google Scholar]
  27. Ellingsen T, Johannesson M, Torsvik G, Tjotta S. 2010. Testing guilt aversion. Games Econ. Behav. 68:95–107 [Google Scholar]
  28. Erev I, Bornstein G, Wallsten TS. 1993. The negative effect of probability assessments on decision quality. Organ. Behav. Hum. Decis. Process. 55:78–94 [Google Scholar]
  29. Fudenberg D, Levine D. 1998. Theory of Learning in Games Cambridge, MA: MIT Press
  30. Gächter S, Renner E. 2010. The effects of (incentivized) belief elicitation in public goods experiments. Exp. Econ. 13:364–77 [Google Scholar]
  31. Gneiting T, Raftery AE. 2007. Strictly proper scoring rules, prediction, and estimation. J. Am. Stat. Assoc. 102:359–78 [Google Scholar]
  32. Grether DM. 1981. Financial incentive effects and individual decision making. Soc. Sci. Work. Pap. 401, Calif. Inst. Technol., Pasadena
  33. Guerra G, Zizzo DJ. 2004. Trust responsiveness and beliefs. J. Econ. Behav. Organ. 55:25–30 [Google Scholar]
  34. Gul F, Pesendorfer W. 2008. The case for mindless economics. The Foundations of Positive and Normative Economics, ed. A Caplin, A Shotter, pp. 3–39. New York: Oxford Univ. Press [Google Scholar]
  35. Harrison GW. 1986. An experimental test for risk aversion. Econ. Lett. 21:7–11 [Google Scholar]
  36. Harrison GW, Martínez-Correa J, Swarthout JT. 2013a. Eliciting subjective probabilities with binary lotteries. CEAR Work. Pap., Ga. State Univ .
  37. Harrison GW, Martínez-Correa J, Swarthout JT, Ulm ER. 2013b. Scoring rules for subjective probability distributions. CEAR Work. Pap., Ga. State Univ .
  38. Holt CA, Laury SK. 2002. Risk aversion and incentive effects. Am. Econ. Rev. 92:1644–55 [Google Scholar]
  39. Holt CA, Smith AM. 2009. An update on Bayesian updating. J. Econ. Behav. Organ. 69:125–34 [Google Scholar]
  40. Hossain T, Okui R. 2013. The binarized scoring rule. Rev. Econ. Stud. 80:984–1001 [Google Scholar]
  41. Huck S, Weizsacker G. 2002. Do players correctly estimate what others do? Evidence of conservatism in beliefs. J. Econ. Behav. Organ. 47:71–85 [Google Scholar]
  42. Hyndman K, Ozbay EY, Schotter A, Ehrblatt WZ. 2012a. Belief formation: an experiment with outside observers. Exp. Econ. 15:176–203 [Google Scholar]
  43. Hyndman K, Ozbay EY, Schotter A, Ehrblatt WZ. 2012b. Convergence: an experimental study of teaching and learning in repeated games. J. Eur. Econ. Assoc. 10:573–604 [Google Scholar]
  44. Hyndman K, Terracol A, Vaksmann J. 2013. Beliefs and (in)stability in normal-form games. Work. Pap., Univ. Tex., Dallas
  45. Ivanov A. 2011. Attitudes to ambiguity in one-shot normal-form games: an experimental study. Games Econ. Behav. 71:366–94 [Google Scholar]
  46. Karni E. 2009. A mechanism for eliciting probabilities. Econometrica 77:603–6 [Google Scholar]
  47. Kawagoe T, Narita Y. 2011. Guilt aversion revisited: an experimental test of a new model. SSRN Work. Pap. 1704884
  48. Manski CF. 2002. Identification of decision rules in experiments on simple games of proposal and response. Eur. Econ. Rev. 46:880–89 [Google Scholar]
  49. Manski CF. 2004. Measuring expectations. Econometrica 72:1329–76 [Google Scholar]
  50. Manski CF, Neri C. 2013. First- and second-order subjective expectations in strategic decision-making: experimental evidence. Games Econ. Behav. 81:232–54 [Google Scholar]
  51. Merlo A, Schotter A. 2003. Learning by not doing: an experimental investigation of observational learning. Games Econ. Behav. 42:116–36 [Google Scholar]
  52. Nelson RG, Bessler DA. 1989. Subjective probabilities elicited under proper and improper scoring rules: a laboratory test of predicted responses. Am. J. Agric. Econ. 71:363–69 [Google Scholar]
  53. Nyarko Y, Schotter A. 2001. Comparing learning models with ideal micro-experimental data sets Unpublished manuscript, New York Univ.
  54. Nyarko Y, Schotter A. 2002. An experimental study of belief learning using elicited beliefs. Econometrica 70:971–1005 [Google Scholar]
  55. Offerman T, Sonnemans J. 2004. What’s causing overreaction? An experimental investigation of recency and the hot hand effect. Scand. J. Econ. 106:533–53 [Google Scholar]
  56. Offerman T, Sonnemans J, van de Kuilen G, Wakker PP. 2009. A truth serum for non-Bayesians: correcting proper scoring rules for risk attitudes. Rev. Econ. Stud. 76:1461–89 [Google Scholar]
  57. Palfrey TR, Wang SW. 2009. On eliciting beliefs in strategic games. J. Econ. Behav. Organ. 71:98–109 [Google Scholar]
  58. Qu X. 2012. A mechanism for eliciting a probability distribution. Econ. Lett. 115:399–400 [Google Scholar]
  59. Rey-Biel P. 2009. Equilibrium play and best response to (stated) beliefs in normal form games. Games Econ. Behav. 65:572–85 [Google Scholar]
  60. Roth AE, Malouf MWK. 1979. Game-theoretic models and the role of information in bargaining. Psychol. Rev. 86:574–94 [Google Scholar]
  61. Rutstrom EE, Wilcox NT. 2009. Stated beliefs versus inferred beliefs: a methodological inquiry and experimental test. Games Econ. Behav. 67:616–32 [Google Scholar]
  62. Savage LJ. 1971. Elicitation of personal probabilities and expectations. J. Am. Stat. Assoc. 66:783–801 [Google Scholar]
  63. Schlag KH, van der Weele J. 2013. Eliciting probabilities, means, medians, variances and covariances without assuming risk neutrality. Theor. Econ. Lett. 3:38–42 [Google Scholar]
  64. Schlag KH, Tremewanyvan J, van der Weele J. 2013. A penny for your thoughts: a survey of incentives and belief elicitation. Unpublished manuscript, Univ. Vienna
  65. Schotter A. 2008. What’s so informative about choice?. See Caplin & Schotter 2008, pp. 70–94 [Google Scholar]
  66. Selten R. 1998. Axiomatic characterization of the quadratic scoring rule. Exp. Econ. 1:43–61 [Google Scholar]
  67. Smith CAB. 1961. Consistency in statistical inference and decision. J. R. Stat. Soc. A 23:1–25 [Google Scholar]
  68. Vanberg C. 2008. Why do people keep their promises? An experimental test of two explanations. Econometrica 76:1467–80 [Google Scholar]
  69. Wilcox NT, Feltovich N. 2000. Thinking like a game theorist: comment. Work. Pap., Dep. Econ., Univ. Houston
  70. Winkler RL, Murphy AH. 1970. Nonlinear utility and the probability score. J. Appl. Meteorol. 9:143–48 [Google Scholar]
/content/journals/10.1146/annurev-economics-080213-040927
Loading
Belief Elicitation in the Laboratory
Annual Review of Economics 6, 103 (2014); https://doi.org/10.1146/annurev-economics-080213-040927
/content/journals/10.1146/annurev-economics-080213-040927
/content/journals/10.1146/annurev-economics-080213-040927
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error