1932

Abstract

This article provides an overview of graded and probabilistic approaches in semantics and pragmatics. These approaches share a common set of core research goals: () a concern with phenomena that are best described as graded, including a vast lexicon of words whose meanings adapt flexibly to the contexts in which they are used, as well as reasoning under uncertainty about interlocutors, their goals, and their strategies; () the need to show that representations are learnable, i.e., that a listener can learn semantic representations and pragmatic reasoning from data; () an emphasis on empirical evaluation against experimental data or corpus data at scale; and () scaling up to the full size of the lexicon. The methods used are sometimes explicitly probabilistic and sometimes not. Previously, there were assumed to be clear boundaries among probabilistic frameworks, classifiers in machine learning, and distributional approaches, but these boundaries have been blurred. Frameworks in semantics and pragmatics use all three of these, sometimes in combination, to address the four core research questions above.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-linguistics-031120-015515
2022-01-14
2024-03-28
Loading full text...

Full text loading...

/deliver/fulltext/linguistics/8/1/annurev-linguistics-031120-015515.html?itemId=/content/journals/10.1146/annurev-linguistics-031120-015515&mimeType=html&fmt=ahah

Literature Cited

  1. Asher N. 2011. Lexical Meaning in Context: A Web of Words Cambridge, UK: Cambridge Univ. Press
  2. Asher N, Van de Cruys T, Abrusán M. 2016. Integrating type theory and distributional semantics: a case study on adjective–noun compositions. Comput. Linguist. 42:4703–25
    [Google Scholar]
  3. Baroni M, Bernardi R, Zamparelli R. 2014. Frege in space: a program for compositional distributional semantics. Linguist. Issues Lang. Technol. 9:65–110
    [Google Scholar]
  4. Baroni M, Lenci A. 2010. Distributional memory: a general framework for corpus-based semantics. Comput. Linguist. 36:4673–721
    [Google Scholar]
  5. Baroni M, Lenci A. 2011. How we BLESSed distributional semantic evaluation. Proceedings of the GEMS 2011 Workshop on Geometrical Models of Natural Language Semantics1–10 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  6. Baroni M, Murphy B, Barbu E, Poesio M 2010. Strudel: a corpus-based semantic model based on properties and types. Cogn. Sci. 34:2222–54
    [Google Scholar]
  7. Baroni M, Zamparelli R. 2010. Nouns are vectors, adjectives are matrices: representing adjective–noun constructions in semantic space. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010)1183–93 Washington, DC: Assoc.Comput. Linguist.
    [Google Scholar]
  8. Barwise J, Perry J. 1999. Situations and Attitudes Stanford, CA: Cent. Study Lang. Inf.
  9. Benz A, Stevens J. 2018. Game-theoretic approaches to pragmatics. Annu. Rev. Linguist. 4:173–91
    [Google Scholar]
  10. Bernardy JP, Blanck R, Chatzikyriakidis S, Lappin S 2018. A compositional Bayesian semantics for natural language. Proceedings of the 1st International Workshop on Language Cognition and Computational Models1–10 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  11. Bernardy JP, Blanck R, Chatzikyriakidis S, Lappin S, Maskharashvili A 2019a. Bayesian inference semantics: a modelling system and a test suite. Proceedings of the 8th Joint Conference on Lexical and Computational Semantics (*SEM 2019)263–72 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  12. Bernardy JP, Blanck R, Chatzikyriakidis S, Lappin S, Maskharashvili A 2019b. Predicates as boxes in Bayesian semantics for natural language. Proceedings of the 22nd Nordic Conference on Computational Linguistics333–37 Turku, Finl: Linköping Univ. Electron. Press
    [Google Scholar]
  13. Blei DM, Ng A, Jordan MI. 2003. Latent Dirichlet allocation. J. Mach. Learn. Res. 3:993–1022
    [Google Scholar]
  14. Boleda G. 2020. Distributional semantics and linguistic theory. Annu. Rev. Linguist. 6:213–34
    [Google Scholar]
  15. Boleda G, Baroni M, Pham TN, McNally L. 2013. Intensionality was only alleged: on adjective–noun composition in distributional semantics. Proceedings of the 10th International Conference on Computational Semantics (IWCS 2013): Long Papers35–46 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  16. Bruni E, Tran NK, Baroni M. 2014. Multimodal distributional semantics. J. Artif. Intell. Res. 49:1–47
    [Google Scholar]
  17. Chersoni E, Santus E, Pannitto L, Lenci A, Blache P, Huang CR. 2019. A structured distributional model of sentence meaning and processing. Nat. Lang. Eng. 25:4483–502
    [Google Scholar]
  18. Church A. 1940. A formulation of the simple theory of types. J. Symb. Log. 5:556–68
    [Google Scholar]
  19. Clark S, Coecke B, Sadrzadeh M 2008. A compositional distributional model of meaning. Proceedings of the 2nd Quantum Interaction Symposium (QI'08)133–40 New York: ACM
    [Google Scholar]
  20. Clark S, Coecke B, Sadrzadeh M 2013. The Frobenius anatomy of relative pronouns. Proceedings of the 13th Meeting on the Mathematics of Language (MoL 13)41–51 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  21. Cohen A. 1999. Generics, frequency adverbs and probability. Linguist. Philos. 22:221–53
    [Google Scholar]
  22. Cohen A 2003. Probabilistic approaches to semantics. Probabilistic Linguistics R Bod, J Hay, S Jannedy 343–79 Cambridge, MA: MIT Press/Bradford
    [Google Scholar]
  23. Cooper R. 2021. From perception to communication: an analysis of meaning and action using a theory of types with records (TTR). Unpubl. Ms. https://github.com/robincooper/ttl/blob/master/ttl.pdf
    [Google Scholar]
  24. Cooper R, Dobnik S, Lappin S, Larsson S. 2015. Probabilistic type theory and natural language semantics. Linguist. Issues Lang. Technol. 10:1–43
    [Google Scholar]
  25. Copestake A. 2009. Slacker semantics: why superficiality, dependency and avoidance of commitment can be the right way to go. Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2009)1–9 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  26. Degen J, Hawkins R, Graf C, Kreiss E, Goodman N 2020. When redundancy is useful: A Bayesian approach to “overinformative” referring expressions. Psychol. Rev. 127:4591–621
    [Google Scholar]
  27. Dinu G, Lapata M. 2010. Measuring distributional similarity in context. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (EMNLP 2010)1162–72 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  28. Emerson G. 2018. Functional distributional semantics: learning linguistically informed representations from a precisely annotated corpus. PhD Thesis Univ. Cambridge Cambridge, UK:
    [Google Scholar]
  29. Emerson G. 2020a. Autoencoding pixies: amortised variational inference with graph convolutions for functional distributional semantics. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics3982–95 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  30. Emerson G. 2020b. Linguists who use probabilistic models love them: quantification in functional distributional semantics. Proceedings of the 2020 Probability and Meaning Conference (PoM 2020)41–52 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  31. Erk K. 2012. Vector space models of word meaning and phrase meaning: a survey. Lang. Linguist. Compass 6:10635–53
    [Google Scholar]
  32. Erk K. 2016. What do you know about an alligator when you know the company it keeps?. Semant. Pragmat. 9:171–63
    [Google Scholar]
  33. Erk K, Herbelot A. 2021. How to marry a star: probabilistic constraints for meaning in context. arXiv:2009.07936 [cs]
  34. Falkum IL, Vicente A 2015. Polysemy: current perspectives and approaches. Lingua 157:1–16
    [Google Scholar]
  35. Fernández R, Larsson S. 2014. Vagueness and learning: a type-theoretic approach. Proceedings of the 3rd Joint Conference on Lexical and Computational Semantics (*SEM 2014)151–59 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  36. Finkelstein L, Gabrilovich E, Matias Y, Rivlin E, Solan Z et al. 2001. Placing search in context: the concept revisited. Proceedings of the 10th World Wide Web Conference406–14 New York: ACM
    [Google Scholar]
  37. Frank MC, Goodman ND. 2012. Predicting pragmatic reasoning in language games. Science 336:6084998–98
    [Google Scholar]
  38. Franke M, Jäger G. 2016. Probabilistic pragmatics, or why Bayes' rule is probably important for pragmatics. Z. Sprachwiss. 35:13–44
    [Google Scholar]
  39. Gärdenfors P. 2004. Conceptual Spaces Cambridge, MA: MIT Press
  40. Gehrke B, McNally L. 2019. Idioms and the syntax/semantics interface of descriptive content versus reference. Linguistics 57:4769–814
    [Google Scholar]
  41. Gerz D, Vulic I, Hill F, Reichart R, Korhonen A 2016. SimVerb-3500: a large-scale evaluation set of verb similarity. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing (EMNLP 2016)2173–82 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  42. Goodman ND, Frank MC. 2016. Pragmatic language interpretation as probabilistic inference. Trends Cogn. Sci. 20:11818–29
    [Google Scholar]
  43. Goodman ND, Lassiter D 2015. Probabilistic semantics and pragmatics: Uncertainty in language and thought. The Handbook of Contemporary Semantic Theory S Lappin, C Fox 655–86 New York: Wiley-Blackwell, 2nd ed..
    [Google Scholar]
  44. Goodman ND, Mansinghka VK, Roy D, Bonawitz K, Tenenbaum JB 2008. Church: a language for generative models. In Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI 2008)220–29 Arlington, VA: AUAI
    [Google Scholar]
  45. Goodman ND, Stuhlmüller A. 2014. The Design and Implementation of Probabilistic Programming Languages. http://dippl.org
  46. Goodman ND, Tenenbaum JB, Gerstenberg T 2015. Concepts in a probabilistic language of thought. The Conceptual Mind: New Directions in the Study of Concepts E Margolis, S Laurence 623–53 Cambridge, MA: MIT Press
    [Google Scholar]
  47. Grefenstette E, Sadrzadeh M 2011. Experimental support for a categorical compositional distributional model of meaning. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP 2011)1394–404 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  48. Grice P 1975. Logic and conversation. Syntax and Semantics 3: Speech Acts P Cole, J Morgan 41–58 New York: Academic
    [Google Scholar]
  49. Hacking I. 2001. An Introduction to Probability and Inductive Logic Cambridge, UK: Cambridge Univ. Press
  50. Herbelot A. 2020. Re-solve it: simulating the acquisition of core semantic competences from small data. Proceedings of the 24th Conference on Computational Natural Language Learning (CoNLL 2020)344–54 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  51. Herbelot A, Copestake A. 2021. Ideal words: a vector-based formalisation of semantic competence. Künst. Intell. 35:271–90
    [Google Scholar]
  52. Hill F, Reichart R, Korhonen A 2015. SimLex-999: evaluating semantic models with (genuine) similarity estimation. Comput. Linguist. 41:4665–95
    [Google Scholar]
  53. Kamp H 1975. Two theories about adjectives. Formal Semantics of Natural Language E Keenan 123–55 Cambridge, UK: Cambridge Univ. Press
    [Google Scholar]
  54. Kintsch W 2007. Meaning in context. Handbook of Latent Semantic Analysis T Landauer, D McNamara, S Dennis, W Kintsch 89–105 Mahwah, NJ: Erlbaum
    [Google Scholar]
  55. Koller D, Friedman N. 2009. Probabilistic Graphical Models: Principles and Techniques. Adapt. Comput. Mach. Learn. Ser. Cambridge, MA: MIT Press
    [Google Scholar]
  56. Larsson S. 2013. Formal semantics for perceptual classification. J. Log. Comput. 25:2335–69
    [Google Scholar]
  57. Lassiter D, Goodman N. 2017. Adjectival vagueness in a Bayesian model of interpretation. Synthese 194:103801–36
    [Google Scholar]
  58. Lenci A. 2008. Distributional semantics in linguistic and cognitive research. Riv. Linguist. 20:11–31
    [Google Scholar]
  59. Lenci A. 2018. Distributional models of word meaning. Annu. Rev. Linguist. 4:151–71
    [Google Scholar]
  60. Liang P, Potts C. 2015. Bringing machine learning and compositional semantics together. Annu. Rev. Linguist. 1:355–76
    [Google Scholar]
  61. Mcclelland J. 2013. Integrating probabilistic models of perception and interactive neural networks: a historical and tutorial review. Front. Psychol. 4:503
    [Google Scholar]
  62. McCready E, Ogata N. 2007. Evidentiality, modality and probability. Linguist. Philos. 30:147–206
    [Google Scholar]
  63. McMahan B, Stone M. 2015. A Bayesian model of grounded color semantics. Trans. Assoc. Comput. Linguist. 3:103–16
    [Google Scholar]
  64. McNally L 2017. Kinds, descriptions of kinds, concepts, and distributions. Bridging Formal and Conceptual Semantics: Selected Papers of BRIDGE-14 K Balogh, W Petersen 39–61 Düsseldorf, Ger: Düsseldorf Univ. Press
    [Google Scholar]
  65. McNally L, Boleda G. 2017. Conceptual versus referential affordance in concept composition. Language, Cognition, and Mind, Vol. 3: Compositionality and Concepts in Linguistics and Psychology J Hampton, Y Winter 245–68 Berlin: Springer
    [Google Scholar]
  66. Merin A. 1999. Information, relevance, and social decisionmaking: some principles and results of decision-theoretic semantics. Logic, Language, and Computation 2 LS Moss, J Ginzburg, M de Rijke 179–221 Stanford, CA: Cent. Study Lang. Inf.
    [Google Scholar]
  67. Monroe W, Hawkins RX, Goodman ND, Potts C. 2017. Colors in context: a pragmatic neural model for grounded language understanding. Trans. Assoc. Comput. Linguist. 5:325–38
    [Google Scholar]
  68. Montague R 1970. English as a formal language. Linguaggi nella societá e nella tecnica B Visentini 189–224 Milan: Ed. Comunitá
    [Google Scholar]
  69. Montague R 1973. The proper treatment of quantification in english. Approaches to Natural Language K Hintikka 221–42 Dordrecht, Neth: Reidel
    [Google Scholar]
  70. Muskens R, Sadrzadeh M. 2018. Static and dynamic vector semantics for lambda calculus models of natural language. J. Lang. Model. 6:2319–51
    [Google Scholar]
  71. Noble B, Breitholz E, Cooper R 2020. Personae under uncertainty: the case of topoi. Proceedings of the 2020 Probability and Meaning Conference (PoM 2020)8–16 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  72. Oaksford M, Chater N. 2007. Bayesian Rationality: The Probabilistic Approach to Human Reasoning Oxford, UK: Oxford Univ. Press
  73. Paperno D, Pham NT, Baroni M. 2014. A practical and linguistically-motivated approach to compositional distributional semantics. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics 1 italicLong Papers90–99 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  74. Parikh P. 2000. Communication, meaning, and interpretation. Linguist. Philos. 23:2185–212
    [Google Scholar]
  75. Pearl J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference San Francisco, CA: Morgan Kaufmann
  76. Pelletier FJ. 2017. Compositionality and concepts—a perspective from formal semantics and philosophy of language. Language, Cognition, and Mind, Vol. 3: Compositionality and Concepts in Linguistics and Psychology J Hampton, Y Winter 31–94 Berlin: Springer
    [Google Scholar]
  77. Qing C, Franke M 2015. Variations on a Bayesian theme: comparing Bayesian models of referential reasoning. Bayesian Natural Language Semantics and Pragmatics H Zeevat, HC Schmitz 201–20 Cham, Switz: Springer
    [Google Scholar]
  78. Sadrzadeh M, Purver M, Hough J, Kempson R. 2018. Exploring semantic incrementality with dynamic syntax and vector space semantics. Proceedings of the 22nd Workshop on the Semantics and Pragmatics of Dialogue (SemDial 2018)122–31 Aix-en-Provence, Fr: AixDial http://semdial.org/anthology/Z18-Sadrzadeh_semdial_0016.pdf
    [Google Scholar]
  79. Schuster A, Stroessner C, Sutton P, Zeevat H 2020. Stochastic frames. Proceedings of the 2020 Probability and Meaning Conference (PoM 2020)78–85 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  80. Solt S. 2015. Vagueness and imprecision: empirical foundations. Annu. Rev. Linguist. 1:107–27
    [Google Scholar]
  81. Sutton P 2015. Towards a probabilistic semantics for vague adjectives. Bayesian Natural Language Semantics and Pragmatics H Zeevat, HC Schmitz 221–46 Cham, Switz: Springer
    [Google Scholar]
  82. Tessler MH, Goodman ND. 2019. The language of generalization. Psychol. Rev. 126:3395–436
    [Google Scholar]
  83. Tuggy D. 1993. Ambiguity, polysemy and vagueness. Cogn. Linguist. 4:2273–90
    [Google Scholar]
  84. van Eijck J, Lappin S 2012. Probabilistic semantics for natural language. Logic and Interactive Rationality Yearbook 2 Z Christoff, P Galeazzi, N Gierasimczuk, A Marcoci, S Smets 17–35 Amsterdam: Amsterdam Dyn. Group
    [Google Scholar]
  85. Vecchi EM, Baroni M, Zamparelli R. 2011. (Linear) maps of the impossible: capturing semantic anomalies in distributional space. Proceedings of the Workshop on Distributional Semantics and Compositionality1–9 Washington, DC: Assoc. Comput. Linguist.
    [Google Scholar]
  86. Venhuizen NJ, Hendriks P, Crocker MW, Brouwer H. 2021. Distributional formal semantics. arXiv:2103.01713 [cs]
  87. White J, Mu J, Goodman N. 2020. Learning to refer informatively by amortizing pragmatic reasoning. Proceedings of the 42nd Annual Meeting of the Cognitive Science Society994–1000 Austin, TX: Cogn. Sci. Soc.
    [Google Scholar]
  88. Zeevat H 2015. Perspectives on Bayesian natural language semantics and pragmatics. Bayesian Natural Language Semantics and Pragmatics H Zeevat, HC Schmitz1–24 Cham, Switz: Springer
    [Google Scholar]
  89. Zeevat H 2019. From semantic memory to semantic content. Language, Logic, and Computation A Silva, S Staton, P Sutton, C Umbach 312–29 Berlin: Springer
    [Google Scholar]
  90. Zeevat H, Grimm S, Hogeweg L, Lestrade S, Smith E 2017. Representing the lexicon. Bridging Formal and Conceptual Semantics: Selected Papers of BRIDGE-14 K Balogh, W Petersen 153–86 Düsseldorf, Ger: Düsseldorf Univ. Press
    [Google Scholar]
/content/journals/10.1146/annurev-linguistics-031120-015515
Loading
/content/journals/10.1146/annurev-linguistics-031120-015515
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error