1932

Abstract

Computational semantics has long been considered a field divided between logical and statistical approaches, but this divide is rapidly eroding with the development of statistical models that learn compositional semantic theories from corpora and databases. This review presents a simple discriminative learning framework for defining such models and relating them to logical theories. Within this framework, we discuss the task of learning to map utterances to logical forms (semantic parsing) and the task of learning from denotations with logical forms as latent variables. We also consider models that use distributed (e.g., vector) representations rather than logical ones, showing that these can be considered part of the same overall framework for understanding meaning and structural complexity.

Loading

Article metrics loading...

/content/journals/10.1146/annurev-linguist-030514-125312
2015-01-14
2024-04-12
Loading full text...

Full text loading...

/deliver/fulltext/linguistics/1/1/annurev-linguist-030514-125312.html?itemId=/content/journals/10.1146/annurev-linguist-030514-125312&mimeType=html&fmt=ahah

Literature Cited

  1. AnderBois S, Brasoveanu A, Henderson R. 2012. The pragmatics of quantifier scope: a corpus study. Proceedings of Sinn und Bedeutung 16 Aguilar-Guevara A, Chernilovskaya A, Nouwen R. 115–28 Cambridge, MA: MIT Work. Pap. Linguist. [Google Scholar]
  2. Artzi Y, Zettlemoyer LS. 2011. Bootstrapping semantic parsers from conversations. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing421–32 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  3. Artzi Y, Zettlemoyer LS. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. Trans. Assoc. Comput. Linguist. 1:49–62 [Google Scholar]
  4. Baklr G, Hofmann T, Schölkopf B, Smola AJ, Taskar B. 2010. Predicting Structured Data Cambridge, MA: MIT Press
  5. Baroni M, Bernardi R, Do NQ, Shan CC. 2012. Entailment above the word level in distributional semantics. Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics23–32 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  6. Berant J, Chou A, Frostig R, Liang P. 2013. Semantic parsing on Freebase from question–answer pairs. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing1533–44 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  7. Berant J, Liang P. 2014. Semantic parsing via paraphrasing. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: Human Language Technologies1415–25 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  8. Bird S, Klein E, Loper E. 2009. Natural Language Processing with Python Sebastopol, CA: O’Reilly
  9. Blackburn P, Bos J. 2003. Computational semantics. Theoria 18:27–45 [Google Scholar]
  10. Blackburn P, Bos J. 2005. Representation and Inference for Natural Language: A First Course in Computational Semantics Stanford, CA: Cent. Study Lang. Inf.
  11. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. Proceedings of the 2008 Assoc. Comput. Mach. SIGMOD International Conference on Management of Data1247–50 New York: Assoc. Comput. Mach. [Google Scholar]
  12. Bottou L. 2010. Large-scale machine learning with stochastic gradient descent. Proceedings of the 19th International Conference on Computational Statistics Lechevallier Y, Saporta G. 177–86 Berlin: Springer [Google Scholar]
  13. Bowman SR. 2014. Can recursive neural tensor networks learn logical reasoning? Presented at 2nd Int. Conf. Learn. Represent., Banff, Can.
  14. Boyd S, Vandenberghe L. 2004. Convex Optimization Cambridge, UK: Cambridge Univ. Press
  15. Branavan S, Silver D, Barzilay R. 2011. Learning to win by reading manuals in a Monte-Carlo framework. J. Artif. Intell. Res. 43:661–704 [Google Scholar]
  16. Branavan S, Zettlemoyer L, Barzilay R. 2010. Reading between the lines: learning to map high-level instructions to commands. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics1268–77 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  17. Cai Q, Yates A. 2013. Large-scale semantic parsing via schema matching and lexicon extension. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics 1423–33 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  18. Carpenter B. 1997. Type-Logical Semantics Cambridge, MA: MIT Press
  19. Chen D. 2012. Fast online lexicon learning for grounded language acquisition. Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics 1430–39 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  20. Chierchia G, Turner R. 1988. Semantics and property theory. Linguist. Philos. 11:261–302 [Google Scholar]
  21. Clark S, Coecke B, Sadrzadeh M. 2011. Mathematical foundations for a compositional distributed model of meaning. Linguist. Anal. 36:345–84 [Google Scholar]
  22. Clarke J, Goldwasser D, Chang MW, Roth D. 2010. Driving semantic parsing from the world’s response. Proceedings of the 14th Conference on Computational Natural Language Learning18–27 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  23. Collobert R, Weston J. 2008. A unified architecture for natural language processing: deep neural networks with multitask learning. Proceedings of the 25th International Conference on Machine Learning160–67 New York: Assoc. Comput. Mach. [Google Scholar]
  24. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12:2493–537 [Google Scholar]
  25. Cooper R. 2012. Book review: Computational semantics with functional programming, Jan van Eijck and Christina Unger. Comput. Linguist. 38:447–49 [Google Scholar]
  26. Dagan I, Glickman O, Magnini B. 2006. The PASCAL recognising textual entailment challenge. Machine Learning Challenges, Vol. 2944: Lecture Notes in Computer Science Quiñonero-Candela J, Dagan I, Magnini B, d’Alché Buc F. 177–90 Berlin: Springer [Google Scholar]
  27. de Marneffe MC, MacCartney B, Manning CD. 2006. Generating typed dependency parses from phrase structure parses. Proceedings of the Fifth International Conference on Language Resources and Evaluation1–8 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  28. Domingos P. 2012. A few useful things to know about machine learning. Commun. Assoc. Comput. Mach. 55:78–87 [Google Scholar]
  29. Dowty D. 2007. Compositionality as an empirical problem. Direct Compositionality Barker C, Jacobson P. 23–101 Oxford, UK: Oxford Univ. Press [Google Scholar]
  30. Firth JR. 1935. The technique of semantics. Trans. Philol. Soc. 34:36–73 [Google Scholar]
  31. Ge R, Mooney RJ. 2005. A statistical semantic parser that integrates syntax and semantics. Proceedings of the Ninth Conference on Computational Natural Language Learning9–16 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  32. Graff D. 2000. Shifting sands: an interest-relative theory of vagueness. Philos. Top. 28:45–81 [Google Scholar]
  33. Grefenstette E, Sadrzadeh M, Clark S, Coecke B, Pulman S. 2011. Concrete sentence spaces for compositional distributional models of meaning. Proceedings of the 9th International Conference on Computational Semantics125–34 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  34. Harris Z. 1954. Distributional structure. Word 10:146–62 [Google Scholar]
  35. Heim I, Kratzer A. 1998. Semantics in Generative Grammar Oxford, UK: Blackwell
  36. Higgins D, Sadock JM. 2003. A machine learning approach to modeling scope preferences. Comput. Linguist. 29:73–96 [Google Scholar]
  37. Jackendoff RS. 1992. Languages of the Mind Cambridge, MA: MIT Press
  38. Jackendoff RS. 1997. The Architecture of the Language Faculty Cambridge, MA: MIT Press
  39. Janssen TMV. 1997. Compositionality. Handbook of Logic and Language van Benthem J, ter Meulen A. 417–73 Cambridge, MA/Amsterdam: MIT Press/North-Holland [Google Scholar]
  40. Jurafsky D, Martin JH. 2009. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition Englewood Cliffs, NJ: Prentice-Hall, 2nd ed..
  41. Kamp H, Partee BH. 1995. Prototype theory and compositionality. Cognition 57:129–91 [Google Scholar]
  42. Kamp H, Reyle U. 1993. From Discourse to Logic: Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory Dordrecht, Neth.: Kluwer
  43. Kate RJ, Mooney RJ. 2006. Using string-kernels for learning semantic parsers. Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics913–20 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  44. Kate RJ, Wong YW, Mooney RJ. 2005. Learning to transform natural to formal languages. Proceedings of the 20th National Conference on Artificial Intelligence 31062–68 Palo Alto, CA: Assoc. Adv. Artif. Intell. [Google Scholar]
  45. Katz JJ. 1972. Semantic Theory New York: Harper & Row
  46. Katz JJ. 1996. Semantics in linguistics and philosophy: an intensionalist perspective. The Handbook of Contemporary Semantic Theory Lappin S. 599–616 Oxford, UK: Blackwell [Google Scholar]
  47. Katz JJ, Fodor JA. 1963. The structure of semantic theory. Language 39:170–210 [Google Scholar]
  48. Kennedy C. 2011. Ambiguity and vagueness. Semantics: An International Handbook of Natural Language Meaning Maienborn C, von Heusinger K, Portner P. 1507–35 Berlin: Mouton de Gruyter [Google Scholar]
  49. Klein E, Sag IA. 1985. Type-driven translation. Linguist. Philos. 8:163–201 [Google Scholar]
  50. Kripke S. 1975. Outline of a theory of truth. J. Philos. 72:690–716 [Google Scholar]
  51. Krishnamurthy J, Kollar T. 2013. Jointly learning to parse and perceive: connecting natural language to the physical world. Trans. Assoc. Comput. Linguist. 1:193–206 [Google Scholar]
  52. Kushman N, Barzilay R. 2013. Using semantic unification to generate regular expressions from natural language. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies826–36 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  53. Kwiatkowksi T, Zettlemoyer L, Goldwater S, Steedman M. 2010. Inducing probabilistic CCG grammars from logical form with higher-order unification. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing1223–33 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  54. Kwiatkowski T, Choi E, Artzi Y, Zettlemoyer L. 2013. Scaling semantic parsers with on-the-fly ontology matching. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing1545–56 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  55. Kwiatkowski T, Zettlemoyer LS, Goldwater S, Steedman M. 2011. Lexical generalization in CCG grammar induction for semantic parsing. Proceedings of the Conference on Empirical Methods in Natural Language Processing1512–23 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  56. Landauer TK, McNamara DS, Dennis S, Kintsch W. 2007. Handbook of Latent Semantic Analysis Hillsdale, NJ: Erlbaum
  57. Lei T, Long F, Barzilay R, Rinard M. 2013. From natural language specifications to program input parsers. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics 11294–303 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  58. Lewis D. 1970. General semantics. Synthese 22:18–67 [Google Scholar]
  59. Lewis M, Steedman M. 2013. Combined distributional and logical semantics. Trans. Assoc. Comput. Linguist. 1:179–92 [Google Scholar]
  60. Liang P, Jordan M, Klein D. 2011. Learning dependency-based compositional semantics. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies590–99 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  61. Liang P, Jordan MI, Klein D. 2013. Learning dependency-based compositional semantics. Comput. Linguist. 39:389–446 [Google Scholar]
  62. Lu W, Ng HT, Lee WS, Zettlemoyer LS. 2008. A generative model for parsing natural language to meaning representations. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing783–92 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  63. MacCartney B, Manning CD. 2009. An extended model of natural logic. Proceedings of the Eighth International Conference on Computational Semantics140–56 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  64. Manning CD, Schütze H. 1999. Foundations of Statistical Natural Language Processing Cambridge, MA: MIT Press
  65. Matuszek C, FitzGerald N, Zettlemoyer LS, Bo L, Fox D. 2012a. A joint model of language and perception for grounded attribute learning. Proceedings of the 29th International Conference on Machine Learning Langford J, Pineau J. 1671–78 Madison, WI: Omnipress [Google Scholar]
  66. Matuszek C, Herbst E, Zettlemoyer LS, Fox D. 2012b. Learning to parse natural language commands to a robot control system. Proceedings of the 13th International Symposium on Experimental Robotics403–15 Berlin: Springer [Google Scholar]
  67. Mitchell J, Lapata M. 2010. Composition in distributional models of semantics. Cogn. Sci. 34:1388–429 [Google Scholar]
  68. Mitchell TM. 1997. Machine Learning New York: McGraw Hill
  69. Montague R. 1970. Universal grammar. Theoria 36:373–398 [Google Scholar]
  70. Montague R. 1974. Formal Philosophy: Selected Papers of Richard Montague New Haven, CT: Yale Univ. Press
  71. Moss LS. 2009. Natural logic and semantics. Proceedings of the 18th Amsterdam Colloquium: Revised Selected Papers Aloni M, Bastiaanse H, de Jager T, van Ormondt P, Schulz K. 71–80 Berlin: Springer [Google Scholar]
  72. Ng HT, Zelle J. 1997. Corpus-based approaches to semantic interpretation in natural language processing. Artif. Intell. Mag. 18:45–64 [Google Scholar]
  73. Palmer M, Gildea D, Xue N. 2010. Semantic Role Labeling San Rafael, CA: Morgan & Claypool
  74. Partee BH. 1980. Semantics—mathematics or psychology?. Semantics from Different Points of View Bäuerle E, von Stechow A. 1–14 Berlin: Springer [Google Scholar]
  75. Partee BH. 1981. Montague grammar, mental representations, and reality. Philosophy and Grammar Kanger S, Öhman S. 59–78 Dordrecht, Neth.: Reidel [Google Scholar]
  76. Partee BH. 1995. Lexical semantics and compositionality. Invitation to Cognitive Science Vol. 1 Gleitman LR, Liberman M. 311–60 Cambridge, MA: MIT Press [Google Scholar]
  77. Partee BH. 2004 (1984). Compositionality. Compositionality in Formal Semantics153–81 Oxford, UK: Blackwell [Google Scholar]
  78. Pereira FCN. 2000. Formal grammar and information theory: together again?. Philos. Trans. R. Soc. 358:1239–53 [Google Scholar]
  79. Reinhart T. 1997. Quantifier scope: how labor is divided between QR and choice functions. Linguist. Philos. 20:335–97 [Google Scholar]
  80. Rumelhart DE, Hinton GE, Williams RJ. 1986a. Learning internal representations by error propagation. Parallel Distributed Processing. Explorations in the Microstructure of Cognition, Vol. 1: Foundations Rumelhart DE, McClelland JL. 318–62 Cambridge, MA: MIT Press [Google Scholar]
  81. Rumelhart DE, Hinton GE, Williams RJ. 1986b. Learning representations by back-propagating errors. Nature 323:533–36 [Google Scholar]
  82. Samuel AL. 1959. Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3:211–29 [Google Scholar]
  83. Samuel AL. 1967. Some studies in machine learning using the game of checkers. II. Recent progress. IBM J. Res. Dev. 11:601–17 [Google Scholar]
  84. Saurí R, Pustejovsky J. 2009. FactBank: a corpus annotated with event factuality. Lang. Resour. Eval. 43:227–68 [Google Scholar]
  85. Smith NA. 2011. Linguistic Structure Prediction San Rafael, CA: Morgan & Claypool
  86. Socher R, Bauer J, Manning CD, Andrew YN. 2013a. Parsing with compositional vector grammars. Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics 1455–65 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  87. Socher R, Huang EH, Pennin J, Manning CD, Ng AY. 2011a. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. Advances in Neural Information Processing Systems 24 Shawe-Taylor J, Zemel RS, Bartlett PL, Pereira FCN, Weinberger KQ. 801–9 Red Hook, NY: Curran [Google Scholar]
  88. Socher R, Huval B, Manning CD, Ng AY. 2012. Semantic compositionality through recursive matrix-vector spaces. Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing1201–11 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  89. Socher R, Pennington J, Huang EH, Ng AY, Manning CD. 2011b. Semi-supervised recursive autoencoders for predicting sentiment distributions. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing151–61 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  90. Socher R, Perelygin A, Wu J, Chuang J, Manning CD et al. 2013b. Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing1631–42 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  91. Szabó ZG. 2012. Compositionality. The Stanford Encyclopedia of Philosophy Zalta EN. Stanford, CA: Cent. Study Lang. Inf. Winter ed http://plato.stanford.edu/entries/compositionality/ [Google Scholar]
  92. Szabolcsi A. 2009. Quantification Cambridge, UK: Cambridge Univ. Press
  93. Tang LR, Mooney RJ. 2001. Using multiple clause constructors in inductive logic programming for semantic parsing. Proceedings of the 12th European Conference on Machine Learning466–77 Berlin: Springer [Google Scholar]
  94. Taskar B, Guestrin C, Koller D. 2003. Max-margin Markov networks. Advances in Neural Information Processing Systems 17 Thrun S, Saul LK, Schölkopf B. 25–32 Cambridge, MA: MIT Press [Google Scholar]
  95. Tellex S, Kollar T, Dickerson S, Walter MR, Banerjee AG et al. 2011. Understanding natural language commands for robotic navigation and mobile manipulation. Proceedings of the 25th AAAI Conference on Artificial Intelligence1507–14 Palo Alto, CA: Assoc. Adv. Artif. Intell. [Google Scholar]
  96. Thompson CA, Mooney RJ. 2003. Acquiring word-meaning mappings for natural language interfaces. J. Artif. Intell. Res. 18:1–44 [Google Scholar]
  97. Turney PD, Pantel P. 2010. From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. 37:141–88 [Google Scholar]
  98. van Eijck J, Unger C. 2010. Computational Semantics with Functional Programming Cambridge, UK: Cambridge Univ. Press
  99. Vogel A, Jurafsky D. 2010. Learning to follow navigational directions. Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics806–14 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  100. Warren DHD, Pereira FCN. 1982. An efficient easily adaptable system for interpreting natural language queries. Am. J. Comput. Linguist. 8:110–22 [Google Scholar]
  101. Werning M, Hinzen W, Machery E. 2012. The Oxford Handbook of Compositionality Oxford, UK: Oxford Univ. Press
  102. Wong YW, Mooney RJ. 2006. Learning for semantic parsing with statistical machine translation. Proceedings of the Human Language Technology Conference of the NAACL, Main Conference439–46 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  103. Wong YW, Mooney RJ. 2007. Learning synchronous grammars for semantic parsing with lambda calculus. Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics960–67 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  104. Woods WA, Kaplan RM, Nash-Webber B. 1972. The lunar sciences natural language information system: final report. Tech. rep., BBN rep. 2378 BBN Technol., Cambridge, MA [Google Scholar]
  105. Yu CNJ, Joachims T. 2009. Learning structural SVMs with latent variables. Proceedings of the 26th International Conference on Machine Learning1169–76 New York: Assoc. Comput. Mach. [Google Scholar]
  106. Zelle JM, Mooney RJ. 1996. Learning to parse database queries using inductive logic programming. Proceedings of the 13th National Conference on Artificial Intelligence 21050–55 Palo Alto, CA: Assoc. Adv. Artif. Intell. [Google Scholar]
  107. Zettlemoyer LS, Collins M. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence658–66 Arlington, VA: Assoc. Uncertain. Artif. Intell. [Google Scholar]
  108. Zettlemoyer LS, Collins M. 2007. Online learning of relaxed CCG grammars for parsing to logical form. Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning678–87 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
/content/journals/10.1146/annurev-linguist-030514-125312
Loading
/content/journals/10.1146/annurev-linguist-030514-125312
Loading

Data & Media loading...

  • Article Type: Review Article
This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error