1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Linguistics
  4. Volume 8, 2022
  5. Article

Annual Review of Linguistics

Volume 8, 2022

Neurocomputational Models of Language Processing

Abstract

Efforts to understand the brain bases of language face the Mapping Problem: At what level do linguistic computations and representations connect to human neurobiology? We review one approach to this problem that relies on rigorously defined computational models to specify the links between linguistic features and neural signals. Such tools can be used to estimate linguistic predictions, model linguistic features, and specify a sequence of processing steps that may be quantitatively fit to neural signals collected while participants use language. Progress has been helped by advances in machine learning, attention to linguistically interpretable models, and openly shared data sets that allow researchers to compare and contrast a variety of models. We describe one such data set in detail in the .

Keyword(s): brain, computational model, deep learning, lexicon, neurolinguistics, parsing
Loading

Article metrics loading...

/content/journals/10.1146/annurev-linguistics-051421-020803
2022-01-14
2024-04-20
Loading full text...

Full text loading...

/deliver/fulltext/linguistics/8/1/annurev-linguistics-051421-020803.html?itemId=/content/journals/10.1146/annurev-linguistics-051421-020803&mimeType=html&fmt=ahah

Literature Cited

  1. Abney S, Johnson M. 1991. Memory requirements and local ambiguities of parsing strategies. J. Psycholinguist. Res. 20:3233–50
     [Google Scholar]
  2. Aliko S, Huang J, Gheorghiu F, Meliss S, Skipper JI 2020. A naturalistic neuroimaging database for understanding the brain using ecological stimuli. Sci. Data 7:1347
     [Google Scholar]
  3. Allen K, Pereira F, Botvinick M, Goldberg AE. 2012. Distinguishing grammatical constructions with fMRI pattern analysis. Brain Lang. 123:3174–82
     [Google Scholar]
  4. Anderson AJ, Kiela D, Binder JR, Fernandino L, Humphries CJ et al. 2021. Deep artificial neural networks reveal a distributed cortical network encoding propositional sentence-level meaning. J. Neurosci. 41:184100–19
     [Google Scholar]
  5. Baroni M. 2021. On the proper role of linguistically-oriented deep net analysis in linguistic theorizing. arXiv:2106.08694 [cs.CL]
  6. Baroni M, Bernardi R, Zamparelli R. 2014. Frege in space: a program for composition distributional semantics. Linguistic Issues in Language Technology, Volume 9, 2014—Perspectives on Semantic Representations for Textual Inference241–346 Stanford, CA: CSLI Publ.
     [Google Scholar]
  7. Bastiaansen M, Hagoort P. 2006. Oscillatory neuronal dynamics during language comprehension. Prog. Brain Res. 159:179–96
     [Google Scholar]
  8. Bever TG 1970. The cognitive basis for linguistic structures. Cognition and the Development of Language J Hayes 279–362 New York: Wiley
     [Google Scholar]
  9. Bhattasali S, Brennan J, Luh WM, Franzluebbers B, Hale J 2020. The Alice Datasets: fMRI & EEG observations of natural language comprehension. Proceedings of the 12th Language Resources and Evaluation Conference120–25 Marseille, Fr: Eur. Lang. Res. Assoc.
     [Google Scholar]
  10. Bhattasali S, Fabre M, Luh WM, Al Saied H, Constant M et al. 2019. Localising memory retrieval and syntactic composition: an fMRI study of naturalistic language comprehension. Lang. Cogn. Neurosci. 34:4491–510
     [Google Scholar]
  11. Blache P 2018. Light-and-deep parsing: a cognitive model of sentence processing. Language, Cognition and Computational Models T Poibeau, A Villavicencio 27–52 Cambridge, UK: Cambridge Univ. Press
     [Google Scholar]
  12. Bornkessel-Schlesewsky I, Schlesewsky M. 2013. Neurotypology. See Sanz et al. 2013 241–52
  13. Bornkessel-Schlesewsky I, Schlesewsky M 2016. The importance of linguistic typology for the neurobiology of language. Linguist. Typol. 20:3615–21
     [Google Scholar]
  14. Brennan JR. 2016. Naturalistic sentence comprehension in the brain: naturalistic comprehension. Lang. Linguist. Compass 10:7299–313
     [Google Scholar]
  15. Brennan JR Forthcoming. Hemodynamic methods. Oxford Handbook of Experimental Syntax J Sprouse Oxford, UK: Oxford Univ. Press
     [Google Scholar]
  16. Brennan JR, Dyer C, Kuncoro A, Hale JT. 2020. Localizing syntactic predictions using recurrent neural network grammars. Neuropsychologia 146:107479
     [Google Scholar]
  17. Brennan JR, Hale JT. 2019. Hierarchical structure guides rapid linguistic predictions during naturalistic listening. PLOS ONE 14:1e0207741
     [Google Scholar]
  18. Brennan JR, Nir Y, Hasson U, Malach R, Heeger DJ, Pylkkänen L. 2012. Syntactic structure building in the anterior temporal lobe during natural story listening. Brain Lang. 120:2163–73
     [Google Scholar]
  19. Brennan JR, Pylkkänen L. 2017. MEG evidence for incremental sentence composition in the anterior temporal lobe. Cogn. Sci. 41:Suppl. 61515–31
     [Google Scholar]
  20. Brennan JR, Stabler EP, Van Wagenen SE, Luh WM, Hale JT 2016. Abstract linguistic structure correlates with temporal activity during naturalistic comprehension. Brain Lang. 157–158:81–94
     [Google Scholar]
  21. Brouwer H, Delogu F, Venhuizen NJ, Crocker MW. 2021. Neurobehavioral correlates of surprisal in language comprehension: a neurocomputational model. Front. Psychol. 12:110
     [Google Scholar]
  22. Buchweitz A, Shinkareva SV, Mason RA, Mitchell TM, Just MA. 2012. Identifying bilingual semantic neural representations across languages. Brain Lang. 120:3282–89
     [Google Scholar]
  23. Caplan D. 1992. Language: Structure, Processing, and Disorders Cambridge, MA: MIT Press
  24. Caucheteux C, Gramfort A, King JR 2021. Disentangling syntax and semantics in the brain with deep networks. Proceedings of Machine Learning Research, Vol. 139: Proceedings of the 38th International Conference on Machine Learning M Meila, T Zhang 1336–48 n.p.: PMLR
     [Google Scholar]
  25. Choe DK, Charniak E. 2016. Parsing as language modeling. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing2331–36 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  26. Chomsky N. 1956. Three models for the description of language. IRE Trans. Inform. Theory 2:3113–24
     [Google Scholar]
  27. Coecke B, Sadrzadeh M, Clark S. 2010. Mathematical foundations for a compositional distributional model of meaning. Linguist. Anal. 36:345–84
     [Google Scholar]
  28. Correia J, Formisano E, Valente G, Hausfeld L, Jansma B, Bonte M 2013. Brain-based translation: fMRI decoding of spoken words in bilinguals reveals language-independent semantic representations in anterior temporal lobe. J. Neurosci. 34:1332–38
     [Google Scholar]
  29. Crinion J, Turner R, Grogan A, Hanakawa T, Noppeney U et al. 2006. Language control in the bilingual brain. Science 312:1537–40
     [Google Scholar]
  30. Cronbach LJ, Meehl PE. 1955. Construct validity in psychological tests. Psychol. Bull. 52:4281–302
     [Google Scholar]
  31. Dikker S, Pylkkänen L. 2013. Predicting language: MEG evidence for lexical preactivation. Brain Lang. 127:155–64
     [Google Scholar]
  32. Dunagan D, Zhang S, Li J, Bhattasali S, Pallier C et al. 2021. Neural correlates of semantic number: a cross-linguistic investigation. bioRxiv. https://doi.org/10.1101/2021.05.11.443670
    [Crossref]
  33. Dyer C, Kuncoro A, Ballesteros M, Smith NA. 2016. Recurrent neural network grammars. Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies199–209 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  34. Eisenstein J. 2019. Introduction to Natural Language Processing Cambridge, MA: MIT Press
  35. Elman J. 1990. Finding structure in time. Cogn. Sci. 14:179–211
     [Google Scholar]
  36. Embick D, Poeppel D. 2015. Towards a computational(ist) neurobiology of language: correlational, integrated, and explanatory neurolinguistics. Lang. Cogn. Neurosci. 30:4357–66
     [Google Scholar]
  37. Ettinger A, Feldman N, Resnik P, Phillips C 2016. Modeling N400 amplitude using vector space models of word representation. Proceedings of the 38th Annual Meeting of the Cognitive Science Society A Papafragou, D Grodner, D Mirman, J Trueswell 1445–50 Austin, TX: Cogn. Sci. Soc.
     [Google Scholar]
  38. Evert S 2009. Corpora and collocations. Corpus Linguistics: An International Handbook 2 A Lüdeling, M Kytö 1212–48 Berlin, Ger: De Gruyter Mouton
     [Google Scholar]
  39. Fillmore CJ. 1968. The case for case. Universals in Linguistic Theory EW Bach, RT Harms 1–88 London: Holt, Rinehart and Winston
     [Google Scholar]
  40. Frank SL, Otten LJ, Galli G, Vigliocco G. 2015. The ERP response to the amount of information conveyed by words in sentences. Brain Lang. 140:1–11
     [Google Scholar]
  41. Frankland SM, Greene JD. 2015. An architecture for encoding sentence meaning in left mid-superior temporal cortex. PNAS 112:3711732–37
     [Google Scholar]
  42. Frazier L 1985. Syntactic complexity. Natural Language Parsing: Psychological, Computational, and Theoretical Perspectives D Dowty, L Karttunen, AM Zwicky 129–87 Cambridge, UK: Cambridge Univ. Press
     [Google Scholar]
  43. Friederici AD. 2017. Language in Our Brain: The Origins of a Uniquely Human Capacity Cambridge, MA: MIT Press
  44. Friston K. 2010. The free-energy principle: a unified brain theory?. Nat. Rev. Neurosci. 11:2127–38
     [Google Scholar]
  45. Goldberg Y. 2017. Neural Network Methods for Natural Language Processing San Rafael, CA: Morgan & Claypool
  46. Hagoort P 2016. MUC (memory, unification, control): a model on the neurobiology of language beyond single word processing. Neurobiology of Language G Hickok, SL Small 339–47 London: Academic
     [Google Scholar]
  47. Hale JT. 2016. Information-theoretical complexity metrics. Lang. Linguist. Compass 10:9397–412
     [Google Scholar]
  48. Hale JT. 2017. Models of human sentence comprehension in computational psycholinguistics. Oxford Research Encyclopedia of Linguistics M Aronoff Oxford, UK: Oxford Univ. Press https://doi.org/10.1093/acrefore/9780199384655.013.377
    [Crossref] [Google Scholar]
  49. Hale JT, Dyer C, Kuncoro A, Brennan JR 2018. Finding syntax in human encephalography with beam search. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Vol. 1: Long Papers2727–36 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  50. Hale JT, Kuncoro A, Hall K, Dyer C, Brennan J. 2019. Text genre and training data size in human-like parsing. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)5846–52 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  51. Hansen PC, Kringelbach ML, Salmelin R. 2010. MEG: An Introduction to Methods Oxford, UK: Oxford Univ. Press
  52. Hashemzadeh M, Kaufeld G, White M, Martin AE, Fyshe A. 2020. From language to language-ish: How brain-like is an LSTM's representation of nonsensical language stimuli?. Findings of the Association for Computational Linguistics: EMNLP 2020645–56 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  53. Heaven D. 2019. Why deep-learning AIs are so easy to fool. Nature 574:7777163–66
     [Google Scholar]
  54. Heilbron M, Armeni K, Schoffelen JM, Hagoort P, de Lange FP. 2021. A hierarchy of linguistic predictions during natural language comprehension. bioRxiv. https://doi.org/10.1101/2020.12.03.410399
    [Crossref]
  55. Hemforth B, Konieczny L 2006. Language processing: construction of mental models or more?. Advances in Psychology, Vol. 138: Mental Models and the Mind C Held, M Knauff, G Vosgerau 189–204 Amsterdam: North-Holland
     [Google Scholar]
  56. Henderson JM, Choi W, Lowder MW, Ferreira F. 2016. Language structure in the brain: a fixation-related fMRI study of syntactic surprisal in reading. NeuroImage 132:293–300
     [Google Scholar]
  57. Hochreiter S, Schmidhuber J. 1997. Long short-term memory. Neural Comput. 9:81735–80
     [Google Scholar]
  58. Hollenstein N, Troendle M, Zhang C, Langer N. 2020. ZuCo 2.0: a dataset of physiological recordings during natural reading and annotation. Proceedings of the 12th Language Resources and Evaluation Conference138–46 Marseille, Fr: Eur. Lang. Res. Assoc.
     [Google Scholar]
  59. Honey CJ, Thompson CR, Lerner Y, Hasson U. 2012. Not lost in translation: neural responses shared across languages. J. Neurosci. 32:4415277–83
     [Google Scholar]
  60. Huth AG, de Heer WA, Griffiths TL, Theunissen FE, Gallant JL. 2016. Natural speech reveals the semantic maps that tile human cerebral cortex. Nature 532:7600453–58
     [Google Scholar]
  61. Jelinek F, Lafferty JD. 1991. Computation of the probability of initial substring generation by stochastic context-free grammars. Comput. Linguist. 17:3315–53
     [Google Scholar]
  62. Jordan MI. 1986. An introduction to linear algebra in parallel distributed processing. See Rumelhart et al. 1986b 1365–422
  63. Joshi AK 1985. Tree adjoining grammars: How much context-sensitivity is required to provide reasonable structural descriptions?. Natural Language Parsing: Psychological, Computational, and Theoretical Perspectives D Dowty, L Karttunen, A Zwicky 206–50 Cambridge, UK: Cambridge Univ. Press
     [Google Scholar]
  64. Joshi AK, Vijay-Shanker K, Weir D. 1991. The convergence of mildly context-sensitive grammatical formalisms. Foundational Issues in Natural Language Processing31–81 Cambridge, MA: MIT Press
     [Google Scholar]
  65. Jurafsky D, Martin JH. 2021. Speech and Language Processing Stanford, CA: Stanford Univ, 3rd ed. draft.
  66. Kallmeyer L. 2010. Parsing Beyond Context-Free Grammars Berlin: Springer
  67. Kaplan RM. 1972. Augmented transition networks as psychological models of sentence comprehension. Artif. Intel. 3:77–100
     [Google Scholar]
  68. Katz JJ, Fodor JA. 1963. The structure of a semantic theory. Language 39:2170–210
     [Google Scholar]
  69. Kemmerer D. 2014. Cognitive Neuroscience of Language New York: Psychol. Press
  70. Kriegeskorte N, Goebel R, Bandettini P. 2006. Information-based functional brain mapping. PNAS 103:103863–68
     [Google Scholar]
  71. Kuncoro A, Ballesteros M, Kong L, Dyer C, Neubig G, Smith NA 2017. What do recurrent neural network grammars learn about syntax?. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Vol. 1: Long Papers1249–58 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  72. Kutas M, Federmeier KD. 2000. Electrophysiology reveals semantic memory use in language comprehension. Trends Cogn. Sci. 4:463–69
     [Google Scholar]
  73. Langendoen DT. 2008. Coordinate grammar. Language 84:4691–709
     [Google Scholar]
  74. Lenneberg EH. 1967. Biological Foundations of Language New York: Wiley
  75. Lerner Y, Honey CJ, Silbert LJ, Hasson U. 2011. Topographic mapping of a hierarchy of temporal receptive windows using a narrated story. J. Neurosci. 31:82906–15
     [Google Scholar]
  76. Li J, Bhattasali S, Pallier C, Hale J. 2021. Le Petit Prince: a multilingual fMRI corpus using ecological stimuli. OpenNeuro. https://openneuro.org/datasets/ds003643
  77. Li J, Brennan J, Mahar A, Hale J 2016. Temporal lobes as combinatory engines for both form and meaning. Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity186–91 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  78. Li J, Hale JT 2019. Grammatical predictors for fMRI timecourses. Minimalist Parsing EP Stabler, RC Berwick 159–73 New York: Oxford Univ. Press
     [Google Scholar]
  79. Li J, Wang S, Luh WM, Pylkkänen L, Yang Y, Hale JT 2020. Modeling pronoun resolution in the brain. bioRxiv. https://www.biorxiv.org/content/10.1101/2020.11.24.396598v1
  80. Linzen T, Baroni M. 2021. Syntactic structure from deep learning. Annu. Rev. Linguist. 7:195–212
     [Google Scholar]
  81. Linzen T, Dupoux E, Goldberg Y 2016. Assessing the ability of LSTMs to learn syntax-sensitive dependencies. Trans. Assoc. Comput. Linguist. 4:521–35
     [Google Scholar]
  82. Lopopolo A, Frank SL, van den Bosch A, Willems RM. 2017. Using stochastic language models (SLM) to map lexical, syntactic, and phonological information processing in the brain. PLOS ONE 12:5e0177794
     [Google Scholar]
  83. Luck SJ. 2014. An Introduction to the Event-Related Potential Technique Cambridge, MA: MIT Press, 2nd ed..
  84. Lyu B, Choi HS, Marslen-Wilson WD, Clarke A, Randall B, Tyler LK 2019. Neural dynamics of semantic composition. PNAS 116:4221318–27
     [Google Scholar]
  85. Manning CD, Schütze H. 2000. Foundations of Statistical Natural Language Processing Cambridge, MA: MIT Press
  86. Marcus MP, Santorini B, Marcinkiewicz MA. 1993. Building a large annotated corpus of English: the Penn Treebank. Comput. Linguist. 19:313–30
     [Google Scholar]
  87. Marslen-Wilson WD. 1975. Sentence perception as an interactive parallel process. Science 189:4198226–28
     [Google Scholar]
  88. Martin AE, Doumas LAA. 2017. A mechanism for the cortical computation of hierarchical linguistic structure. PLOS Biol. 15:3e2000663
     [Google Scholar]
  89. Martin AE, Doumas LAA. 2019. Tensors and compositionality in neural systems. Philos. Trans. B 375:20190306
     [Google Scholar]
  90. Matchin W, Hammerly C, Lau EF. 2016. The role of the IFG and pSTS in syntactic prediction: evidence from a parametric study of hierarchical structure in fMRI. Cortex 88:106–23
     [Google Scholar]
  91. McClelland JL, Kawamoto AH. 1986. Mechanisms of sentence processing: assigning roles to constituents. See Rumelhart et al. 1986b 2272–325
  92. Meyer L, Sun Y, Martin AE 2020. Synchronous, but not entrained: exogenous and endogenous cortical rhythms of speech and language processing. Lang. Cogn. Neurosci. 35:91089–99
     [Google Scholar]
  93. Mitchell J, Lapata M 2010. Composition in distributional models of semantics. Cogn. Sci. 34:81388–429
     [Google Scholar]
  94. Mitchell TM, Shinkareva SV, Carlson A, Chang KM, Malave VL et al. 2008. Predicting human brain activity associated with the meanings of nouns. Science 320:58801191–95
     [Google Scholar]
  95. Murphy B, Wehbe L, Fyshe A 2018. Decoding language from the brain. Language, Cognition, and Computational Models T Poibeau, A Villavicencio 53–80 Cambridge, UK: Cambridge Univ. Press
     [Google Scholar]
  96. Nastase SA, Liu YF, Hillman H, Zadbood A, Hasenfratz L et al. 2020. Narratives: fMRI data for evaluating models of naturalistic language comprehension. bioRxiv. https://www.biorxiv.org/content/10.1101/2020.12.23.424091v1
  97. Nelson MJ, El Karoui I, Giber K, Yang X, Cohen L et al. 2017. Neurophysiological dynamics of phrase-structure building during sentence processing. PNAS 114:18E3669–78
     [Google Scholar]
  98. O'Donnell TJ. 2015. Productivity and Reuse in Language: A Theory of Linguistic Computation and Storage Cambridge, MA: MIT Press
  99. Pallier C, Devauchelle AD, Dehaene S. 2011. Cortical representation of the constituent structure of sentences. PNAS 108:62522–27
     [Google Scholar]
  100. Partee BH, ter Meulen A, Wall RE. 1993. Mathematical Methods in Linguistics Dordrecht, Neth: Kluwer
  101. Pereira F, Lou B, Pritchett B, Ritter S, Gershman SJ et al. 2018. Toward a universal decoder of linguistic meaning from brain activation. Nat. Commun. 9:1963
     [Google Scholar]
  102. Pereira F, Mitchell T, Botvinick M 2009. Machine learning classifiers and fMRI: a tutorial overview. NeuroImage 45:Suppl. 1S199–209
     [Google Scholar]
  103. Pesetsky D. 1995. Zero Syntax: Experiencers and Cascades Cambridge, MA: MIT Press
  104. Phillips C. 2013. Parser-grammar relations: We don't understand everything twice. See Sanz et al. 2013 294–315
  105. Poeppel D. 2012. The maps problem and the mapping problem: two challenges for a cognitive neuroscience of speech and language. Cogn. Neuropsychol. 29:1–234–55
     [Google Scholar]
  106. Pylkkänen L. 2019. The neural basis of combinatory syntax and semantics. Science 366:646162–66
     [Google Scholar]
  107. Qian P, Qiu X, Huang X. 2016. Bridging LSTM architecture and the neural dynamics during reading. Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence1953–59 Palo Alto, CA: AAAI
     [Google Scholar]
  108. Rabovsky M, Hansen SS, McClelland JL. 2018. Modelling the N400 brain potential as change in a probabilistic representation of meaning. Nat. Hum. Behav. 2:9693–705
     [Google Scholar]
  109. Reddy AJ, Wehbe L. 2021. Can fMRI reveal the representation of syntactic structure in the brain?. bioRxiv. https://doi.org/10.1101/2020.06.16.155499
    [Crossref]
  110. Rescorla M. 2020. The computational theory of mind. The Stanford Encyclopedia of Philosophy https://plato.stanford.edu/entries/computational-mind/
    [Google Scholar]
  111. Rohde DL. 2002. A connectionist model of sentence comprehension and production PhD Thesis, Carnegie Mellon Univ. Pittsburgh, PA:
  112. Rumelhart DE, Hinton GE, McClelland JL. 1986a. Learning internal representations by error propagation. See Rumelhart et al. 1986b 1318–62
  113. Rumelhart DE, McClelland JPDP Res. Group 1986b. Parallel Distributed Processing: Explorations in the Microstructure of Cognition 1–2 Cambridge, MA: MIT Press
  114. Salmelin R, Baillet S. 2009. Electromagnetic brain imaging. Hum. Brain Mapp. 30:61753–57
     [Google Scholar]
  115. Sanz M, Laka I, Tanenhaus MK 2013. Language Down the Garden Path: The Cognitive and Biological Basis of Linguistic Structures Oxford, UK: Oxford Univ. Press
  116. Schoffelen JM, Oostenveld R, Lam NHL, Uddén J, Hultén A, Hagoort P. 2019. A 204-subject multimodal neuroimaging dataset to study language processing. Sci. Data 6:117
     [Google Scholar]
  117. Schrimpf M, Blank IA, Tuckute G, Kauf C, Hosseini EAet al 2021. The neural architecture of language: integrative modeling converges on predictive processing. PNAS 118:45e2105646118
     [Google Scholar]
  118. Shain C, Blank IA, van Schijndel M, Schuler W, Fedorenko E. 2020. fMRI reveals language-specific predictive coding during naturalistic sentence comprehension. Neuropsychologia 138:107307
     [Google Scholar]
  119. Smith GT. 2005. On construct validity: issues of method and measurement. Psychol. Assess. 17:4396–408
     [Google Scholar]
  120. Smolensky P. 2015. Four facts about tensor product representations. Talk presented at NIPS Workshop: Cognitive Computation: Integrating Neural and Symbolic Approaches Montreal, Can.: Dec. 12. https://youtu.be/teuJ4SngxjQ
    [Google Scholar]
  121. Smolensky P, Legendre G. 2006. The Harmonic Mind Cambridge, MA: MIT Press
  122. Stabler EP. 1983. How are grammars represented?. Behav. Brain Sci. 6:391–421
     [Google Scholar]
  123. Stabler EP 2001. Minimalist grammars and recognition. Linguistic Form and Its Computation C Rohrer, A Roßdeutscher, H Kamp 327–52 Stanford, CA: CSLI Publ.
     [Google Scholar]
  124. Stabler EP. 2013. The epicenter of linguistic behavior. See Sanz et al. 2013 316–23
  125. Stanojević M, Bhattasali S, Dunagan D, Campanelli L, Steedman M et al. 2021. Modeling incremental language comprehension in the brain with Combinatory Categorial Grammar. Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics E Chersoni, N Hollenstein, C Jacobs, Y Oseki, L Prévot, E Santus 23–38 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  126. Steedman M. 1999. Connectionist sentence processing in perspective. Cogn. Sci. 23:4615–34
     [Google Scholar]
  127. Steedman M. 2000. The Syntactic Process Cambridge, MA: MIT Press
  128. Stehwien S, Henke L, Hale J, Brennan J, Meyer L. 2020. The Little Prince in 26 languages: towards a multilingual neuro-cognitive corpus. Proceedings of the Second Workshop on Linguistic and Neurocognitive Resources43–49 Marseille, Fr: Eur. Lang. Res. Assoc.
     [Google Scholar]
  129. Stowe LA, Broere CA, Paans AM, Wijers AA, Mulder G et al. 1998. Localizing components of a complex task: sentence processing and working memory. NeuroReport 9:132995–99
     [Google Scholar]
  130. Stowe LA, Haverkort M, Zwarts F. 2005. Rethinking the neurological basis of language. Lingua 115:7997–1042
     [Google Scholar]
  131. Swaab TY, Ledoux K, Camblin CC, Boudewyn MA 2012. Language-related ERP components. The Oxford Handbook of Event-Related Potential Components SJ Luck, ES Kappenman 397–439 Oxford, UK: Oxford Univ. Press
     [Google Scholar]
  132. Toneva M, Wehbe L 2019. Interpreting and improving natural-language processing (in machines) with natural language-processing (in the brain). Advances in Neural Information Processing Systems 32 (NeurIPS 2019) H Wallach, H Larochelle, A Beygelzimer, F d'Alché Buc, E Fox, R Garnett 14954–64 Red Hook, NY: Curran Assoc.
     [Google Scholar]
  133. Townsend DJ, Bever TG. 2001. Sentence Comprehension: The Integration of Habits and Rules Cambridge, MA: MIT Press
  134. Ullman MT. 2004. Contributions of memory circuits to language: the declarative/procedural model. Cognition 92:1–2231–70
     [Google Scholar]
  135. van Rij J, van Rijn H, Hendriks P. 2013. How WM load influences linguistic processing in adults: a computational model of pronoun interpretation in discourse. Top. Cogn. Sci. 5:3564–80
     [Google Scholar]
  136. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L et al. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (NIPS 2017)6000–10 n.p.: Neural Inf. Process. Syst.
     [Google Scholar]
  137. Vinyals O, Kaiser Ł, Koo T, Petrov S, Sutskever I, Hinton G. 2015. Grammar as a foreign language. Advances in Neural Information Processing Systems 28 (NIPS 2015) C Cortes, N Lawrence, D Lee, M Sugiyama, R Garnett Red Hook, NY: Curran Assoc https://papers.nips.cc/paper/2015/hash/277281aada22045c03945dcb2ca6f2ec-Abstract.html
    [Google Scholar]
  138. Wedekind J, Kaplan RM. 2020. Tractable Lexical-Functional Grammar. Comput. Linguist. 46:3515–69
     [Google Scholar]
  139. Wehbe L, Murphy B, Talukdar P, Fyshe A, Ramdas A, Mitchell T 2014a. Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses. PLOS ONE 9:11e112575
     [Google Scholar]
  140. Wehbe L, Vaswani A, Knight K, Mitchell T. 2014b. Aligning context-based statistical models of language with brain activity during reading. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP)233–43 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  141. Werbos PJ. 1990. Backpropagation through time: what it does and how to do it. Proc. IEEE 78:101550–60
     [Google Scholar]
  142. Willems RM, Frank SL, Nijhof AD, Hagoort P, van den Bosch A. 2016. Prediction during natural language comprehension. Cereb. Cortex 26:62506–16
     [Google Scholar]
  143. Yarkoni T, Speer NK, Balota DA, McAvoy MP, Zacks JM. 2008. Pictures of a thousand words: investigating the neural mechanisms of reading with extremely rapid event-related fMRI. NeuroImage 42:2973–87
     [Google Scholar]
  144. Yu L, Ettinger A 2020. Assessing phrasal representation and composition in transformers. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)4896–907 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  145. Zhang S, Li J, Yang Y, Hale J 2021. Decoding the silence: neural bases of zero pronoun resolution in Chinese. Brain Lang. In press
    [Google Scholar]
/content/journals/10.1146/annurev-linguistics-051421-020803
Loading
Neurocomputational Models of Language Processing
Annual Review of Linguistics 8, 427 (2022); https://doi.org/10.1146/annurev-linguistics-051421-020803
/content/journals/10.1146/annurev-linguistics-051421-020803
/content/journals/10.1146/annurev-linguistics-051421-020803
Loading

Data & Media loading...

Supplemental Material

Supplementary Data

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error