1932
0

Annual Review of Linguistics

Volume 3, 2017

Computational Learning of Morphology

Abstract

This article reviews research on the unsupervised learning of morphology, that is, the induction of morphological knowledge with no prior knowledge of the language beyond the training texts. This is an area of considerable activity over the period from the mid 1990s to the present. It is of particular interest to linguists because it provides a good example of a domain in which complex structures must be induced by the language learner, and successes in this area have all relied on quantitative models that in various ways focus on model complexity and on goodness of fit to the data.

Keyword(s): adaptor grammars, Gibbs sampling, grammar induction, language induction, minimum description length, morphology, unsupervised learning
Loading

Article metrics loading...

/content/journals/10.1146/annurev-linguistics-011516-034017
2017-01-14
2025-06-24
Loading full text...

Full text loading...

/deliver/fulltext/linguistics/3/1/annurev-linguistics-011516-034017.html?itemId=/content/journals/10.1146/annurev-linguistics-011516-034017&mimeType=html&fmt=ahah

Literature Cited

  1. Adamson GW, Boreham J. 1974. The use of an association measure based on character structure to identify semantically related pairs of words and document titles. Inf. Storage Retr. 10:253–60 [Google Scholar]
  2. Ahlberg M, Hulden M, Forsberg M. 2014. Semi-supervised learning of morphological paradigms and lexicons. Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics567–78 Gothenburg, Swed: Assoc. Comput. Linguist. [Google Scholar]
  3. Baroni M, Matiasek J, Trost H. 2002. Unsupervised discovery of morphologically related words based on orthographic and semantic similarity. Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning48–57 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  4. Bati TB. 2002. Automatic morphological analyzer for Amharic: an experiment employing unsupervised learning and autosegmental analysis approach MS thesis, Dep. Inf. Sci., Addis Ababa Univ., Ethiop. 90 [Google Scholar]
  5. Beesley KR, Karttunen L. 2003. Finite State Morphology Stanford, CA: Cent. Study Lang. Inf. [Google Scholar]
  6. Beniamine S, Sagot B. 2015. Segmentation strategies for inflection class inference Presented at Décembrettes 9: Colloq. Int. Morphol., Toulouse, France [Google Scholar]
  7. Botha JA, Blunsom P. 2013. Adaptor grammars for learning non-concatenative morphology. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP13)345–56 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  8. Boullier P. 2000. Range concatenation grammars. Proceedings of the Sixth International Workshop on Parsing Technologies (IWPT 2000)53–64 Trento, Italy: Inst. Sci. Technol. Res. [Google Scholar]
  9. Brent MR. 1996. Advances in the computational study of language acquisition. Cognition 61:1–38 [Google Scholar]
  10. Brent MR, Murthy SK, Lundberg A. 1995. Discovering morphemic suffixes: a case study in minimum description length induction. Proceedings of the 5th International Workshop on Artificial Intelligence and Statistics D Fisher, H-J Lenz 264–71 New York: Springer [Google Scholar]
  11. Brown D, Evans R. 2012. Morphological complexity and unsupervised learning: validating Russian inflectional classes using high frequency data. Current Issues in Morphological Theory: (Ir)regularity, Analogy and Frequency F Kiefer, M Ladányi, P Siptár 135–62 Amsterdam/Philadelphia: Benjamins [Google Scholar]
  12. Clark A. 2007. Supervised and unsupervised learning of Arabic morphology. Text, Speech and Language Technology 38 Arabic Computational Morphology A Soudi, A Bosch, G Neumann 181–200 Amsterdam: Springer [Google Scholar]
  13. Creutz M, Lagus K. 2002. Unsupervised discovery of morphemes. Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning21–30 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  14. Creutz M, Lagus K. 2007. Unsupervised models for morpheme segmentation and morphology learning. ACM Trans. Speech Lang. Process. 4:3 [Google Scholar]
  15. de Marcken CG. 1996. Unsupervised language acquisition PhD thesis, Dep. Comput. Sci., MIT Cambridge, MA: [Google Scholar]
  16. De Pauw G, Wagacha PW. 2007. Bootstrapping morphological analysis of G kũyũ using unsupervised maximum entropy learning. Proceedings of the 8th Annual Conference of the International Speech Communication Association (INTERSPEECH 2007)1449–52 Red Hook, NY: Curran [Google Scholar]
  17. De Roeck A, Al-Fares W. 2000. A morphologically sensitive clustering algorithm for identifying Arabic roots. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics199–206 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  18. Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R. 1990. Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41:391–407 [Google Scholar]
  19. Dempster AP, Laird NM, Rubin DB. 1977. Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39:1–38 [Google Scholar]
  20. Desai S, Pawar J, Bhattacharyya P. 2014. A framework for learning morphology using suffix association matrix. Proceedings of the 5th Workshop on South and Southeast Asian Natural Language Processing (WSSANLP 2014)28–36 Red Hook, NY: Curran [Google Scholar]
  21. Dreyer M, Eisner J. 2011. Discovering morphological paradigms from plain text using a Dirichlet process mixture model. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP11)616–27 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  22. Durrett G, DeNero J. 2013. Supervised learning of complete morphological paradigms. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies1185–95 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  23. Earl LL. 1966. Structural definition of affixes from multisyllable words. Mech. Transl. Comput. Linguist. 9:34–37 [Google Scholar]
  24. Elghamry K. 2004. A constraint-based algorithm for the identification of Arabic roots. Proceedings of the 1st Midwest Computational Linguistics Colloquium Bloomington: Indiana Univ. [Google Scholar]
  25. Flenner G. 1994. Ein quantitatives Morphsegmentierungssystem für spanische Wortformen. Comput. Linguist. 2:31–62 [Google Scholar]
  26. Frank S, Keller F, Goldwater S. 2013. Exploring the utility of joint morphological and syntactic learning from child-directed speech. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing (EMNLP13)30–41 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  27. Fullwood MA, O'Donnell TJ. 2013. Learning non-concatenative morphology. Proceedings of the 4th Annual Workshop on Cognitive Modeling and Computational Linguistics21–27 Sofia, Bulg.: Assoc. Comput. Linguist. [Google Scholar]
  28. Gaussier É. 1999. Unsupervised learning of derivational morphology from inflectional lexicons. Proceedings of the ACL'99 Workshop: Unsupervised Learning in Natural Language Processing24–30 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  29. Goldsmith JA. 2001. Unsupervised learning of the morphology of a natural language. Comput. Linguist. 27:153–98 [Google Scholar]
  30. Goldsmith JA. 2006. An algorithm for the unsupervised learning of morphology. Nat. Lang. Eng. 12:353–71 [Google Scholar]
  31. Goldsmith JA. 2010. Segmentation and morphology. The Handbook of Computational Linguistics and Natural Language Processing364–93 New York: Wiley-Blackwell [Google Scholar]
  32. Goldsmith JA. 2015. Towards a new empiricism for linguistics. Empiricism and Language Learnability A Clark, A Perfors, JA Goldsmith, N Chater 58–105 Oxford, UK: Oxford Univ. Press [Google Scholar]
  33. Goldsmith JA, O'Brien J. 2006. Learning inflectional classes. Lang. Learn. Dev. 2:219–50 [Google Scholar]
  34. Goldsmith JA, Xanthos A. 2009. Learning phonological categories. Language 85:4–38 [Google Scholar]
  35. Goldwater S, Johnson M. 2004. Priors in Bayesian learning of phonological rules. Proceedings of the 7th Meeting of the ACL Special Interest Group in Computational Phonology (SIGPHON04)35–42 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  36. Hafer MA, Weiss SF. 1974. Word segmentation by letter successor varieties. Inf. Storage Retr. 10:371–85 [Google Scholar]
  37. Hammarström H, Borin L. 2011. Unsupervised learning of morphology. Comput. Linguist. 37:309–50 [Google Scholar]
  38. Harris ZS. 1955. From phoneme to morpheme. Language 31:190–222 [Google Scholar]
  39. Harris ZS. 1967. Morpheme boundaries within words: report on a computer test. Transf. Discourse Anal. Pap.73. 24 [Google Scholar]
  40. Higgins D. 2002. A multi-modular approach to model selection in statistical natural language processing PhD thesis, Dep. Linguist., Univ Chicago:208 [Google Scholar]
  41. Hull DA. et al. 1996. Stemming algorithms: a case study for detailed evaluation. J. Assoc. Inf. Sci. Technol. 47:70–84 [Google Scholar]
  42. Jacquemin C. 1997. Guessing morphology from terms and corpora. Proceedings of the 20th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval NJ Belkin, AD Narasimhalu, P Willett, W Hersh, F Can, E Voorhees 156–67 New York: ACM [Google Scholar]
  43. Johnson M. 2008. Using adaptor grammars to identify synergies in the unsupervised acquisition of linguistic structures. Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics399–406 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  44. Johnson M, Griffiths TL, Goldwater S. 2007a. Adaptor grammars: a framework for specifying compositional nonparametric Bayesian models. Advances in Neural Information Processing Systems 19: Proceedings of the 2006 Conference B Schölkopf, J Platt, T Hoffman 641–48 Cambridge, MA: MIT Press [Google Scholar]
  45. Johnson M, Griffiths TL, Goldwater S. 2007b. Bayesian inference for PCFGs via Markov chain Monte Carlo. Human Language Technologies 2007: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics139–46 Rochester, NY: Assoc. Comput. Linguist. [Google Scholar]
  46. Kent A, Berry MM, Luehrs FU, Perry JW. 1955. Machine literature searching. VIII. Operational criteria for designing information retrieval systems. Am. Doc. 6:93–101 [Google Scholar]
  47. Khaliq B, Carroll J. 2013a. Induction of root and pattern lexicon for unsupervised morphological analysis of Arabic. Proceedings of the 6th International Joint Conference on Natural Language Processing1012–16 Nagoya, Jpn.: Asian Fed. Nat. Lang. Proc. [Google Scholar]
  48. Khaliq B, Carroll J. 2013b. Unsupervised induction of Arabic root and pattern lexicons using machine learning. Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2013)350–56 Hissar, Bulg.: Incoma [Google Scholar]
  49. Kučera H, Francis WN. 1967. Computational Analysis of Present-Day American English Providence, RI: Brown Univ. Press http://clu.uni.no/icame/manuals/BROWN/INDEX.htm [Google Scholar]
  50. Kumar A, Padró L, Oliver A. 2015. Learning agglutinative morphology of Indian languages with linguistically motivated adaptor grammars. Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2015)307–12 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  51. Lee JL. 2014. Automatic morphological alignment and clustering Tech. rep. TR-2014-07 Dep. Comput. Sci., Univ. Chicago: [Google Scholar]
  52. Lee JL. 2015. Morphological paradigms: computational structure and unsupervised learning. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop161–67 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  53. Lee JL, Goldsmith JA. 2016. Linguistica 5: unsupervised learning of linguistic structure Presented at Conf. N. Am. Chapter Assoc. Comput. Linguist, San Diego [Google Scholar]
  54. Lee YK, Haghighi A, Barzilay R. 2011. Modeling syntactic context improves morphological segmentation. Proceedings of the 15th Conference on Computational Natural Language Learning1–9 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  55. Lignos C, Yang C. Morphology and language acquisition. Cambridge Handbook of Morphology Cambridge, UK: Cambridge Univ. Press [Google Scholar]
  56. Mayfield J, McNamee P. 2003. Single n-gram stemming. Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR03)415–16 New York: ACM [Google Scholar]
  57. McCallum A. 2009. Joint inference for natural language processing. Proceedings of the 13th Conference on Computational Natural Language Learning (CoNLL)1 Stroudsburg, PA: Assoc. Comput. Linguist http://www.aclweb.org/anthology/W09-1101 [Google Scholar]
  58. Monson C, Carbonell J, Lavie A, Levin L. 2007. ParaMor: finding paradigms across morphology. Advances in Multilingual and Multimodal Information Retrieval900–7 Berlin: Springer [Google Scholar]
  59. Monson C, Lavie A, Carbonell J, Levin L. 2004. Unsupervised induction of natural language morphology inflection classes. Proceedings of the 7th Meeting of the ACL Special Interest Group in Computational Phonology: Current Themes in Computational Phonology and Morphology (SIGMorPhon '04)52–61 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  60. Neuvel S, Fulop SA. 2002. Unsupervised learning of morphology without morphemes. Proceedings of the 6th Workshop of the ACL Special Interest Group in Computational Phonology (SIGPHON02)31–40 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  61. O'Donnell TJ, Snedeker J, Tenenbaum JB, Goodman ND. 2011. Productivity and reuse in language. Proceedings of the 33rd Annual Meeting of the Cognitive Science Society L Carlson, C Hoelscher, TF Shipley 1613–18 Austin, TX: Cogn. Sci. Soc. [Google Scholar]
  62. O'Donnell TJ. 2015. Productivity and Reuse in Language: A Theory of Linguistic Computation and Storage Cambridge, MA: MIT Press [Google Scholar]
  63. Paice CD. 1994. An evaluation method for stemming algorithms. Proceedings of the 17th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval42–50 New York: Springer [Google Scholar]
  64. Pham M, Lee JL. 2014. Combining successor and predecessor frequencies to model truncation in Brazilian Portuguese Tech. rep. TR-2014-15, Dep Comput. Sci., Univ Chicago: [Google Scholar]
  65. Poon H, Domingos P. 2007. Joint inference in information extraction. Proceedings of the 22nd National Conference on Artificial Intelligence (AAAI07)913–18 Palo Alto, CA: AAAI [Google Scholar]
  66. Resnikoff HL, Dolby JL. 1965. The nature of affixing in written English. Mech. Transl. Comput. Linguist. 8:84–89 [Google Scholar]
  67. Resnikoff HL, Dolby JL. 1966. The nature of affixing in written English, part II. Mech. Transl. Comput. Linguist. 9:23–33 [Google Scholar]
  68. Rissanen J. 1989. Stochastic Complexity in Statistical Inquiry Singapore: World Sci. [Google Scholar]
  69. Roark B, Sproat RW. 2007. Computational Approaches to Morphology and Syntax Oxford, UK: Oxford Univ. Press [Google Scholar]
  70. Rodrigues P, Ćavar D. 2005. Learning Arabic morphology using information theory. Proceedings of the 41st Annual Meeting of the Chicago Linguistics Society49–60 Chicago: Chicago Univ. [Google Scholar]
  71. Rodrigues P, Ćavar D. 2007. Learning Arabic morphology using statistical constraint-satisfaction models. Perspectives on Arabic Linguistics 19: Papers from the 19th Annual Symposium on Arabic Linguistics E Benmamoun 63–75 Amsterdam/Philadelphia: Benjamins [Google Scholar]
  72. Ruokolainen T, Kohonen O, Sirts K, Grönroos SA, Kurimo M, Virpioja S. 2016. A comparative study on minimally supervised morphological segmentation. Comput. Linguist. 42:91–120 [Google Scholar]
  73. Schone P, Jurafsky D. 2000. Knowledge-free induction of morphology using latent semantic analysis. Proceedings of the 4th Conference on Computational Natural Language Learning (CoNLL 2000) and the 2nd Learning Language in Logic Workshop (LLL 2000) C Cardie, W Daelemans, C Nedellec, ETK Sang 67–72 New Brunswick, NJ: Assoc. Comput. Linguist. [Google Scholar]
  74. Schone P, Jurafsky D. 2001. Knowledge-free induction of inflectional morphologies. Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies1–9 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  75. Singla P, Domingos P. 2006. Entity resolution with Markov logic. Proceedings of the 6th International Conference on Data Mining (ICDM06)572–82 Piscataway, NJ: IEEE [Google Scholar]
  76. Sirts K, Alumäe T. 2012. A hierarchical Dirichlet process model for joint part-of-speech and morphology induction. Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies407–16 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  77. Snover MG, Brent MR. 2001. A Bayesian model for morpheme and paradigm identification. Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics490–98 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  78. Snover MG, Brent MR. 2002. A probabilistic model for learning concatenative morphology. Advances in Neural Information Processing Systems S Thrun, LK Obermayer 1513–20 Cambridge, MA: MIT Press [Google Scholar]
  79. Snover MG, Jarosz GE, Brent MR. 2002. Unsupervised learning of morphology using a novel directed search algorithm: taking the first step. Proceedings of the ACL-02 Workshop on Morphological and Phonological Learning11–20 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  80. Soricut R, Och F. Unsupervised morphology induction using word embeddings. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics1627–37 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  81. Sproat RW. 1992. Morphology and Computation Cambridge, MA: MIT Press [Google Scholar]
  82. Virpioja S, Turunen VT, Spiegler S, Kohonen O, Kurimo M. 2011. Empirical comparison of evaluation methods for unsupervised learning of morphology. Trait. Autom. Lang. 52:45–90 [Google Scholar]
  83. Wicentowski R. 2002. Minimally supervised morphological analysis by multimodal alignment PhD thesis, Dep. Comput. Sci., Johns Hopkins Univ., Baltimore, MD [Google Scholar]
  84. Wicentowski R. 2004. Multilingual noise-robust supervised morphological analysis using the WordFrame model. Proceedings of the 7th Meeting of the ACL Special Interest Group in Computational Phonology: Current Themes in Computational Phonology and Morphology70–77 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  85. Xanthos A. 2008. Sciences pour la communication 88: Apprentissage automatique de la morphologie. Le cas des structures racine-schème. Frankfurt: Peter Lang [Google Scholar]
  86. Yarowsky D, Wicentowski R. 2000. Minimally supervised morphological analysis by multimodal alignment. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics207–16 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  87. Zeman D. 2009. Using unsupervised paradigm acquisition for prefixes. Evaluating Systems for Multilingual and Multimodal Information Access: Proceedings of the 9th Workshop of the Cross-Language Evaluation Forum983–90 Berlin: Springer [Google Scholar]
  88. Zhang BT, Kim YT. 1990. Morphological analysis and synthesis by automated discovery and acquisition of linguistic rules. Proceedings of the 13th Conference on Computational Linguistics431–36 Stroudsburg, PA: Assoc. Comput. Linguist. [Google Scholar]
  89. Zipf GK. 1935. The Psychobiology of Language: An Introduction to Dynamic Philology Cambridge, MA: MIT Press [Google Scholar]
  90. Zipf GK. 1949. Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Cambridge, MA: Addison-Wesley [Google Scholar]
/content/journals/10.1146/annurev-linguistics-011516-034017
Loading
Computational Learning of Morphology
Annual Review of Linguistics 3, 85 (2017); https://doi.org/10.1146/annurev-linguistics-011516-034017
/content/journals/10.1146/annurev-linguistics-011516-034017
/content/journals/10.1146/annurev-linguistics-011516-034017
Loading

Data & Media loading...

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error