1932
0
  1. Home
  2. A-Z Publications
  3. Annual Review of Statistics and Its Application
  4. Volume 6, 2019
  5. Article

Annual Review of Statistics and Its Application

Volume 6, 2019

Sentiment Analysis

Abstract

Sentiment analysis labels a body of text as expressing either a positive or negative opinion, as in summarizing the content of an online product review. In this sense, sentiment analysis can be considered the challenge of building a classifier from text. Sentiment analysis can be done by counting the words from a dictionary of emotional terms, by fitting traditional classifiers such as logistic regression to word counts, or, most recently, by employing sophisticated neural networks. These methods progressively improve classification at the cost of increased computation and reduced transparency. A common sentiment analysis task, the classification of IMDb (Internet Movie Database) movie reviews, is used to illustrate the methods on a common task that appears frequently in the literature.

Keyword(s): classification, dictionary, lasso, lexicon, logistic regression, natural language processing, neural network, opinion mining
Loading

Article metrics loading...

/content/journals/10.1146/annurev-statistics-030718-105242
2019-03-07
2024-04-18
Loading full text...

Full text loading...

/deliver/fulltext/statistics/6/1/annurev-statistics-030718-105242.html?itemId=/content/journals/10.1146/annurev-statistics-030718-105242&mimeType=html&fmt=ahah

Literature Cited

  1. Agarwal A, Xie B, Vovsha I, Rambow O, Passonneau R. 2011. Sentiment analysis of Twitter data. Proceedings of the Workshop on Languages in Social Media30–38 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  2. Baldwin AL. 1942. Personal structure analysis: a statistical method for investigating the single personality. J. Abnorm. Soc. Psychol. 37:163–83
     [Google Scholar]
  3. Banea C, Mihalcea R, Wiebe J, Hassan S. 2008. Multilingual subjectivity analysis using machine translation. Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing127–35 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  4. Bengio Y, Courville A, Vincent P. 2013. Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intel. 35:1798–828
     [Google Scholar]
  5. Bengio Y, Ducharme R, Vincent P. 2003. A neural probabilistic language model. J. Mach. Learn. Res. 3:1137–55
     [Google Scholar]
  6. Blei DM, McAuliffe JD. 2007. Supervised topic models. Advances in Neural Information Processing Systems 20 (NIPS 2007) JC Platt, D Koller, Y Singer, ST Roweis121–28 Red Hook, NY: Curran
     [Google Scholar]
  7. Borgholt L, Simonsen P, Hovy D. 2015. The rating game: sentiment rating reproducibility from text. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing2527–32 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  8. Boudoukh J, Feldman R, Kogan S, Richardson M. 2013. Which news moves stock prices? A textual analysis. NBER Work. Pap. 18725
  9. Boyd-Graber J, Resnik P. 2010. Holistic sentiment analysis across languages: multilingual supervised latent Dirichlet allocation. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing45–55 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  10. Chan WS. 2003. Stock price reaction to news and no-news: drift and reversal after headlines. J. Financ. Econ. 70:223–60
     [Google Scholar]
  11. Chelba C, Mikolov T, Schuster M, Ge Q, Brants T et al. 2013. One billion word benchmark for measuring progress in statistical language modeling. arXiv:1312.3005 [cs.CL]
    [Google Scholar]
  12. Chen X, Sun Y, Athiwaratkun B, Cardie C, Weinberger K. 2016. Adversarial deep averaging networks for cross-lingual sentiment classification. arXiv:1606.01614 [cs.CL]
    [Google Scholar]
  13. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y. 2014. On the properties of neural machine translation: Encoder-decoder approaches. arXiv:1409.1259 [cs.CL]
    [Google Scholar]
  14. Chollet F, Allaire JJ 2017. Deep Learning with R Shelter Island, NY: Manning
  15. Dai AM, Le QV. 2015. Semi-supervised sequence learning. Advances in Neural Information Processing Systems 28 (NIPS 2015) C Cortes, ND Lawrence, DD Lee, M Sugiyama, R Garnett3079–87 Red Hook, NY: Curran:
     [Google Scholar]
  16. Dave K, Lawrence S, Pennock DM. 2003. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. Proceedings of the 12th International Conference on World Wide Web519–28 New York: ACM
     [Google Scholar]
  17. Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R. 1990. Indexing by latent semantic analysis. J. Am. Soc. Inform. Sci. 41:391–407
     [Google Scholar]
  18. Dhillon P, Foster DP, Kakade S, Ungar L. 2013. A risk comparison of ordinary least squares vs. ridge regression. J. Mach. Learn. Res. 14:1505–11
     [Google Scholar]
  19. Dhillon P, Rodu J, Foster D, Ungar L. 2012. Two step CCA: a new spectral method for estimating vector models of words. arXiv:1206.6403 [cs.CL]
    [Google Scholar]
  20. Dieng AB, Wang C, Gao J, Paisley J. 2016. TopicRNN: a recurrent neural network with long-range semantic dependency. arXiv:1611.01702 [cs.CL]
    [Google Scholar]
  21. Donoho D. 2017. 50 years of data science. J. Comput. Graph. Stat. 26:745–66
     [Google Scholar]
  22. Gehring J, Auli M, Grangier D, Dauphin YN. 2016. A convolutional encoder model for neural machine translation. arXiv:1611.02344 [cs.CL]
    [Google Scholar]
  23. Goldberg Y. 2016. A primer on neural network models for natural language processing. J. Artif. Intel. Res. 57:345–420
     [Google Scholar]
  24. Goldberg Y 2017. Neural Network Methods for Natural Language Processing Williston, VT: Morgan and Claypool
  25. González-Ibáñez R, Muresan S, Wacholder N. 2011. Identifying sarcasm in Twitter: a closer look. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics581–86 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  26. Goodfellow I, Bengio Y, Courville A 2016. Deep Learning Cambridge, MA: MIT Press
  27. Goodfellow IJ, Shlens J, Szegedy C. 2015. Explaining and harnessing adversarial examples. Presented at International Conference on Learning Representations (ICLR), San Diego, CA, May 7–9
  28. Gottschalk LA, Gleser GC 1969. The Measurement of Psychological States Through the Content Analysis of Verbal Behavior Berkeley: Univ. Calif. Press
  29. Gottschalk LA, Hambridge G. 1955. Verbal behavior analysis: a systematic approach to the problem of quantifying psychological processes. J. Proj. Tech. 19:387–409
     [Google Scholar]
  30. Hobbes T 1651. Leviathan London: Andrew Crooke
  31. Hochreiter S, Schmidhuber J. 1997. LSTM can solve hard long time lag problems. Advances in Neural Information Processing Systems (NIPS 9) MC Mozer, MI Jordan, T Petsche473–79 Cambridge, MA: MIT Press
     [Google Scholar]
  32. Hu M, Liu B. 2004. Mining and summarizing customer reviews. Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining168–77 New York: ACM
     [Google Scholar]
  33. Iyyer M, Manjunatha V, Boyd-Graber J, Daumé H III 2015. Deep unordered composition rivals syntactic methods for text classification. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing1681–91 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  34. Johnson R, Zhang T. 2015. Semi-supervised convolutional neural networks for text categorization via region embedding. Advances in Neural Information Processing Systems 28 (NIPS 2015) C Cortes, ND Lawrence, DD Lee, M Sugiyama, R Garnett919–27 Red Hook, NY: Curran
     [Google Scholar]
  35. Johnson R, Zhang T. 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. arXiv:1602.02373 [stat.ML]
    [Google Scholar]
  36. Jurafsky D, Martin JH. 2009. Speech and Language Processing. Upper Saddle River, NJ: Pearson 2nd ed.
  37. Kim Y, Jernite Y, Sontag D, Rush AM. 2016. Character-aware neural language models. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence2741–49 Palo Alto, CA: AAAI
     [Google Scholar]
  38. Kingma DP, Ba J. 2014. Adam: a method for stochastic optimization. arXiv:1412.6980 [cs.LG]
    [Google Scholar]
  39. Kouloumpis E, Wilson T, Moore JD. 2011. Twitter sentiment analysis: the good the bad and the OMG. Proceedings of the Fifth International AAAI Conference on Weblogs and Social Media538–41 Palo Alto, CA: AAAI
     [Google Scholar]
  40. Lai S, Xu L, Liu K, Zhao J. 2015. Recurrent convolutional neural networks for text classification. Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence2267–73 Palo Alto, CA: AAAI
     [Google Scholar]
  41. Le Q, Mikolov T. 2014. Distributed representations of sentences and documents. Proceedings of the 31st International Conference on Machine Learning EP Xing, T Jebara1188–96 Brookline, MA: Microtome
     [Google Scholar]
  42. Liu B 2012. Sentiment Analysis and Opinion Mining Williston, VT: Morgan and Claypool
  43. Liu B 2015. Sentiment Analysis: Mining Opinions, Sentiments, and Emotions Cambridge, UK: Cambridge Univ. Press
  44. Lloyd L, Kechagias D, Skiena S. 2005. Lydia: a system for large-scale news analysis. String Processing and Information Retrieval M Consens, G Navarro161–66 Berlin: Springer
     [Google Scholar]
  45. Loughran T, McDonald B. 2011. When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks. J. Finance 66:35–65
     [Google Scholar]
  46. Luca M. 2016. Reviews, reputation, and revenue: the case of yelp.com. Tech. Rep. 12-016, Harvard Bus. Sch., Cambridge, MA
    [Google Scholar]
  47. Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C. 2011. Learning word vectors for sentiment analysis. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics142–50 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  48. Manning CD, Ragahavan P, Schütze H 2009. An Introduction to Information Retrieval Cambridge, UK: Cambridge Univ. Press
  49. Manning CD, Schütz H 1999. Foundations of Statistical Natural Language Processing Cambridge, MA: MIT Press
  50. Mikolov T, Yih W, Zweig G. 2013. Linguistic regularities in continuous space word representations. Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies746–51 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  51. Miyato T, Dai AM, Goodfellow I. 2016a. Adversarial training methods for semi-supervised text classification. arXiv:1605.07725 [stat.ML]
    [Google Scholar]
  52. Miyato T, Maeda S, Koyama M, Nakae K, Ishii S. 2016b. Distributional smoothing with virtual adversarial training. arXiv:1507.00677 [stat.ML]
    [Google Scholar]
  53. Mohammad SM, Turney PD. 2013. Crowdsourcing a word-emotion association lexicon. Comput. Intel. 29:436–65
     [Google Scholar]
  54. Mosteller F, Wallace DL 1964. Inference and Disputed Authorship: The Federalist Boston: Addison-Wesley
  55. Ng V, Dasgupta S, Arifin SMN. 2006. Examining the role of linguistic knowledge sources in the automatic identification and classification of reviews. Proceedings of the COLING/ACL 2006 Main Conference Poster Sessions611–18 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  56. Nielsen FA. 2011. A new ANEW: Evaluation of a word list for sentiment analysis in microblogs. Proceedings of the ESWC2011 Workshop on Making Sense of Microposts M Rowe, M Stankovic, AS Dadzie, M Hardey93–98. http://ceur-ws.org/Vol-718/paper_16.pdf
     [Google Scholar]
  57. Nopp C, Hanbury A. 2015. Detecting risks in the banking system by sentiment analysis. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing591–600 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  58. Pang B, Lee L. 2004. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  59. Pang B, Lee L. 2005. Seeing stars: exploiting class relationships for sentiment categorization with respect to rating scales. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics115–24 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  60. Pang B, Lee L. 2008. Opinion mining and sentiment analysis. Found. Trends Inform. Retr. 2:1–135
     [Google Scholar]
  61. Pang B, Lee L, Vaithyanathan S. 2002. Thumbs up? Sentiment classification using machine learning techniques. Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing79–96 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  62. Pennington J, Socher R, Manning C. 2014. Glove: global vectors for word representation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing1532–43 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  63. Porter MF. 1980. An algorithm for suffix stripping. Program 14:130–37
     [Google Scholar]
  64. Qu Y, Shanahan J, Wiebe J 2004. Proceedings of the AAAI Spring Symposium on Exploring Attitude and Affect in Text: Theories and Applications Palo Alto, CA: AAAI Press
  65. Radford A, Jozefowicz R, Sutskever I. 2017. Learning to generate reviews and discovering sentiment. arXiv:1704.01444 [cs.LG]
    [Google Scholar]
  66. Riloff E, Qadir A, Surve P, Silva LD, Gilbert N, Huang R. 2013. Sarcasm as contrast between a positive sentiment and negative situation. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing704–714 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  67. Riloff E, Wiebe J. 2003. Learning extraction patterns for subjective expressions. Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing105–112 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  68. Rosenthal S, Farra N, Nakov P. 2017. Semeval-2017 task 4: sentiment analysis in Twitter. Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017)501–18 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  69. Silge J, Robinson D 2017. Text Mining with R: A Tidy Approach Sebastopol, CA: O'Reilly
  70. Socher R, Pennington J, Huang EH, Ng AY, Manning CD. 2011. Semi-supervised recursive autoencoders for predicting sentiment distributions. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing151–61 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  71. Socher R, Perelygin A, Wu J, Chuang J, Manning CD et al. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing1631–42 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  72. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15:1929–58
     [Google Scholar]
  73. Stone PJ, Murphy DC, Smith MS, Ogilvie DM 1966. The General Inquirer: A Computer Approach to Content Analysis Cambridge, MA: MIT Press
  74. Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D et al. 2014. Intriguing properties of neural networks. arXiv:1312.6199 [cs.CV]
    [Google Scholar]
  75. Taddy M. 2013. Multinomial inverse regression for text analysis. J. Am. Stat. Assoc. 108:755–70
     [Google Scholar]
  76. Tang D, Wei F, Qin B, Zhou M, Liu T. 2014. Building large-scale Twitter-specific sentiment lexicon: a representation learning approach. Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers172–82 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  77. Tang H, Tan S, Cheng X. 2009. A survey on sentiment detection of reviews. Expert Syst. Appl. 36:10760–73
     [Google Scholar]
  78. Tetlock PC. 2007. Giving content to investor sentiment: the role of media in the stock market. J. Finance 62:1139–68
     [Google Scholar]
  79. Thisted R, Efron B. 1987. Did Shakespeare write a newly-discovered poem. Biometrika 74:445–55
     [Google Scholar]
  80. Tong R. 2001. An operational system for detecting and tracking opinions in on-line discussions. Proceedings of the SIGIR Workshop on Operational Text Classification1–6 New York: ACM
     [Google Scholar]
  81. Turney PD. 2002. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics417–24 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  82. Turney PD, Pantel P. 2010. From frequency to meaning: vector space models of semantics. J. Artif. Intel. Res. 37:141–88
     [Google Scholar]
  83. Wang J, Shen X, Sun Y, Qu A. 2016. Classification with unstructured predictors and an application to sentiment analysis. J. Am. Stat. Assoc. 111:1242–53
     [Google Scholar]
  84. Wilson T, Wiebe J, Hoffmann P. 2005. Recognizing contextual polarity in phrase-level sentiment analysis. Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing347–54 Stroudsburg, PA: Assoc. Comput. Linguist.
     [Google Scholar]
  85. Zhang C, Bengio S, Hardt M, Recht B, Vinyals O. 2016. Understanding deep learning requires rethinking generalization. arXiv:1611.03530 [cs.LG]
    [Google Scholar]
  86. Zhang X, Zhao J, LeCun Y. 2015. Character-level convolutional networks for text classification. Advances in Neural Information Processing Systems 28 (NIPS 2015) C Cortes, ND Lawrence, DD Lee, M Sugiyama, R Garnett649–57 Red Hook, NY: Curran
     [Google Scholar]
/content/journals/10.1146/annurev-statistics-030718-105242
Loading
Sentiment Analysis
Annual Review of Statistics and Its Application 6, 287 (2019); https://doi.org/10.1146/annurev-statistics-030718-105242
/content/journals/10.1146/annurev-statistics-030718-105242
/content/journals/10.1146/annurev-statistics-030718-105242
Loading

Data & Media loading...

  • Article Type: Review Article

Most Cited Most Cited RSS feed

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error