Text sentiment analysis based on gated recurrent unit and capsule features

doi:10.11772/j.issn.1001-9081.2020010128

Abstract

Abstract: Aiming at the problems that simple Recurrent Neural Network (RNN) cannot memorize information for a long time and single Convolutional Neural Network (CNN) lacks the ability to capture the semantics of text context, in order to improve the accuracy of text classification, a sentiment analysis model G-Caps (Gated Recurrent Unit-Capsule) was proposed, which combines Gated Recurrent Unit (GRU) and capsule features. First, the contextual global features of the text were captured through GRU in order to obtain the global scalar information. Second, the captured information was iterated through the dynamic routing algorithm at the initial capsule layer to obtain the vectorized feature information representing the overall attributes of the text. Finally, the features were combined in the main capsule part to obtain more accurate text attributes, and the sentiment polarity of the text was analyzed according to the intensity of each feature. Experimental results on the benchmark dataset MR (Movie Reviews) showed that compared with the CNN + INI (Convolutional Neural Network + Initializing convolutional filters) and CL_CNN (Critic Learning_Convolutional Neural Network) methods, G-Caps had the classification accuracy increased by 3.1 percentage points and 0.5 percentage points respectively. It can be seen that the G-Caps model effectively improves the accuracy of text sentiment analysis in practice.

Key words: sentiment analysis, weight sharing, capsule model, Gated Recurrent Unit (GRU) dynamic routing, text attribute

摘要： 针对简单的循环神经网络（RNN）无法长时间记忆信息和单一的卷积神经网络（CNN）缺乏捕获文本上下文语义的能力的问题，为提升文本分类的准确率，提出一种门控循环单元（GRU）和胶囊特征融合的情感分析模型G-Caps。首先通过GRU捕捉文本的上下文全局特征，获得整体标量信息；其次在初始胶囊层将捕获的信息通过动态路由算法进行迭代，获取到表示文本整体属性的向量化的特征信息；最后在主胶囊部分进行特征间的组合以求获得更准确的文本属性，并根据各个特征的强度大小分析文本的情感极性。在基准数据集MR上进行的实验的结果表明，与初始卷积滤波器的CNN（CNN+INI）和批判学习的CNN（CL_CNN）方法相比，G-Caps的分类准确率分别提升了3.1个百分点和0.5个百分点。由此可见，G-Caps模型有效地提高了实际应用中文本情感分析的准确性。

关键词: 情感分析, 权重共享, 胶囊模型, 门控循环单元动态路由, 文本属性

CLC Number:

YANG Yunlong, SUN Jianqiang, SONG Guochao. Text sentiment analysis based on gated recurrent unit and capsule features[J]. Journal of Computer Applications, 2020, 40(9): 2531-2535.

杨云龙, 孙建强, 宋国超. 基于门控循环单元和胶囊特征的文本情感分析[J]. 计算机应用, 2020, 40(9): 2531-2535.

References

[1] SOCHER R, PENNINGTON J, HUANG E H, et al. Semisupervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the 2011 Conference on empirical methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2011:151-161.
[2] SOCHER R,PERELYGIN A,WU J,et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2013:1631-1642.
[3] MIKOLOV T. Statistical language models based on neural networks[D]. Brno:Brno University of Technology,2012:47-61.
[4] TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-term memory networks[EB/OL].[2019-09-07]. https://arxiv.org/pdf/1503.00075.pdf.
[5] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[EB/OL].[2019-09-07]. https://arxiv.org/pdf/1404.2188.pdf.
[6] KIM Y. Convolutional neural networks for sentence classification[EB/OL].[2019-09-13]. https://arxiv.org/pdf/1408.5882.pdf.
[7] 刘书齐, 王以松, 陈攀峰. 基于CNN-ATTBiLSTM的文本情感分析[J]. 贵州大学学报(自然科学版),2019,36(2):85-89.(LIU S Q,WANG Y S,CHEN P F. A CNN-ATTBiLSTM model neural network for text sentiment analysis[J]. Journal of Guizhou University(Natural Sciences),2019,36(2):85-89.)
[8] 王丽亚, 刘昌辉, 蔡敦波, 等. 基于CNN-BiLSTM网络引入注意力模型的文本情感分析[J]. 武汉工程大学学报,2019,41(4):386-391.(WANG L Y,LIU C H,CAI D B,et al. Text sentiment analysis based on CNN-BiLSTM network and attention model[J]. Journal of Wuhan Institute of Technology,2019,41(4):386-391.)
[9] ZHANG B,XU X,LI X,et al. Sentiment analysis through critic learning for optimizing convolutional neural networks with rules[J]. Neurocomputing,2019,356:21-30.
[10] YIN W,SCHÜTZE H,XIANG B,et al. ABCNN:attention-based convolutional neural network for modeling sentence pairs[J]. Transactions of the Association for Computational Linguistics, 2016,4:259-272.
[11] ZHOU P,SHI W,TIAN J,et al. Attention-based bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:207-212.
[12] 杨开漠, 吴明芬, 陈涛. 广义文本情感分析综述[J]. 计算机应用,2019,39(S2):6-14.(YANG K M,WU M F,CHEN T. Generalized text sentiment analysis review[J]. Journal of Computer Applications,2019,39(S2):6-14.)
[13] HOCHREITER S,SCHMIDHUBER J. Long short-term memory[J]. Neural Computation,1997,9(8):1735-1780.
[14] 蔡国永, 林强, 任凯琪. 基于域对抗网络和BERT的跨领域文本情感分析[J]. 山东大学学报(工学版),2020,50(1):1-7,20. (CAI G Y,LIN Q,REN K Q. Cross-domain text sentiment classification based on domain-adversarial network and BERT[J]. Journal of Shandong University(Engineering Science),2020,50(1):1-7,20.)
[15] KIRITCHENKO S,ZHU X,CHERRY C,et al. NRC-Canada-2014:detecting aspects and sentiment in customer reviews[C]//Proceedings of the 8th International Workshop on Semantic Evaluation. Stroudsburg, PA:Association for Computational Linguistics,2014:437-442.
[16] 林江豪, 顾也力, 周咏梅, 等. 基于表情符号的情感词典的构建研究[J]. 计算机技术与发展,2019,29(6):181-185.(LIN J H,GU Y L,ZHOU Y M,et al. Research on building sentiment lexicon based on emoticons[J]. Computer Technology and Development,2019,29(6):181-185.)
[17] QIAN Q,HUANG M,LEI J,et al. Linguistically regularized LSTM for sentiment classification[EB/OL].[2019-08-18]. https://arxiv.org/pdf/1611.03949.pdf.
[18] VO D T,ZHANG Y. Don't count,predict! an automatic approach to learning sentiment lexicons for short text[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:219-224.
[19] WANG Y,HUANG M,ZHU X,et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2016:606-615.
[20] WANG Y,SUN A,HAN J,et al. Sentiment analysis by capsules[C]//Proceedings of the 2018 World Wide Web Conference. Republic and Canton of Geneva,CHE:International World Wide Web Conferences Steering Committee,2018:1165-1174.
[21] SABOUR S,FROSST N,HINTON G E. Dynamic routing between capsules[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc.,2017:3856-3866.
[22] PANG B,LEE L. Movie review data[EB/OL].[2019-10-08]. https://www.cs.cornell.edu/people/pabo/movie-review-data/.
[23] WANG J,YU L C,LAI K R,et al. Dimensional sentiment analysis using a regional CNN-LSTM model[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2016:225-230.
[24] CHEN T,XU R,HE Y,et al. Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN[J]. Expert Systems with Applications,2017,72:221-230.
[25] HU Z,MA X,LIU Z,et al. Harnessing deep neural networks with logic rules[EB/OL].[2019-09-25]. https://arxiv.org/pdf/1603.06318.pdf.
[26] LI S,ZHAO Z,LIU T,et al. Initializing convolutional filters with semantic features for text classification[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2017:1884-1889.