Abstract：In the traditional Convolutional Neural Network (CNN), the information cannot be transmitted to each other between the neurons of the same layer, the feature information at the same layer cannot be fully utilized, making the lack of the representation of the characteristics of the sentence system. As the result, the feature learning ability of model is limited and the text classification effect is influenced. Aiming at the problem, a model based on joint network CNN-BiGRU and attention mechanism was proposed. In the model, the CNN-BiGRU joint network was used for feature learning. Firstly, deep-level phrase features were extracted by CNN. Then, the Bidirectional Gated Recurrent Unit (BiGRU) was used for the serialized information learning to obtain the characteristics of the sentence system and strengthen the association of CNN pooling layer features. Finally, the effective feature filtering was completed by adding attention mechanism to the hidden state weighted calculation. Comparative experiments show that the method achieves 91.93% F1 value and effectively improves the accuracy of text classification with small time cost and good application ability.
王丽亚, 刘昌辉, 蔡敦波, 卢涛. CNN-BiGRU网络中引入注意力机制的中文文本情感分析[J]. 计算机应用, 2019, 39(10): 2841-2846.
WANG Liya, LIU Changhui, CAI Dunbo, LU Tao. Chinese text sentiment analysis based on CNN-BiGRU network with attention mechanism. Journal of Computer Applications, 2019, 39(10): 2841-2846.
[1] 魏韡, 向阳, 陈千. 中文文本情感分析综述[J]. 计算机应用, 2011, 31(12):3321-3323. (WEI W, XIANG Y, CHEN Q. Survey on Chinese text sentiment analysis[J]. Journal of Computer Applications, 2011, 31(12):3321-3323.) [2] TURNEY P D. Thumbs up or thumbs down?:semantic orientation applied to unsupervised classification of reviews[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2002:417-424. [3] NI X, XUE G, LING X, et al. Exploring in the weblog space by detecting informative and affective articles[C]//Proceedings of the 16th International Conference on World Wide Web. New York:ACM, 2007:281-290. [4] PANG B, LEE L, VAITHYANATHAN S. Thumbs up?:sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2002:79-86. [5] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003, 3:1137-1155. [6] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL].[2017-08-04]. http://www.surdeanu.info/mihai/teaching/ista555-spring15/readings/mikolov2013.pdf. [7] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[EB/OL].[2019-01-10]. http://papers.nips.cc/paper/5021-distributed-representations-of-words-and-phrases-and-their-compositionality.pdf. [8] MNIH A, HINTON G E. A scalable hierarchical distributed language model[C]//Proceedings of the 21st International Conference on Neural Information Processing. New York:Curran Associates Inc., 2008:1081-1088. [9] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2014:655-665. [10] KIM Y. Convolutional neural networks for sentence classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proceeding. Stroudsburg, PA:Association for Computational Linguistics, 2014:1746-1751. [11] LEE J Y, DERNONCOURT F. Sequential short-text classification with recurrent and convolutional neural networks[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016, 515-520. [12] CHO K, van MERRIENBOER B, GULCEHRE C, et al. Learning phrase representions using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing.Stroudsburg, PA:Association for Computational Linguistics, 2014:1724-1734. [13] EBRAHIMI J, DOU D. Chain based RNN for relation classification[C]//Proceedings of the 2015 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2015:1244-1249. [14] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780. [15] ZHOU C, SUN C, LIU Z, et al. A C-LSTM neural network for text classification[EB/OL].[2019-01-09].https://arxiv.org/abs/1511.08630. [16] XIAO Z, LIANG P. Chinese sentiment analysis using bidirectional LSTM with word embedding[C]//Proceedings of the 2016 International Conference on Cloud Computing and Security, LNSC 10040. Berlin:Springer, 2016:601-610. [17] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL].[2018-03-20]. https://arxiv.org/pdf/1409.0473v7.pdf. [18] MNIH V, HEESS N, GRAVES A, et al. Recurrent models of visual attention[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 2014:2204-2212. [19] XU K, BA J, KIROS R, et al. Show, attend and tell:neural image caption generation with visual attention[EB/OL].[2018-03-20]. https://arxiv.org/pdf/1502.03044.pdf. [20] LUONG M PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2015:1412-1421. [21] 胡荣磊, 芮璐, 齐筱, 等. 基于循环神经网络和注意力模型的文本情感分析[J/OL]. 计算机应用研究, 2019, 36(11).[2018-12-10]. http://www.arocmag.com/article/02-2019-11-025.html. (HU R L, RUI L, QI X, et al. Text sentiment analysis based on recurrent neural network and attention model[J/OL]. Application Research of Computers, 2019, 36(11).[2018-12-10]. http://www.arocmag.com/article/02-2019-11-025.html.) [22] 王伟, 孙玉霞, 齐庆杰, 等. 基于BiGRU-Attention神经网络的文本情感分类模型[J/OL]. 计算机应用研究, 2018, 36(12)[2018-12-10]. http://www.arocmag.com/article/02-2019-12-045.html (WANG W, SUN Y X, QI Q J, et al. Text sentiment classification model based on BiGRU-Attention neural network[J/OL]. Application Research of Computers, 2018, 36(12)[2018-12-10]. http://www.arocmag.com/article/02-2019-12-045.html.) [23] 陈洁, 邵志清, 张欢欢, 等. 基于并行混合神经网络模型的短文本情感分析[J/OL]. 计算机应用, 2019.[2018-12-10]. http://kns.cnki.net/kcms/detail/51.1307.TP.20190329.1643.008.html. (CHEN J, SHAO Z Q, ZHANG H H, et al. Short text sentiment analysis based on parallel hybrid neural network model[J/OL]. Journal of Computer Applications, 2019.[2018-12-10]. http://kns.cnki.net/kcms/detail/51.1307.TP.20190329.1643.008.html.) [24] 常丹, 王玉珍. 基于SVM的用户评论情感分析方法研究[J]. 枣庄学院学报, 2019, 36(2):73-78. (CHANG D, WANG Y Z. Research on the method of user comment sentiment analysis based on SVM[J]. Journal of Zaozhuang University, 2019, 36(2):73-78.) [25] 王煜涵, 张春云, 赵宝林, 等. 卷积神经网络下的Twitter文本情感分析[J]. 数据采集与处理, 2018, 33(5):921-927. (WANG Y H, ZHANG C Y, ZHAO B L, et al. Sentiment analysis of twitter data based on CNN[J]. Journal of Data Acquisition and Processing, 2018, 33(5):921-927.)
2019, Vol. 39 
