基于一维卷积混合神经网络的文本情感分类

doi:10.11772/j.issn.1001-9081.2018122477

计算机应用 ›› 2019, Vol. 39 ›› Issue (7): 1936-1941.DOI: 10.11772/j.issn.1001-9081.2018122477

基于一维卷积混合神经网络的文本情感分类

陈郑淏, 冯翱, 何嘉

成都信息工程大学计算机学院, 成都 610225

收稿日期:2018-12-17 修回日期:2019-02-12 出版日期:2019-07-10 发布日期:2019-03-29
通讯作者: 何嘉
作者简介:陈郑淏(1993-),男,四川成都人,硕士研究生,CCF会员,主要研究方向:自然语言处理、深度学习;冯翱(1978-),男,四川广安人,副教授,博士,CCF会员,主要研究方向:信息检索、数据挖掘、机器学习;何嘉(1968-),女,四川成都人,教授,博士,CCF会员,主要研究方向:计算智能、人工智能。
基金资助:
四川省科技厅应用基础重点项目（2017JY0011）。

Text sentiment classification based on 1D convolutional hybrid neural network

CHEN Zhenghao, FENG Ao, HE Jia

School of Computer Science, Chengdu University of Information Technology, Chengdu Sichuan 610225, China

Received:2018-12-17 Revised:2019-02-12 Online:2019-07-10 Published:2019-03-29
Supported by:
This work is partially supported by the Key Project of Applied Basic Research of Sichuan Science and Technology Department (2017JY0011).

摘要/Abstract

摘要：

针对情感分类中传统二维卷积模型对特征语义信息的损耗以及时序特征表达能力匮乏的问题，提出了一种基于一维卷积神经网络（CNN）和循环神经网络（RNN）的混合模型。首先，使用一维卷积替换二维卷积以保留更丰富的局部语义特征；再由池化层降维后进入循环神经网络层，整合特征之间的时序关系；最后，经过softmax层实现情感分类。在多个标准英文数据集上的实验结果表明，所提模型在SST和MR数据集上的分类准确率与传统统计方法和端到端深度学习方法相比有1至3个百分点的提升，而对网络各组成部分的分析验证了一维卷积和循环神经网络的引入有助于提升分类准确率。

关键词: 情感分类, 卷积神经网络, 循环神经网络, 词向量, 深度学习

Abstract:

Traditional 2D convolutional models suffer from loss of semantic information and lack of sequential feature expression ability in sentiment classification. Aiming at these problems, a hybrid model based on 1D Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) was proposed. Firstly, 2D convolution was replaced by 1D convolution to retain richer local semantic features. Then, a pooling layer was used to reduce data dimension and the output was put into the recurrent neural network layer to extract sequential information between the features. Finally, softmax layer was used to realize the sentiment classification. The experimental results on multiple standard English datasets show that the proposed model has 1-3 percentage points improvement in classification accuracy compared with traditional statistical method and end-to-end deep learning method. Analysis of each component of network verifies the value of introduction of 1D convolution and recurrent neural network for better classification accuracy.

Key words: sentiment classification, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), word embedding, deep learning

中图分类号:

TP391.1
TP18

陈郑淏, 冯翱, 何嘉. 基于一维卷积混合神经网络的文本情感分类[J]. 计算机应用, 2019, 39(7): 1936-1941.

CHEN Zhenghao, FENG Ao, HE Jia. Text sentiment classification based on 1D convolutional hybrid neural network[J]. Journal of Computer Applications, 2019, 39(7): 1936-1941.

参考文献

[1] 周立柱,贺宇凯,王建勇.情感分析研究综述[J].计算机应用,2008,28(11):2725-2728.(ZHOU L Z, HE Y K, WANG J Y. Survey on research of sentiment analysis[J]. Journal of Computer Applications, 2008,28(11):2725-2728.)
[2] 赵妍妍,秦兵,刘挺.文本情感分析[J].软件学报,2010,21(8):1834-1848.(ZHAO Y Y, QIN B, LIU T. Sentiment analysis[J]. Journal of Software, 2010, 21(8):1834-1848.)
[3] ZHANG Y, WALLACE B. A sensitivity analysis of (and practitioners' guide to) convolutional neural networks for sentence classification[EB/OL]. (2016-04-06)[2018-06-07]. https://arxiv.org/abs/1510.03820.
[4] KIM Y. Convolutional neural networks for sentence classification[EB/OL]. (2014-09-03)[2018-06-01]. https://arxiv.org/abs/1408.5882.
[5] ZHANG L, WANG S, LIU B. Deep learning for sentiment analysis:a survey[J]. Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery, 2018, 8(4):e1253.
[6] KIM S-M, HOVY E. Extracting opinions, opinion holders, and topics expressed in online news media text[C]//Proceedings of the 2006 Workshop on Sentiment and Subjectivity in Text. Stroudsburg, PA:Association for Computational Linguistics, 2006:1-8.
[7] TURNEY P D. Thumbs up or thumbs down?:semantic orientation applied to unsupervised classification of reviews[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2002:417-424.
[8] HU M, LIU B. Mining and summarizing customer reviews[C]//Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2004:168-177.
[9] PANG B, LEE L, VAITHYANATHAN S. Thumbs up?:sentiment classification using machine learning techniques[C]//Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing-Volume 10. Stroudsburg, PA:Association for Computational Linguistics, 2002:79-86.
[10] MOHAMMAD S M, KIRITCHENKO S, ZHU X. NRC-Canada:building the state-of-the-art in sentiment analysis of tweets[EB/OL]. (2013-08-28)[2018-07-02]. https://arxiv.org/abs/1308.6242.
[11] KIM S-M, HOVY E. Automatic identification of pro and con reasons in online reviews[C]//Proceedings of the 2006 COLING/ACL on Main Conference Poster Sessions. Stroudsburg, PA:Association for Computational Linguistics, 2006:483-490.
[12] MEDHAT W, HASSAN A, KORASHY H. Sentiment analysis algorithms and applications:a survey[J]. Ain Shams Engineering Journal, 2014, 5(4):1093-1113.
[13] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. Journal of Machine Learning Research, 2003, 3:1137-1155.
[14] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//NIPS'13:Proceedings of the 26th International Conference on Neural Information Processing Systems. North Miami Beach, FL:Curran Associates Inc., 2013:3111-3119.
[15] PENNINGTON J, SOCHER R, MANNING C. GloVe:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2014:1532-1543.
[16] SOCHER R, PERELYGIN A, WU J, et al. Recursive deep models for semantic compositionality over a sentiment treebank[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2013:1631-1642.
[17] SOCHER R, PENNINGTON J, HUANG E H, et al. Semi-supervised recursive autoencoders for predicting sentiment distributions[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2011:151-161.
[18] QIAN Q, TIAN B, HUANG M, et al. Learning tag embeddings and tag-specific composition functions in recursive neural network[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2015:1365-1374.
[19] TAI K S, SOCHER R, MANNING C D. Improved semantic representations from tree-structured long short-term memory networks[EB/OL]. (2015-05-30)[2018-08-10]. https://arxiv.org/abs/1503.00075.
[20] IRSOY O, CARDIE C. Opinion mining with deep recurrent neural networks[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2014:720-728.
[21] LIU P, QIU X, HUANG X. Recurrent neural network for text classification with multi-task learning[EB/OL]. (2016-05-17)[2018-08-01]. https://arxiv.org/abs/1605.05101.
[22] QIAN Q, HUANG M, LEI J, et al. Linguistically regularized LSTMs for sentiment classification[EB/OL]. (2017-04-25)[2018-08-15]. https://arxiv.org/abs/1611.03949.
[23] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[EB/OL]. (2014-04-08)[2018-07-16]. https://arxiv.org/abs/1404.2188.
[24] ZHOU C, SUN C, LIU Z, et al. A C-LSTM neural network for text classification[EB/OL]. (2015-11-30)[2018-08-22]. https://arxiv.org/abs/1511.08630.
[25] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural lan-guage processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12:2493-2537.
[26] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[27] GRAVES A, JAITLY N, MOHAMED A. Hybrid speech recognition with deep bidirectional LSTM[C]//Proceedings of the 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. Piscataway, NJ:IEEE, 2013:273-278.
[28] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. (2013-09-07)[2018-09-02]. https://arxiv.org/abs/1301.3781.
[29] LIU B. Sentiment Analysis and Opinion Mining[M]. San Rafael, CA:Morgan and Claypool Publishers, 2012:1-167.
[30] McCANN B, BRADBURY J, XIONG C, et al. Learned in translation:contextualized word vectors[C]//NIPS 2017:Proceedings of the 31st Annual Conference on Neural Information Processing Systems. North Miami Beach, FL:Curran Associates Inc., 2017:6297-6308.
[31] PETERS M E, NEUMANN M, IYYER M et al. Deep contextualized word representations[EB/OL]. (2018-03-22)[2018-10-21]. https://arxiv.org/abs/1802.05365.
[32] HOWARD J, RUDER S. Universal language model fine-tuning for text classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2018:328-339.
[33] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. (2018-06-11)[2018-10-22]. https://blog.openai.com/language-unsupervised/.
[34] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//NIPS 2017:Proceedings of the 31st Annual Conference on Neural Information Processing Systems. North Miami Beach, FL:Curran Associates Inc., 2017:5998-6008.
[35] DEVLIN J, CHANG M-W, LEE K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL]. (2018-10-11)[2018-11-13]. https://arxiv.org/abs/1810.04805.

基于一维卷积混合神经网络的文本情感分类

Text sentiment classification based on 1D convolutional hybrid neural network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[2]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[3]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[4]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[5]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[6]	刘子辰, 李小娟, 韦伟. 基于循环神经网络的专利价格自动评估[J]. 计算机应用, 2021, 41(9): 2532-2538.
[7]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[8]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[9]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[10]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[11]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[12]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[13]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[14]	王伟, 赵尔平, 崔志远, 孙浩. 基于HowNet义原和Word2vec词向量表示的多特征融合消歧方法[J]. 计算机应用, 2021, 41(8): 2193-2198.
[15]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.