基于多尺度卷积和门控机制的注意力情感分析模型

doi:10.11772/j.issn.1001-9081.2021081448

摘要/Abstract

摘要：

针对现有的文档级情感分析模型大多只是考虑从词级对文本进行编码的问题，提出了一种基于多尺度卷积和门控机制的注意力情感分析模型。首先，使用多尺度卷积捕获不同粒度的局部相关性，从而得到更多不同层次的文本语义信息并形成更丰富的文本表示；其次，考虑到用户个性及产品信息对文本情感分类的影响，将全局用户产品信息融合到注意力中捕捉与用户和产品相关度较高的关键语义成分来生成文档表示；然后，引入门控机制来控制情感信息流向汇集层的路径；最后，通过全连接层和argmax函数实现情感分类。实验结果表明，与基准模型中性能最好的相比，所提模型在IMDB和Yelp2014两个数据集上的情感分类准确率分别提高了1.2个百分点和0.7个百分点，并且在IMDB和Yelp2013数据集上获得了最小的均方根误差（RMSE）。

关键词: 情感分析, 卷积神经网络, 注意力机制, 多尺度卷积, 门控机制

Abstract:

Aiming at the problem that most of existing models for document-level sentiment analysis only consider encoding text at word level， an attention sentiment analysis model based on multi-scale convolution and gating mechanism was proposed. Firstly， in order to obtain more different levels of text semantic information and form a richer text representation， the multi-scale convolution was used to capture local correlations of different granularities. Secondly， considering the influence of user personality and product information on text sentiment classification， the global information of users and products was integrated into attention to capture key semantic components which were highly related to users and products to form the document representation. Thirdly， a gating mechanism was introduced to control the path of emotional information flowing to collection layer. Finally， the sentiment classification was realized through the fully connected layer and argmax function. The experimental results show that， compared with the baseline model with the most advanced performance， the proposed algorithm has the sentiment classification accuracy on IMDB and Yelp2014 datasets improved by 1.2 percentage points and 0.7 percentage points respectively， and obtains the smallest Root Mean Squared Error （RMSE） on IMDB and Yelp2013 datasets.

Key words: sentiment analysis, Convolutional Neural Network (CNN), attention mechanism, multi-scale convolution, gating mechanism

中图分类号:

TP391

衡红军, 徐天宝. 基于多尺度卷积和门控机制的注意力情感分析模型[J]. 计算机应用, 2022, 42(9): 2674-2679.

Hongjun HENG, Tianbao XU. Attention sentiment analysis model based on multi-scale convolution and gating mechanism[J]. Journal of Computer Applications, 2022, 42(9): 2674-2679.

图/表 5

参考文献 21

1	PANG B， LEE L， VAITHYANATHAN S. Thumbs up？ sentiment classification using machine learning techniques［C］// Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2002：79-86. 10.3115/1118693.1118704
2	ABBASI A， FRANCE S， ZHANG Z， et al. Selecting attributes for sentiment classification using feature relation networks［J］. IEEE Transactions on Knowledge and Data Engineering， 2011， 23（3）：447-462. 10.1109/tkde.2010.110
3	HU X， TANG L， TANG J， et al. Exploiting social relations for sentiment analysis in microblogging［C］// Proceedings of the 6th ACM International Conference on Web Search and Data Mining. New York： ACM， 2013：537-546. 10.1145/2433396.2433465
4	KIM Y. Convolutional neural networks for sentence classification［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2014：1746-1751. 10.3115/v1/d14-1181
5	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）：1735-1780. 10.1162/neco.1997.9.8.1735
6	TANG D Y， QIN B， LIU T. Learning semantic representations of users and products for document level sentiment classification［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2015： 1014-1023. 10.3115/v1/p15-1098
7	CHEN H M， SUN M S， TU C C， et al. Neural sentiment classification with user and product attention［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2016： 1650-1659. 10.18653/v1/d16-1171
8	KIM J， AMPLAYO R K， LEE K， et al. Categorical metadata representation for customized text classification［J］. Transactions of the Association for Computational Linguistics， 2019， 7： 201-215. 10.1162/tacl_a_00263
9	蒋宗礼，张静. 融合用户和产品信息的多头注意力情感分类模型［J］. 计算机系统应用， 2020，29（7）：131-138. 10.15888/j.cnki.csa.007447]
	JIANG Z L， ZHANG J. Multi-head attention model with user and product information for sentiment classification［J］. Computer Systems and Applications， 2020， 29（7）：131-138. 10.15888/j.cnki.csa.007447]
10	LIN J Y， SUN X， MA S M， et al. Global encoding for abstractive summarization［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 163-169. 10.18653/v1/p18-2027
11	DAUPHIN Y N， FAN A， AULI M， et al. Language modeling with gated convolutional networks［C］// Proceedings of the 34th International Conference on Machine Learning. New York： JMLR.org， 2017： 933-941.
12	MANNING C D， SURDEANU M， BAUER J， et al. The Stanford CoreNLP natural language processing toolkit［C］// Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2014： 55-60. 10.3115/v1/p14-5010
13	PENNINGTON J， SOCHER R， MANNING C D. GloVe： global vectors for word representation［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2014：1532-1543. 10.3115/v1/d14-1162
14	ZEILER M D. ADADELTA： an adaptive learning rate method［EB/OL］. （2012-12-22）［2021-11-07］..
15	GUI L， XU R F， HE Y L， et al. Intersubjectivity and sentiment： from language to knowledge［C］// Proceedings of the 25th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2016： 2789-2795. 10.1142/9789813223615_0010
16	CHEN T， XU R. Learning user and product representations for sentiment analysis using sequence modeling［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics，2016： 1-11.
17	ZHU P C， YANG Y J. Parallel multi-feature attention on neural sentiment classification［C］// Proceedings of the 8th International Symposium on Information and Communication Technology. New York： ACM， 2017： 181-188. 10.1145/3155133.3155193
18	MA D H， LI S J， ZHANG X D， et al. Cascading multiway attention for document-level sentiment classification［C］// Proceedings of the 8th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. ［S.l.］： Asian Federation of Natural Language Processing， 2017： 634-643.
19	DOU Z Y. Capturing user and product information for document level sentiment analysis with deep memory network［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2017： 521-526. 10.18653/v1/d17-1054
20	LONG Y F， MA M Y， LU Q， et al. Dual memory network model for biased product review classification［C］// Proceedings of the 9th Workshop on Computational Approaches to Subjectivity， Sentiment and Social Media Analysis. Stroudsburg， PA： Association for Computational Linguistics， 2018： 140-148. 10.18653/v1/w18-6220
21	AMPLAYO R K， KIM J， SUNG S， et al. Cold-start aware user and product attention for sentiment classification［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 2535-2544. 10.18653/v1/p18-1236

数据集	类别数	评论数	用户数	产品数	评论/用户	评论/产品
IMDB	10	84 919	1 310	1 635	64.82	51.94
Yelp2013	5	231 163	4 818	4 194	48.42	48.36
Yelp2014	5	78 966	1 631	1 633	47.97	55.11

数据集	类别数	评论数	用户数	产品数	评论/用户	评论/产品
IMDB	10	84 919	1 310	1 635	64.82	51.94
Yelp2013	5	231 163	4 818	4 194	48.42	48.36
Yelp2014	5	78 966	1 631	1 633	47.97	55.11

模型	IMDB		Yelp2013		Yelp2014
模型	Accuracy/%	RMSE	Accuracy/%	RMSE	Accuracy/%	RMSE
UPNN	43.5	1.602	59.6	0.748	60.8	0.746
InterSub	47.6	1.392	62.3	0.714	63.5	0.690
UPDMN	46.5	1.351	61.3	0.720	63.9	0.662
TUPCNN	48.8	1.451	63.9	0.694	63.9	0.688
NSC	53.3	1.281	65.0	0.692	66.7	0.654
PMA	54.0	1.301	65.8	0.668	67.5	0.641
DUPMN	53.9	1.279	66.2	0.667	67.6	0.639
CMA	54.0	1.191	66.4	0.677	67.6	0.637
HCSC	54.2	1.213	65.7	0.660	—	—
BLBC	—	—	67.1	0.662	—	—
HUPMA	54.5	1.253	67.0	0.659	67.6	0.641
MSC-GTU	54.7	1.191	66.3	0.673	68.1	0.643
MSC-GLU	54.9	1.200	66.2	0.669	68.2	0.640
MSC-GTUU	55.7	1.165	66.9	0.657	68.3	0.649

模型	IMDB		Yelp2013		Yelp2014
模型	Accuracy/%	RMSE	Accuracy/%	RMSE	Accuracy/%	RMSE
UPNN	43.5	1.602	59.6	0.748	60.8	0.746
InterSub	47.6	1.392	62.3	0.714	63.5	0.690
UPDMN	46.5	1.351	61.3	0.720	63.9	0.662
TUPCNN	48.8	1.451	63.9	0.694	63.9	0.688
NSC	53.3	1.281	65.0	0.692	66.7	0.654
PMA	54.0	1.301	65.8	0.668	67.5	0.641
DUPMN	53.9	1.279	66.2	0.667	67.6	0.639
CMA	54.0	1.191	66.4	0.677	67.6	0.637
HCSC	54.2	1.213	65.7	0.660	—	—
BLBC	—	—	67.1	0.662	—	—
HUPMA	54.5	1.253	67.0	0.659	67.6	0.641
MSC-GTU	54.7	1.191	66.3	0.673	68.1	0.643
MSC-GLU	54.9	1.200	66.2	0.669	68.2	0.640
MSC-GTUU	55.7	1.165	66.9	0.657	68.3	0.649

模型	IMDB		Yelp2013		Yelp2014
模型	Accuracy/%	RMSE	Accuracy/%	RMSE	Accuracy/%	RMSE
NoMSC-GTUU	54.1	1.233	65.8	0.683	67.2	0.667
MSC-NoGate	54.7	1.261	66.1	0.678	67.9	0.646
MSC-GTUU（no up）	49.4	1.381	63.9	0.697	64.5	0.680
MSC-GTUU	55.7	1.165	66.9	0.657	68.3	0.649