Aspect sentiment analysis with aspect item and context representation

doi:10.11772/j.issn.1001-9081.2022101482

Abstract

Abstract:

When predicting the emotional polarity of a specific aspect， there is a problem of only depending on a single aspect item and ignoring the emotional dependence between aspect items in the same sentence， a Multi-layer Multi-hop Memory network with Aspect Item and Context Representation （AICR-M³net） was proposed. Firstly， the position weighting information was fused by Bi-directional Gated Recurrent Unit （Bi-GRU）， and the hidden layer output was used as the input of the mixed context coding layer to obtain a context representation with higher semantic relevance to the context. Then， Multi-layer Multi-hop Memory Networks （M³net） was introduced to match aspect words and context many times and word by word to generate aspect word vectors of specific context. At the same time， the emotional dependence between specific aspect item and other aspect items in the sentence was modeled to guide the generation of context vector of specific aspect item. Experimental results on Restaurant， Laptop and Twitter datasets show that the proposed model has the classification accuracy improved by 1.34， 3.05 and 2.02 percentage points respectively， and the F1 score increased by 3.90， 3.78 and 2.94 percentage points respectively， compared with AOA-MultiACIA （Attention-Over-Attention Multi-layer Aspect-Context Interactive Attention）. The above verifies that the proposed model can deal with the mixed information with multiple aspects in context more effectively， and has certain advantages in dealing with the sentiment classification task in specific aspects.

Key words: aspect-specific sentiment analysis, emotional dependence, Memory Network (MN), Multi-Head Attention (MHA) mechanism, Gated Recurrent Unit (GRU)

摘要：

针对预测特定方面情感极性时存在只依赖单一方面项而忽略了同一句子中方面项之间间情感依赖关系的问题，提出一种具有方面项和上下文表示的多层多跳记忆网络（AICR-M³net）。首先，通过双向门控循环单元（Bi-GRU）融合位置加权信息，并将隐藏层输出作为混合上下文编码层的输入以获取与上下文语义关联度更高的上下文表示；其次，引入多层多跳记忆网络（M³net）多次逐词匹配方面词和上下文，从而生成特定上下文的方面词向量；同时，建模特定方面项与句子中其他方面项的情感依赖性，从而引导特定方面项的上下文向量的生成。在Restaurant、Laptop和Twitter数据集上的实验结果表明，与AOA-MultiACIA （Attention-Over-Attention Multi-layer Aspect-Context Interactive Attention）相比，所提模型的分类准确率分别提高了1.34、3.05和2.02个百分点，F1值分别提高了3.90、3.78和2.94个百分点。以上验证了所提模型能更有效地处理上下文中多方面的混合信息，且在处理特定方面情感分类任务中具有一定的优势。

关键词: 特定方面情感分析, 情感依赖, 记忆网络, 多头注意力机制, 门控循环单元

CLC Number:

TP391.1

Dan XU, Hongfang GONG, Rongrong LUO. Aspect sentiment analysis with aspect item and context representation[J]. Journal of Computer Applications, 2023, 43(10): 3086-3092.

徐丹, 龚红仿, 罗容容. 具有方面项和上下文表示的方面情感分析[J]. 《计算机应用》唯一官方网站, 2023, 43(10): 3086-3092.

Figures/Tables 7

References 23

1	TANG D， QIN B， FENG X， et al. Effective LSTMs for target-dependent sentiment classification［C］// Proceedings of the 26th International Conference on Computational Linguistics： Technical Papers. ［S.l.］： The COLING 2016 Organizing Committee， 2016： 3298-3307.
2	HUANG B， OU Y， CARLEY K M. Aspect level sentiment classification with attention-over-attention neural networks［C］// Proceedings of the 2018 International Conference on Social Computing， Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation， LNCS 10899. Cham： Springer， 2018： 197-206.
3	MA D， LI S， ZHANG X， et al. Interactive attention networks for aspect-level sentiment classification［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 4068-4074. 10.24963/ijcai.2017/568
4	WESTON J， CHOPRA S， BORDES A. Memory networks［EB/OL］. （2015-11-29）［2022-04-13］..
5	SUKHBAATAR S， SZLAM A， WESTON J， et al. End-to-end memory networks［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 2. Cambridge： MIT Press， 2015： 2440-2448.
6	TANG D， QIN B， LIU T. Aspect level sentiment classification with deep memory network［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2016： 214-224. 10.18653/v1/d16-1021
7	CHEN P， SUN Z， BING L， et al. Recurrent attention network on memory for aspect sentiment analysis［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2017： 452-461. 10.18653/v1/d17-1047
8	WANG S， MAZUMDER S， LIU B， et al. Target-sensitive memory networks for aspect sentiment classification［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2018： 957-967. 10.18653/v1/p18-1088
9	ZHANG C， LI Q， SONG D. Aspect-based sentiment classification with aspect-specific graph convolutional networks［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2019： 4568-4578. 10.18653/v1/d19-1464
10	XIAO Y， ZHOU G. Syntactic edge-enhanced graph convolutional networks for aspect-level sentiment classification with interactive attention［J］. IEEE Access， 2020， 8： 157068-157080. 10.1109/access.2020.3019277
11	ZHA Y， XIE Y， HUANG Q， et al. Aspect level sentiment classification with multi-scale information［C］// Proceedings of the 2nd International Conference on Electronics， Communications and Information Technology. Piscataway： IEEE， 2021： 279-285. 10.1109/cecit53797.2021.00056
12	HAN H， QIN X， ZHAO Q. Interactive attention graph convolution networks for aspect-level sentiment classification［C］// Proceedings of the 3rd International Conference on Artificial Intelligence and Advanced Manufacture. Piscataway： IEEE， 2021： 271-275. 10.1109/aiam54119.2021.00062
13	BAI X， LIU P， ZHANG Y. Investigating typed syntactic dependencies for targeted sentiment classification using graph attention neural network［J］. IEEE/ACM Transactions on Audio， Speech， and Language Processing， 2021， 29： 503-514. 10.1109/taslp.2020.3042009
14	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
15	SONG Y， WANG J， JIANG T， et al. Attentional encoder network for targeted sentiment classification［EB/OL］. （2019-04-01）［2022-07-05］.. 10.1007/978-3-030-30490-4_9
16	卢天兰，陈荔. 面向方面级别情感分析的端到端多跳记忆网络［J］. 计算机应用研究， 2021， 38（5）： 1409-1415， 1427.
	LU T L， CHEN L. End-to-end multi-hop memory network for aspect-level sentiment analysis［J］. Application Research of Computers， 2021， 38（5）： 1409-1415， 1427.
17	DONG L， WEI F， TAN C， et al. Adaptive recursive neural network for target-dependent Twitter sentiment classification［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg， PA： ACL， 2014： 49-54. 10.3115/v1/p14-2009
18	PONTIKI M， GALANIS D， PAVLOPOULOS J， et al. SemEval-2014 Task 4： aspect based sentiment analysis［C］// Proceedings of the 8th International Workshop on Semantic Evaluation. Stroudsburg， PA： ACL， 2014：27-35. 10.3115/v1/s14-2004
19	WANG Y， HUANG M， ZHU X， et al. Attention-based LSTM for aspect-level sentiment classification［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2016： 606-615. 10.18653/v1/d16-1058
20	SUN C， LV L， TIAN G， et al. Deep interactive memory network for aspect-level sentiment analysis［J］. ACM Transactions on Asian and Low-Resource Language Information Processing， 2021， 20（1）： No.3. 10.1145/3402886
21	LIN P， YANG M， LAI J. Deep mask memory network with semantic dependency and context moment for aspect level sentiment classification［C］// Proceedings of the 28th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2019： 5088-5094. 10.24963/ijcai.2019/707
22	WU Z， LI Y， LIAO J， et al. Aspect-context interactive attention representation for aspect-level sentiment classification［J］. IEEE Access， 2020， 8： 29238-29248. 10.1109/access.2020.2972697
23	JIA Z， BAI X， PANG S. Hierarchical gated deep memory network with position-aware for aspect-based sentiment analysis［J］. IEEE Access， 2020， 8： 136340-136347. 10.1109/access.2020.3011318

数据集		积极	消极	中性
Restaurant	训练集	994	870	464
Restaurant	测试集	341	128	169
Laptop	训练集	2 164	807	637
Laptop	测试集	728	196	196
Twitter	训练集	1 567	1 563	3 127
Twitter	测试集	174	174	346

数据集		积极	消极	中性
Restaurant	训练集	994	870	464
Restaurant	测试集	341	128	169
Laptop	训练集	2 164	807	637
Laptop	测试集	728	196	196
Twitter	训练集	1 567	1 563	3 127
Twitter	测试集	174	174	346

模型	Restaurant		Laptop		Twitter
模型	ACC	F1	ACC	F1	ACC	F1
LSTM	74.28	64.71	66.45	62.79	―	―
TD-LSTM	75.50	62.30	67.70	61.30	65.20	64.90
ATAE-LSTM	77.20	63.50	68.70	63.30	69.40	67.70
DIMN	80.60	―	74.70	―	―	―
AEN	80.98	72.14	73.15	69.04	72.83	69.81
IAN	79.26	70.09	72.05	67.38	72.50	70.81
DMMN-SDCM	81.16	71.50	77.59	73.61	―	―
AOA-MultiACIA	82.59	72.13	75.27	70.24	72.40	69.40
HGDM-PA	81.95	74.07	74.82	72.53	―	―
AICR-M³net	83.93	76.03	78.32	74.02	74.42	72.34

模型	Restaurant		Laptop		Twitter
模型	ACC	F1	ACC	F1	ACC	F1
LSTM	74.28	64.71	66.45	62.79	―	―
TD-LSTM	75.50	62.30	67.70	61.30	65.20	64.90
ATAE-LSTM	77.20	63.50	68.70	63.30	69.40	67.70
DIMN	80.60	―	74.70	―	―	―
AEN	80.98	72.14	73.15	69.04	72.83	69.81
IAN	79.26	70.09	72.05	67.38	72.50	70.81
DMMN-SDCM	81.16	71.50	77.59	73.61	―	―
AOA-MultiACIA	82.59	72.13	75.27	70.24	72.40	69.40
HGDM-PA	81.95	74.07	74.82	72.53	―	―
AICR-M³net	83.93	76.03	78.32	74.02	74.42	72.34

跳级层数	Restaurant		Laptop		Twitter
跳级层数	准确率	F1	准确率	F1	准确率	F1
1	81.34	70.53	65.02	59.43	71.82	70.45
2	80.71	67.64	75.71	70.53	73.12	71.05
3	81.43	70.65	78.32	74.02	72.25	70.48
4	81.87	71.68	76.80	71.19	71.24	69.75
5	83.93	76.30	76.65	72.09	72.54	70.42
6	83.30	74.43	76.96	72.29	74.42	72.34
7	82.05	71.91	73.82	67.87	73.27	70.75
8	81.43	70.72	71.32	66.80	72.69	70.57