Aspect-level sentiment analysis model based on alternating‑attention mechanism and graph convolutional network

doi:10.11772/j.issn.1001-9081.2023040497

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (4): 1058-1064.DOI: 10.11772/j.issn.1001-9081.2023040497

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Aspect-level sentiment analysis model based on alternating‑attention mechanism and graph convolutional network

Xianfeng YANG¹(), Yilei TANG¹, Ziqiang LI²

^1.School of Computer Science and Software Engineering，Southwest Petroleum University，Chengdu Sichuan 610500，China
^2.College of Movie and Media，Sichuan Normal University，Chengdu Sichuan 610066，China

Received:2023-04-28 Revised:2023-06-13 Accepted:2023-06-30 Online:2024-04-22 Published:2024-04-10
Contact: Xianfeng YANG
About author:YANG Xianfeng， born in 1974， M. S.， professor. Her research interests include computer image processing， wisdom education.
TANG Yilei， born in 2000， M. S. candidate. His research interests include natural language processing.
LI Ziqiang， born in 1970，Ph. D.， professor. His research interests include machine learning， wisdom education， natural language processing.
Supported by:
National Natural Science Foundation of China(61802321);Key Research and Development Program of Science and Technology Department of Sichuan Province(2020YFN0019)

基于交替注意力机制和图卷积网络的方面级情感分析模型

杨先凤¹(), 汤依磊¹, 李自强²

^1.西南石油大学计算机与软件学院，成都 610500
^2.四川师范大学影视与传媒学院，成都 610066

通讯作者: 杨先凤
作者简介:杨先凤（1974—），女，四川南部人，教授，硕士，主要研究方向：计算机图像处理、智慧教育 565695835@qq.com
汤依磊（2000—），男，重庆人，硕士研究生，主要研究方向：自然语言处理
李自强（1970—），男，四川青神人，教授，博士，CCF会员，主要研究方向：机器学习、智慧教育、自然语言处理。
基金资助:
国家自然科学基金资助项目(61802321);四川省科技厅重点研发计划项目(2020YFN0019)

Abstract

Abstract:

Aspect-level sentiment analysis aims to predict the sentiment polarity of specific target in given text. Aiming at the problem of ignoring the syntactic relationship between aspect words and context and reducing the attention difference caused by average pooling， an aspect-level sentiment analysis model based on Alternating-Attention （AA） mechanism and Graph Convolutional Network （AA-GCN） was proposed. Firstly， the Bidirectional Long Short-Term Memory （Bi-LSTM） network was used to semantically model context and aspect words. Secondly， the GCN based on syntactic dependency tree was used to learn location information and dependencies， and the AA mechanism was used for multi-level interactive learning to adaptively adjust the attention to the target word. Finally， the final classification basis was obtained by splicing the corrected aspect features and context features. Compared with the Target-Dependent Graph Attention Network （TD-GAT）， the accuracies of the proposed model on four public datasets increased by 1.13%-2.67%， and the F1 values on five public datasets increased by 0.98%-4.89%， indicating the effectiveness of using syntactic relationships and increasing keyword attention.

Key words: Natural Language Processing (NLP), deep learning, aspect-level sentiment analysis, Alternating?Attention (AA) mechanism, Graph Convolutional Network (GCN)

摘要：

方面级情感分析旨在预测给定文本中特定目标的情感极性。针对忽略方面词和上下文之间的句法关系和平均池化带来的注意力差异性变小的问题，提出一种基于交替注意力（AA）机制和图卷积网络（GCN）的方面级情感分析模型（AA-GCN）。首先，利用双向长短期记忆（Bi-LSTM）网络对上下文和方面词进行语义建模；其次，通过基于句法依存树的GCN学习位置信息和依赖关系，再利用AA机制进行多层次交互学习，自适应地调整对目标词的关注度；最后，拼接修正后的方面特征和上下文特征，得到最终的分类依据。相较于基于目标依赖的图注意力网络（TD-GAT），所提模型在4个公开数据集上准确率提升了1.13%~2.67%，在5个公开数据集上F1值提升了0.98%~4.89%，验证了利用句法关系和提升关键词关注度的有效性。

关键词: 自然语言处理, 深度学习, 方面级情感分析, 交替注意力机制, 图卷积网络

CLC Number:

TP391.1

Xianfeng YANG, Yilei TANG, Ziqiang LI. Aspect-level sentiment analysis model based on alternating‑attention mechanism and graph convolutional network[J]. Journal of Computer Applications, 2024, 44(4): 1058-1064.

杨先凤, 汤依磊, 李自强. 基于交替注意力机制和图卷积网络的方面级情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1058-1064.

Figures/Tables 8

References 37

1	ZHANG W， LI X， DENG Y， et al. A survey on aspect-based sentiment analysis： tasks， methods， and challenges ［J］. IEEE Transactions on Knowledge and Data Engineering， 2023， 35（11）： 11019-11038. 10.1109/tkde.2022.3230975
2	WAGNER J， ARORA P， CORTES S， et al. DCU： aspect-based polarity classification for SemEval Task 4［C］// Proceedings of the 8th International Workshop on Semantic Evaluation. Stroudsburg： ACL， 2014： 223-229. 10.3115/v1/s14-2036
3	刘全，梁斌，徐进，等. 一种用于基于方面情感分析的深度分层网络模型［J］. 计算机学报， 2018， 41（12）： 2637-2652. 10.11897/SP.J.1016.2018.02637
	LIU Q， LIANG B， XU J， et al. A deep hierarchical neural network model for aspect-based sentiment analysis［J］. Chinese Journal of Computers， 2018， 41（12）： 2637-2652. 10.11897/SP.J.1016.2018.02637
4	DONG L， WEI F， TAN C， et al. Adaptive recursive neural network for target-dependent twitter sentiment classification［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2014： 49-54. 10.3115/v1/p14-2009
5	XUE W， LI T. Aspect based sentiment analysis with gated convolutional networks［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2018： 2514-2523. 10.18653/v1/p18-1234
6	AYETIRAN E F. Attention-based aspect sentiment classification using enhanced learning through CNN-BiLSTM networks［J］. Knowledge-Based Systems， 2022， 252： 109409. 10.1016/j.knosys.2022.109409
7	TANG D， QIN B， FENG X， et al. Effective LSTMs for target-dependent sentiment classification［C］// Proceedings of the 26th International Conference on Computational Linguistics： Technical Papers. ［S.l.］： The COLING 2016 Organizing Committee， 2016：3298-3307.
8	WANG Y， HUANG M， ZHU X， et al. Attention-based LSTM for aspect-level sentiment classification［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2016： 606-615. 10.18653/v1/d16-1058
9	MA D， LI S， ZHANG X， et al. Interactive attention networks for aspect-level sentiment classification［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2017： 4068-4074. 10.24963/ijcai.2017/568
10	HUANG B， OU Y， CARLEY K M. Aspect level sentiment classification with attention-over-attention neural networks［C］// Proceedings of the 2018 International Conference on Social Computing， Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Cham： Springer， 2018： 197-206. 10.1007/978-3-319-93372-6_22
11	ZENG B， YANG H， XU R， et al. LCF： a local context focus mechanism for aspect-based sentiment classification［J］. Applied Sciences， 2019， 9（16）：3389. 10.3390/app9163389
12	DEVLIN J， CHANG M-W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding ［EB/OL］. （2018-10-11）［2022-10-21］. . 10.18653/v1/n18-2
13	KARIMI A， ROSSI L， PRATI A. Adversarial training for aspect-based sentiment analysis with BERT［C］// Proceedings of the 2020 25th International Conference on Pattern Recognition. Piscataway： IEEE， 2021：8797-8803. 10.1109/icpr48806.2021.9412167
14	ANSAR W， GOSWAMI S， CHAKRABARTI A， et al. An efficient methodology for aspect-based sentiment analysis using BERT through refined aspect extraction［J］. Journal of Intelligent & Fuzzy Systems， 2021， 40（5）： 9627-9644. 10.3233/jifs-202140
15	SU J， TANG J， JIANG H， et al. Enhanced aspect-based sentiment analysis models with progressive self-supervised attention learning［J］. Artificial Intelligence， 2021， 296： 103477. 10.1016/j.artint.2021.103477
16	ZENG J， LIU T， JIA W， et al. Relation construction for aspect-level sentiment classification［J］. Information Sciences， 2022， 586： 209-223. 10.1016/j.ins.2021.11.081
17	PHAN H T， NGUYEN N T， HWANG D. Convolutional attention neural network over graph structures for improving the performance of aspect-level sentiment analysis［J］. Information Sciences， 2022， 589： 416-439. 10.1016/j.ins.2021.12.127
18	ZHAO M， YANG J， ZHANG J， et al. Aggregated graph convolutional networks for aspect-based sentiment classification［J］. Information Sciences， 2022， 600： 73-93. 10.1016/j.ins.2022.03.082
19	FENG S， WANG B， YANG Z， et al. Aspect-based sentiment analysis with attention-assisted graph and variational sentence representation［J］. Knowledge-Based Systems， 2022， 258： 109975. 10.1016/j.knosys.2022.109975
20	LI R， CHEN H， FENG F， et al. Dual graph convolutional networks for aspect-based sentiment analysis［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 6319-6329. 10.18653/v1/2021.acl-long.494
21	LIANG Y， MENG F， ZHANG J， et al. A dependency syntactic knowledge augmented interactive architecture for end-to-end aspect-based sentiment analysis ［EB/OL］. （2020-04-04）［2023-03-10］. . 10.1016/j.neucom.2021.05.028
22	TIAN Y， CHEN G， SONG Y. Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble［C］// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2021： 2910-2922. 10.18653/v1/2021.naacl-main.231
23	衡红军杨鼎诚. 知识增强的方面词交互图神经网络［J］.计算机应用， 2023， 43（8）： 2412-2419.
	HENG H J， YANG D C. Knowledge enhanced aspect word interactive graph neural network［J］. Journal of Computer Applications， 2023， 43（8）： 2412-2419.
24	ZHANG C， LI Q， SONG D. Aspect-based sentiment classification with aspect-specific graph convolutional networks［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2019： 4568-4578. 10.18653/v1/d19-1464
25	KIPF T N， WELLING M. Semi‑supervised classification with graph convolutional networks ［EB/OL］. （2017-02-22）［2023-03-21］. .
26	LI X， BING L D， LAM W， et al. Transformation networks for target-oriented sentiment classification［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2018： 946-956. 10.18653/v1/p18-1087
27	XIONG C， ZHONG V， SOCHER R. Dynamic coattention networks for question answering ［EB/OL］. （2016-05-19）［2023-03-21］. .
28	LU J， YANG J， BATRA D， et al. Hierarchical question-image co-attention for visual question answering ［EB/OL］. （2017-01-19）［2023-03-21］. .
29	PONTIKI M， GALANIS D， PAVLOPOULOS J， et al. SemEval-2014 Task 4： aspect based sentiment analysis［C］// Proceedings of the 8th International Workshop on Semantic Evaluation. Stroudsburg： ACL， 2014： 27-35. 10.3115/v1/s14-2004
30	PONTIKI M， GALANIS D， PAPAGEORGIOU H， et al. SemEval-2015 Task 12： aspect based sentiment analysis［C］// Proceedings of the 9th International Workshop on Semantic Evaluation. Stroudsburg： ACL， 2015： 486-495. 10.18653/v1/s15-2082
31	PONTIKI M， GALANIS D， PAPAGEORGIOU H， et al. SemEval-2016 Task 5： aspect based sentiment analysis［C］// Proceedings of the 10th International Workshop on Semantic Evaluation. Stroudsburg： ACL， 2016： 19-30. 10.18653/v1/s15-2082
32	TANG D， QIN B， LIU T. Aspect level sentiment classification with deep memory network［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2016： 214-224. 10.18653/v1/d16-1021
33	YADAV R K， JIAO L， GOODWIN M， et al. Positionless aspect based sentiment analysis using attention mechanism［J］. Knowledge-Based Systems， 2021， 226： 107136. 10.1016/j.knosys.2021.107136
34	ZHU X， ZHU L， GUO J， et al. GL-GCN： global and local dependency guided graph convolutional networks for aspect-based sentiment classification［J］. Expert Systems with Applications， 2021， 186：115712. 10.1016/j.eswa.2021.115712
35	HUANG B， CARLEY K. Syntax-aware aspect level sentiment classification with graph attention networks［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2019： 5469-5477. 10.18653/v1/d19-1549
36	ZHOU J， HUANG J X， HU Q V， et al. SK-GCN： modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification［J］. Knowledge-Based Systems， 2020， 205： 106292. 10.1016/j.knosys.2020.106292
37	YAO L， MAO C， LUO Y. Graph convolutional networks for text classification ［EB/OL］. （2018-09-15）［2023-06-01］. . 10.1609/aaai.v33i01.33017370

实验环境	具体信息
操作系统	Windows 10
CPU	Intel Xeon CPU E5-2678 v3 @2.50 GHz
显卡	NVIDA Tesla K80
内存	8 GB
开发语言	Python 3.7
开发平台	PyTorch 1.7.1
开发工具	PyCharm 2021.1.1

实验环境	具体信息
操作系统	Windows 10
CPU	Intel Xeon CPU E5-2678 v3 @2.50 GHz
显卡	NVIDA Tesla K80
内存	8 GB
开发语言	Python 3.7
开发平台	PyTorch 1.7.1
开发工具	PyCharm 2021.1.1

数据集		#Pos	#Neu	#Neg
Twitter^［4］	训练集	1 561	3 127	1 560
Twitter^［4］	测试集	173	346	173
LAP14^［29］	训练集	994	464	870
LAP14^［29］	测试集	341	169	128
REST14^［29］	训练集	2 164	637	807
REST14^［29］	测试集	728	196	196
REST15^［30］	训练集	1 178	50	382
REST15^［30］	测试集	439	35	328
REST16^［31］	训练集	1 620	88	709
REST16^［31］	测试集	597	38	190

数据集		#Pos	#Neu	#Neg
Twitter^［4］	训练集	1 561	3 127	1 560
Twitter^［4］	测试集	173	346	173
LAP14^［29］	训练集	994	464	870
LAP14^［29］	测试集	341	169	128
REST14^［29］	训练集	2 164	637	807
REST14^［29］	测试集	728	196	196
REST15^［30］	训练集	1 178	50	382
REST15^［30］	测试集	439	35	328
REST16^［31］	训练集	1 620	88	709
REST16^［31］	测试集	597	38	190

参数	值	参数	值
Learning Rate	0.001	Dropout	0.3
Batch Size	32	initializer	xavier_uniform_
L2正则化	10^-5

Aspect-level sentiment analysis model based on alternating‑attention mechanism and graph convolutional network

基于交替注意力机制和图卷积网络的方面级情感分析模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 37

Related Articles 15

Recommended Articles

Metrics

模型	Twitter		LAP14		REST14		REST15		REST16
模型	准确率	F1	准确率	F1	准确率	F1	准确率	F1	准确率	F1
LSTM	69.32	67.84	70.53	64.19	77.77	66.89	76.38	55.47	87.34	65.88
MemNet	71.77	70.24	71.79	67.54	79.58	70.67	78.04	60.47	85.88	67.84
AOA	71.92	69.56	72.93	67.82	80.36	70.62	78.60	58.89	87.82	70.76
IAN	70.37	68.24	73.56	68.34	79.49	70.32	78.66	59.32	87.01	65.59
PAM^*	—	—	75.39	70.50	81.37	72.06	80.88	62.48	89.30	66.93
RMN^*	—	—	74.50	69.79	81.16	73.17	80.69	64.41	88.75	71.54
GL-GCN^*	73.26	71.26	76.91	72.76	82.11	73.46	80.81	64.99	88.47	69.64
AGCN^*	73.98	72.48	75.07	70.96	80.02	71.02	80.07	62.70	87.98	65.78
TD-GAT^*	72.20	70.45	75.24	70.74	81.32	71.12	80.38	60.50	87.71	67.87
SK-GCN^*	71.97	70.22	73.20	69.18	80.36	70.43	80.12	60.70	85.17	68.08
AA-GCN	74.13	72.44	76.09	71.43	82.35	74.36	79.77	63.46	89.32	70.90

例句	方面词	预测值		真实值
例句	方面词	AOA	AA-GCN	真实值
The staff should be a bit more friendly.	staff	positive	negative	negative
Great food but the service was dreadful！	food	positive	positive	positive
Did not enjoy the new Windows 8 and touchscreen functions.	Windows 8	positive	negative	negative
After numerous attempts of trying -LRB- including setting the clock in BIOS setup directly -RRB-， I gave up -LRB- I am a techie -RRB-	clock in BIOS setup	neutral	negative	negative

[1]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[2]	Qi SHUAI, Hairui WANG, Guifu ZHU. Chinese story ending generation model based on bidirectional contrastive training [J]. Journal of Computer Applications, 2024, 44(9): 2683-2688.
[3]	Chuanlin PANG, Rui TANG, Ruizhi ZHANG, Chuan LIU, Jia LIU, Shibo YUE. Distributed power allocation algorithm based on graph convolutional network for D2D communication systems [J]. Journal of Computer Applications, 2024, 44(9): 2855-2862.
[4]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[5]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[6]	Guixiang XUE, Hui WANG, Weifeng ZHOU, Yu LIU, Yan LI. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network [J]. Journal of Computer Applications, 2024, 44(9): 2952-2957.
[7]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[8]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[9]	Quanmei ZHANG, Runping HUANG, Fei TENG, Haibo ZHANG, Nan ZHOU. Automatic international classification of disease coding method incorporating heterogeneous information [J]. Journal of Computer Applications, 2024, 44(8): 2476-2482.
[10]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[11]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[12]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[13]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[14]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[15]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.

模型	Twitter		LAP14		REST14		REST15		REST16
模型	准确率	F1	准确率	F1	准确率	F1	准确率	F1	准确率	F1
AA-GCN	74.13	72.44	75.55	71.43	82.35	74.36	79.77	63.46	89.12	70.90
mine w/o ac	73.04	72.86	74.97	70.79	81.63	73.74	79.09	62.33	88.15	69.03
mine w/o aa	71.19	69.04	75.18	71.17	81.31	72.86	78.04	62.71	87.77	67.74
mine w/o alter	73.74	72.03	75.44	70.70	82.35	74.50	79.95	64.13	88.53	69.72
mine w/0 pw	74.08	72.51	75.13	70.79	81.25	73.39	78.95	61.93	88.26	70.36
mine w/0 gcn	72.54	70.97	72.94	68.94	79.46	69.18	79.27	62.08	86.85	65.14
mine w/0 mas	72.83	71.54	73.77	69.61	80.30	70.49	79.95	62.43	88.31	66.24

模型	Twitter		LAP14		REST14		REST15		REST16
模型	准确率	F1	准确率	F1	准确率	F1	准确率	F1	准确率	F1
AA-GCN	74.13	72.44	75.55	71.43	82.35	74.36	79.77	63.46	89.12	70.90
mine w/o ac	73.04	72.86	74.97	70.79	81.63	73.74	79.09	62.33	88.15	69.03
mine w/o aa	71.19	69.04	75.18	71.17	81.31	72.86	78.04	62.71	87.77	67.74
mine w/o alter	73.74	72.03	75.44	70.70	82.35	74.50	79.95	64.13	88.53	69.72
mine w/0 pw	74.08	72.51	75.13	70.79	81.25	73.39	78.95	61.93	88.26	70.36
mine w/0 gcn	72.54	70.97	72.94	68.94	79.46	69.18	79.27	62.08	86.85	65.14
mine w/0 mas	72.83	71.54	73.77	69.61	80.30	70.49	79.95	62.43	88.31	66.24