Aspect-based sentiment analysis model fused with multi-window local information

doi:10.11772/j.issn.1001-9081.2022060891

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (6): 1796-1802.DOI: 10.11772/j.issn.1001-9081.2022060891

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Aspect-based sentiment analysis model fused with multi-window local information

Zhixiong ZHENG¹^,², Jianhua LIU¹^,²(), Shuihua SUN¹^,², Ge XU³, Honghui LIN¹^,²

^1.College of Computer Science and Mathematics，Fujian University of Technology，Fuzhou Fujian 350118，China
^2.Fujian Provincial Key Laboratory of Big Data Mining and Applications （Fujian University of Technology），Fuzhou Fujian 350118，China
^3.College of Computer and Control Engineering，Minjiang University，Fuzhou Fujian 350108，China

Received:2022-06-20 Revised:2022-09-23 Accepted:2022-10-11 Online:2022-11-07 Published:2023-06-10
Contact: Jianhua LIU
About author:ZHENG Zhixiong， born in 1996， M. S. candidate. His research interests include aspect-based sentiment analysis.
SUN Shuihua， born in 1962， Ph. D.， professor. Her research interests include natural language processing， machine translation.
XU Ge， born in 1978， Ph. D.， professor. His research interests include intelligent quiz bot， sentiment analysis.
LIN Honghui， born in 1996， M. S. candidate. His research interests include natural language processing.
Supported by:
National Natural Science Foundation of China(62172095);Fuzhou Science and Technology Innovation Platform Program(2021-P-052);Fujian University of Technology Development Foundation(GY-Z20046);Central Leading Local Project of China(2020L3024)

融合多窗口局部信息的方面级情感分析模型

郑智雄¹^,², 刘建华¹^,²(), 孙水华¹^,², 徐戈³, 林鸿辉¹^,²

^1.福建工程学院计算机科学与数学学院，福州 350118
^2.福建省大数据挖掘与应用技术重点实验室（福建工程学院），福州 350118
^3.闽江学院计算机与控制工程学院，福州 350108

通讯作者: 刘建华
作者简介:郑智雄（1996—），男，福建莆田人，硕士研究生，CCF会员，主要研究方向：方面级情感分析
刘建华（1967—），男，江西吉安人，教授，博士，CCF会员，主要研究方向：智能计算、机器学习Email：jhliu@fjnu.edu.cn
孙水华（1962—），女，福建宁德人，教授，博士，主要研究方向：自然语言处理、机器翻译
徐戈（1978—），男，浙江淳安人，教授，博士，CCF会员，主要研究方向：智能问答机器人、情感分析
林鸿辉（1996—），男，福建福州人，硕士研究生，CCF会员，主要研究方向：自然语言处理。
基金资助:
国家自然科学基金资助项目(62172095);福州市科技创新平台项目(2021?P?052);福建工程学院发展基金资助项目(GY?Z20046);中央引导地方项目(2020L3024)

Abstract

Abstract:

Focused on the issue that the current Aspect-Based Sentiment Analysis （ABSA） models rely too much on the syntactic dependency tree with relatively sparse relationships to learn feature representations， which leads to the insufficient ability of the model to learn local information， an ABSA model fused with multi-window local information called MWGAT （combining Multi-Window local information and Graph ATtention network） was proposed. Firstly， the local contextual features were learned through the multi-window local feature learning mechanism， and the potential local information contained in the text was mined. Secondly， Graph ATtention network （GAT）， which can better understand the syntactic dependency tree， was used to learn the syntactic structure information represented by the syntactic dependency tree， and syntax-aware contextual features were generated. Finally， these two types of features representing different semantic information were fused to form the feature representation containing both the syntactic information of syntactic dependency tree and the local information， so that the sentiment polarities of aspect words were discriminated by the classifier efficiently. Three public datasets， Restaurant， Laptop， and Twitter were used for experiment. The results show that compared with the T-GCN （Type-aware Graph Convolutional Network） model combined with the syntactic dependency tree， the proposed model has the Macro-F1 score improved by 2.48%， 2.37% and 0.32% respectively. It can be seen that the proposed model can mine potential local information effectively and predict the sentiment polarities of aspect words more accurately.

Key words: graph neural network, attention mechanism, Aspect-Based Sentiment Analysis (ABSA), local feature learning, Graph ATtention network (GAT), gated mechanism

摘要：

针对目前方面级情感分析（ABSA）模型过多依赖关系较为稀疏的句法依赖树学习特征表示，导致模型学习局部信息能力不足的问题，提出了一种融合多窗口局部信息的ABSA模型MWGAT（combining Multi-Window local information and Graph ATtention network）。首先，通过多窗口局部特征学习机制学习局部上下文特征，并挖掘文本包含的潜在局部信息；其次，采用能够较好理解依赖树的图注意力网络（GAT）学习句法依赖树所表示的语法结构信息，并生成语法感知的上下文特征；最后，将这两种表示不同语义信息的特征融合，形成既包含句法依赖树的语法信息又包含局部信息的特征表示，从而便于分类器高效判别方面词的情感极性。在Restaurant、Laptop和Twitter这3个公开数据集上进行实验，结果表明与结合了句法依赖树的T-GCN（Type-aware Graph Convolutional Network）模型相比，所提模型的Macro-F1分数分别提高了2.48%、2.37%和0.32%。可见，所提模型能够有效挖掘潜在的局部信息，并更为精确地预测方面词的情感极性。

关键词: 图神经网络, 注意力机制, 方面级情感分析, 局部特征学习, 图注意力网络, 门控机制

CLC Number:

TP391.1

Zhixiong ZHENG, Jianhua LIU, Shuihua SUN, Ge XU, Honghui LIN. Aspect-based sentiment analysis model fused with multi-window local information[J]. Journal of Computer Applications, 2023, 43(6): 1796-1802.

郑智雄, 刘建华, 孙水华, 徐戈, 林鸿辉. 融合多窗口局部信息的方面级情感分析模型[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1796-1802.

Figures/Tables 11

References 27

1	张严，李天瑞.面向评论的方面级情感分析综述［J］.计算机科学，2020，47（6）：194-200. 10.11896/jsjkx.200200127
	ZHANG Y， LI T R. Review of comment-oriented aspect-level sentiment analysis［J］. Computer Science， 2020， 47（6）：194-200. 10.11896/jsjkx.200200127
2	史伟，付月. 考虑语境的微博短文本挖掘：情感分析的方法［J］. 计算机科学， 2021， 48（6A）：158-164. 10.11896/jsjkx.210200089
	SHI W， FU Y. Microblog short text mining considering context： a method of sentiment analysis ［J］. Computer Science， 2021， 48（6A）：158-164. 10.11896/jsjkx.210200089
3	AKOURY N， KRISHNA K， IYYER M. Syntactically supervised transformers for faster neural machine translation ［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2019： 1269-1281. 10.18653/v1/p19-1122
4	PHAN M H， OGUNBONA P O. Modelling context and syntactical features for aspect-based sentiment analysis ［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2020： 3211-3220. 10.18653/v1/2020.acl-main.293
5	WANG Y Q， HUANG M L， ZHU X Y， et al. Attention-based LSTM for aspect-level sentiment classification ［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2016： 606-615. 10.18653/v1/d16-1058
6	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 6000-6010.
7	MA D H， LI S J， ZHANG X D， et al. Interactive attention networks for aspect-level sentiment classification［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 4068-4074. 10.24963/ijcai.2017/568
8	LI X， BING L D， LAM W， et al. Transformation networks for target-oriented sentiment classification ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2018： 946-956. 10.18653/v1/p18-1087
9	ZENG B Q， YANG H， XU R Y， et al. LCF： a local context focus mechanism for aspect-based sentiment classification［J］. Applied Sciences， 2019， 9（16）： No.3389. 10.3390/app9163389
10	HUANG B X， CARLEY K M. Syntax-aware aspect level sentiment classification with graph attention networks［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2019： 5469-5477. 10.18653/v1/d19-1549
11	YAN H， DAI J Q， JI T， et al. A unified generative framework for aspect-based sentiment analysis［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2021： 2416-2429. 10.18653/v1/2021.acl-long.188
12	MA L H， RABBANY R， ROMERO-SORIANO A. Graph attention networks with positional embeddings ［C］// Proceedings of the 2021 Pacific-Asia Conference on Knowledge Discovery and Data Mining， LNCS 12712. Cham： Springer， 2021： 514-527.
13	BAI X F， LIU P B， ZHANG Y. Investigating typed syntactic dependencies for targeted sentiment classification using graph attention neural network［J］. IEEE/ACM Transactions on Audio， Speech， and Language Processing， 2021， 29： 503-514. 10.1109/taslp.2020.3042009
14	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780. 10.1162/neco.1997.9.8.1735
15	CHO K， van MERRIËNBOER B， GU̇LÇEHRE Ç， et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2014： 1724-1734. 10.3115/v1/d14-1179
16	JOZEFOWICZ R， ZAREMBA W， SUTSKEVER I. An empirical exploration of recurrent network architectures ［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 2342-2350.
17	DOZAT T， MANNING C D. Deep biaffine attention for neural dependency parsing［EB/OL］. （2017-03-10）［2022-06-19］.. 10.18653/v1/k17-3002
18	FAN F F， FENG Y S， ZHAO D Y. Multi-grained attention network for aspect-level sentiment classification ［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2018： 3433-3442. 10.18653/v1/d18-1380
19	SONG Y W， WANG J H， JIANG T， et al. Attentional encoder network for targeted sentiment classification ［EB/OL］. （2019-04-01）［2022-06-19］.. 10.1007/978-3-030-30490-4_9
20	JIANG Q N， CHEN L， XU R F， et al. A challenge dataset and effective models for aspect-based sentiment analysis［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2019： 6280-6285. 10.18653/v1/d19-1654
21	SABOUR S， FROSST N， HINTON G E. Dynamic routing between capsules ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 3859-3869.
22	NGUYEN T H， SHIRAI K. PhraseRNN： phrase recursive neural network for aspect-based sentiment analysis［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2015： 2509-2514. 10.18653/v1/d15-1298
23	DONG L， WEI F R， TAN C Q， et al. Adaptive recursive neural network for target-dependent Twitter sentiment classification ［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg， PA： ACL， 2014： 49-54. 10.3115/v1/p14-2009
24	NGUYEN H T， LE NGUYEN M. Effective attention networks for aspect-level sentiment classification ［C］// Proceedings of the 10th International Conference on Knowledge and Systems Engineering. Piscataway： IEEE， 2018： 25-30. 10.1109/kse.2018.8573324
25	SUN K， ZHANG R C， MENSAH S， et al. Aspect-level sentiment analysis via convolution over dependency tree ［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2019： 5679-5688. 10.18653/v1/d19-1569
26	WANG K， SHEN W Z， YANG Y Y， et al. Relational graph attention network for aspect-based sentiment analysis［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2020： 3229-3238. 10.18653/v1/2020.acl-main.295
27	TIAN Y H， CHEN G M， SONG Y. Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble［C］// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： ACL， 2021： 2910-2922. 10.18653/v1/2021.naacl-main.231

数据集	积极		消极		中性
数据集	训练	测试	训练	测试	训练	测试
Restaurant	2 164	727	807	196	637	196
Laptop	976	337	851	128	455	167
Twitter	1 507	172	1 528	169	3 016	336

数据集	积极		消极		中性
数据集	训练	测试	训练	测试	训练	测试
Restaurant	2 164	727	807	196	637	196
Laptop	976	337	851	128	455	167
Twitter	1 507	172	1 528	169	3 016	336

参数	值	说明
BERT Model	bert-base-uncased	BERT模型版本
Embedding Dimension	768	词嵌入维度
BERT Dropout	0.1	BERT随机失活比例
MWFLL Dropout	0.1	MWFLL随机失活比例
Number of GAT Head	4	GAT的注意力头数
Learning Rate	2×10^-5	学习率
L2 Regularization Term	10^-5	L2正则化超参数

参数	值	说明
BERT Model	bert-base-uncased	BERT模型版本
Embedding Dimension	768	词嵌入维度
BERT Dropout	0.1	BERT随机失活比例
MWFLL Dropout	0.1	MWFLL随机失活比例
Number of GAT Head	4	GAT的注意力头数
Learning Rate	2×10^-5	学习率
L2 Regularization Term	10^-5	L2正则化超参数

参数	值	说明
Operating System	Windows 10	操作系统
GPU	Nvidia RTX 3070	图像处理器
GPU Memory	8.0GB	图像处理器内存
Development Tool	PyCharm 2020.3.1	开发工具
Deep Learning Framework	PyTorch 1.10.0	深度学习框架

Aspect-based sentiment analysis model fused with multi-window local information

融合多窗口局部信息的方面级情感分析模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 27

Related Articles 15

Recommended Articles

Metrics

类别	模型	Restaurant		Laptop		Twitter
类别	模型	Acc	MF1	Acc	MF1	Acc	MF1
w/o Syn	ATAE-LSTM	77.20	—	68.70	—	—	—
	IAN	78.60	—	72.10	—	—	—
	MGAN	81.25	71.94	75.39	72.47	72.54	70.81
	AEN	80.98	72.14	73.51	69.04	72.83	69.81
	AEN-BERT	83.12	73.76	79.93	76.31	74.71	73.13
	CapsNet	80.79	—	—	—	79.78	—
W Syn	PhraseRNN	66.20	59.32	—	—	—	—
	LSTM+SynAtt	80.45	71.26	72.57	69.13	—	—
	TD-GAT	81.20	—	74.00	—	—	—
	CDT	82.30	74.02	77.19	72.99	74.66	73.66
	R-GAT+BERT	85.44	79.17	78.01	75.57	75.93	74.60
	T-GCN	86.12	79.95	80.32	76.82	76.45	75.25
	MWGAT	87.21	81.93	81.56	78.64	76.74	75.49

窗口尺寸	不同数据集上的预测准确率
窗口尺寸	Restaurant	Laptop
3	86.51	80.70
5	87.12	81.25
7	86.15	79.06
9	85.08	79.22
11	85.88	78.75

窗口数	窗口尺寸组合	不同数据集上的预测准确率
窗口数	窗口尺寸组合	Restaurant	Laptop
1	（5）	87.12	81.25
2	（3，5）	87.21	81.56
3	（3，5，7）	86.86	80.78
4	（3，5，7，9）	86.60	79.69
5	（3，5，7，9，11）	84.71	77.81

模型	Restaurant		Laptop
模型	Acc	MF1	Acc	MF1
w/o MWFLL	86.32	79.58	80.16	76.93
w/o GAT	85.61	78.78	78.91	74.92
MWGAT	87.21	81.93	81.56	78.64

[1]	Tingjie TANG, Jiajin HUANG, Jin QIN. Session-based recommendation with graph auxiliary learning [J]. Journal of Computer Applications, 2024, 44(9): 2711-2718.
[2]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[3]	Xingyao YANG, Yu CHEN, Jiong YU, Zulian ZHANG, Jiaying CHEN, Dongxiao WANG. Recommendation model combining self-features and contrastive learning [J]. Journal of Computer Applications, 2024, 44(9): 2704-2710.
[4]	Yu DU, Yan ZHU. Constructing pre-trained dynamic graph neural network to predict disappearance of academic cooperation behavior [J]. Journal of Computer Applications, 2024, 44(9): 2726-2731.
[5]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[6]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[7]	Hang YANG, Wanggen LI, Gensheng ZHANG, Zhige WANG, Xin KAI. Multi-layer information interactive fusion algorithm based on graph neural network for session-based recommendation [J]. Journal of Computer Applications, 2024, 44(9): 2719-2725.
[8]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[9]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[10]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[11]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[12]	Fan YANG, Yao ZOU, Mingzhi ZHU, Zhenwei MA, Dawei CHENG, Changjun JIANG. Credit card fraud detection model based on graph attention Transformation neural network [J]. Journal of Computer Applications, 2024, 44(8): 2634-2642.
[13]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[14]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[15]	Dahai LI, Zhonghua WANG, Zhendong WANG. Dual-branch low-light image enhancement network combining spatial and frequency domain information [J]. Journal of Computer Applications, 2024, 44(7): 2175-2182.