Aspect sentiment triplet extraction based on aspect-aware attention enhancement

doi:10.11772/j.issn.1001-9081.2023040411

Abstract

Abstract:

For fine-grained sentiment analysis in Natural Language Processing （NLP）， in order to explore the influence of Pre-trained Language Models （PLMs） with structural biases on the end-to-end sentiment triple extraction task， and solve the problem of low fault tolerance rate of aspect semantic feature dependence that is common in previous studies， combining aspect-aware attention mechanism and Graph Convolutional Network （GCN）， an Aspect-aware attention Enhanced GCN （AE-GCN） model was proposed for aspect sentiment triple extraction tasks. Firstly， multiple types of relations were introduced for the aspect sentiment triple extraction task. Then， these relations were embedded into the adjacent tensors between words in the sentence by using the double affine attention mechanism. At the same time， the aspect-aware attention mechanism was introduced to obtain the sentence attention scoring matrix， and the aspect-related semantic features were further mined. Next， a sentence was converted into a multi-channel graph through the graph convolutional neural network， to learn a relation-aware node representation by treating words and relation adjacent tensors as edges and nodes， respectively. Finally， an effective word pair representation refinement strategy was used to determine whether word pairs matched， which was used to consider the implicit results of aspect and opinion extraction. Experimental results show that， on ASTE-D1 benchmark dataset， the F1 values of the proposed model on the 14res， 14lap， 15res and 16res sub-datasets are improved by 0.20， 0.21， 1.25 and 0.26 percentage points compared with the Enhanced Multi-Channel Graph Convolutional Network （EMC-GCN） model； on ASTE-D2 benchmark dataset， the F1 values of the proposed model on the 14lap， 15res and 16res sub-datasets are increased by 0.42， 0.31 and 2.01 percentage points compared with the EMC-GCN model. It can be seen that the proposed model has great improvement in precision and effectiveness compared with the EMC-GCN model.

Key words: Natural Language Processing (NLP), sentiment analysis, sentiment triplet extraction, aspect-aware attention, Graph Convolutional Network (GCN)

摘要：

在自然语言处理（NLP）的细粒度情感分析问题中，为探索携带结构偏差的预训练语言模型（PLM）对端到端式情感三元组抽取任务的影响，解决方面语义特征依赖容错率低的问题，结合方面感知注意力机制和图卷积网络（GCN），提出用于方面情感三元组抽取任务的方面感知注意力增强图卷积网络（AE-GCN）模型。首先，在方面情感三元组抽取任务中引入多种类型的关系；其次，采用双仿射注意力机制将这些关系嵌入句子中单词之间的相邻张量，并引入方面感知注意力机制以获取句子注意力评分矩阵，深入挖掘与方面相关的语义特征；再次，GCN通过将单词和关系相邻张量分别视为边和节点，将句子转换为多通道图以学习关系感知节点表示；最后，使用一种有效的词对表示细化策略确定词对是否匹配，以考虑方面和意见抽取的隐含结果。在ASTE-D1基准数据集上的实验结果表明，所提模型在14res、14lap、15res和16res子数据集上的F1值相较于增强型多通道图卷积网络（EMC-GCN）模型提升了0.20、0.21、1.25和0.26个百分点；在ASTE-D2基准数据集上，所提模型在14lap、15res和16res子数据集上的F1值相较于EMC-GCN模型提升了0.42、0.31和2.01个百分点。可见所提模型相较于EMC-GCN模型在精确率和有效性方面有较大改进。

关键词: 自然语言处理, 情感分析, 情感三元组抽取, 方面感知注意力, 图卷积网络

CLC Number:

TP391.1

Longtao GAO, Nana LI. Aspect sentiment triplet extraction based on aspect-aware attention enhancement[J]. Journal of Computer Applications, 2024, 44(4): 1049-1057.

高龙涛, 李娜娜. 基于方面感知注意力增强的方面情感三元组抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1049-1057.

Figures/Tables 14

Tab. 1 Seven subtasks of aspect-level sentiment analysis

任务	输入	输出	任务类型
AE	S	（a₁，a₂）	抽取
OE	S	（o₁，o₂）	抽取
ALSC	S+a₁	s₁	分类
ALSC	S+a₂	s₂	分类
AOE	S+a₁	o₁	抽取
AOE	S+a₂	o₂	抽取
AESC	S	（a₁，s₁），（a₂，s₂）	抽取&分类
Pair	S	（a₁，o₁），（a₂，o₂）	抽取
TE	S	（a₁，o₁，s₁），（a₂，o₂，s₂）	抽取&分类

Fig. 1 Framework of proposed model

Fig. 2 Example of aspect sentiment triple extraction

Tab. 2 Label relationships between words

序号	输入	输出
1	B-A	方面项的开始
2	I-A	在方面项中
3	A	词对 $(w i, w j)$ 属于同一方面项
4	B-O	意见项的开始
5	I-O	在意见项中
6	O	词对 $(w i, w j)$ 属于同一意见项
7	POS	词对 $(w i, w j)$ 中的 $w i$ 和 $w j$ 分别属于方面词和意见词，它们与积极、中立、消极这3种情绪极性构成了对
8	NEU
9	NEG
10	NONE	词对 $(w i, w j)$ 之间没有上述关系

Tab. 2 Label relationships between words

序号	输入	输出
1	B-A	方面项的开始
2	I-A	在方面项中
3	A	词对 $(w i, w j)$ 属于同一方面项
4	B-O	意见项的开始
5	I-O	在意见项中
6	O	词对 $(w i, w j)$ 属于同一意见项
7	POS	词对 $(w i, w j)$ 中的 $w i$ 和 $w j$ 分别属于方面词和意见词，它们与积极、中立、消极这3种情绪极性构成了对
8	NEU
9	NEG
10	NONE	词对 $(w i, w j)$ 之间没有上述关系

Tab. 3 Relationship table filling

单词	单词
单词	The	gourmet	food	is	delicious	but	the	service	is	poor
The	N	N	N	N	N	N	N	N	N	N
gourmet	N	B‑A	A	N	POS	N	N	N	N	N
food	N	A	I‑A	N	POS	N	N	N	N	N
is	N	N	N	N	N	N	N	N	N	N
delicious	N	POS	POS	N	B‑O	N	N	N	N	N
but	N	N	N	N	N	N	N	N	N	N
the	N	N	N	N	N	N	N	N	N	N
service	N	N	N	N	N	N	N	B‑A	N	NEG
is	N	N	N	N	N	N	N	N	N	N
poor	N	N	N	N	N	N	N	NEG	N	B‑O

Fig. 3 Structural differences between self-attention and self-attention with structural bias adapter

Fig. 4 AE-GCN model integrated with PLM structure bias adapter （part）

Fig. 5 Example of dependency tree

Fig. 6 Framework of AE-GCN model

Tab. 4 Statistics of two datasets

数据集		14res		14lap		15res		16res
数据集		#S	#T	#S	#T	#S	#T	#S	#T
D1	训练集	1 259	2 356	899	1 452	603	1 038	863	1 421
	验证集	315	580	225	383	151	239	216	348
	测试集	493	1 008	332	547	325	493	328	525
D2	训练集	1 266	2 338	906	1 460	605	1 013	857	1 394
	验证集	310	577	219	346	148	249	210	339
	测试集	492	994	328	543	322	485	326	514

Tab. 5 Experimental results on D1 dataset

模型	14res			14lap			15res			16res
模型	P	R	F₁	P	R	F₁	P	R	F₁	P	R	F₁
Peng-two-stage+IOG	58.89	60.41	59.64	48.62	45.52	47.02	51.70	46.04	48.71	59.25	58.09	58.67
IMN+IOG	59.57	63.88	61.65	49.21	46.23	47.68	55.24	52.33	53.75	—	—	—
GTS-CNN	70.79	61.71	65.94	55.93	47.52	51.38	60.09	53.57	56.64	62.63	66.98	64.73
GTS-BiLSTM	67.28	61.91	64.49	59.42	45.13	51.30	63.26	50.71	56.29	66.07	65.05	65.56
S³E²	69.08	64.55	66.74	59.43	46.23	52.01	61.06	56.44	58.66	71.08	63.13	66.87
GTS-BERT	70.92	69.49	70.20	57.52	51.92	54.58	59.29	58.07	58.67	68.58	66.60	67.58
BMRC	—	—	70.01	—	—	57.83	—	—	58.74	—	—	67.49
EMC-GCN	71.85	72.12	71.98	61.46	55.56	58.32	59.89	61.05	60.38	65.08	71.66	68.18
AE-GCN	70.82	73.59	72.18	59.47	57.62	58.53	60.43	62.88	61.63	65.79	71.32	68.44

Tab. 6 Experimental results on D2 dataset

模型	14res			14lap			15res			16res
模型	P	R	F₁	P	R	F₁	P	R	F₁	P	R	F₁
Peng-two-stage+IOG	43.24	63.66	51.46	37.38	50.38	42.87	48.07	57.51	52.32	46.96	64.24	54.21
OTE-MTL	62.00	55.97	58.71	49.53	39.22	43.42	56.37	40.94	47.13	62.88	52.10	56.96
JET-BERT	70.56	55.94	62.40	55.39	47.33	51.04	64.45	51.96	57.53	70.42	58.37	63.83
GTS-BERT	68.09	69.54	68.81	59.40	51.94	55.42	59.28	57.93	58.60	68.32	66.86	67.58
BMRC	75.61	61.77	67.99	70.55	48.98	57.82	68.51	53.40	60.02	71.20	61.08	65.75
BART-ABSA	65.52	64.99	65.25	61.41	56.19	58.69	59.14	59.38	59.26	66.60	68.68	67.62
EMC-GCN	71.21	72.39	71.78	61.70	56.26	58.81	61.54	62.47	61.93	65.62	71.30	68.33
AE-GCN	71.37	72.02	71.70	60.08	58.41	59.23	62.63	61.86	62.24	67.44	73.49	70.34

Fig. 7 Comparison of model results on two datasets

Tab. 7 Experimental results （F1 values） of ablation study on D2 dataset

AE	结构偏差	14res	14lap	15res	16res
—	—	71.78	58.81	61.93	68.33
√	—	72.17	60.59	62.12	68.84
—	√	71.84	59.11	62.07	69.35
√	√	71.70	59.23	62.24	70.34

References 34

1	赵妍妍，秦兵，刘挺. 文本情感分析［J］. 软件学报，2010，21（8）：1834-1848. 10.3724/sp.j.1001.2010.03832
	ZHAO Y Y. QIN B， LIU T. Sentiment analysis ［J］. Journal of Software， 2010，21（8）： 1834-1848. 10.3724/sp.j.1001.2010.03832
2	HU M， LIU B. Mining and summarizing customer reviews［C］// Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2004： 168-177. 10.1145/1014052.1014073
3	XUE W， LI T. Aspect based sentiment analysis with gated convolutional networks ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2018： 2514-2523. 10.18653/v1/p18-1234
4	PENG H， XU L， BING L， et al. Knowing what， how and why： a near complete solution for aspect-based sentiment analysis ［C］// Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 8600-8607. 10.1609/aaai.v34i05.6383
5	ZHANG Z， ZHOU Z， WANG Y. SSEGCN： syntactic and semantic enhanced graph convolutional network for aspect-based sentiment analysis ［C］// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2022： 4916-4925. 10.18653/v1/2022.naacl-main.362
6	ZHANG C， REN L， MA F， et al. Structural bias for aspect sentiment triplet extraction ［C］// Proceedings of the 29th International Conference on Computational Linguistics. Stroudsburg： ACL， 2022： 6736-6745.
7	YAN H， DAI J， JI T， et al. A unified generative framework for aspect-based sentiment analysis ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2020： 2416-2429. 10.18653/v1/2021.acl-long.188
8	LIU B. Synthesis Lectures on Human Language Technologies［M］. Sentiment Analysis and Opinion Mining. ［S.l.］： Springer， 2012： 1-167. 10.2200/s00416ed1v01y201204hlt016
9	LI X， LAM W. Deep multi-task learning for aspect term extraction with memory interaction ［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2017： 2886-2892. 10.18653/v1/d17-1310
10	XU H， LIU B， SHU L， et al. Double embeddings and CNN-based sequence labeling for aspect extraction ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg： ACL， 2018： 592-598. 10.18653/v1/p18-2094
11	HE R， LEE W S， NG H T， et al. An interactive multi-task learning network for end-to-end aspect-based sentiment analysis ［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2019： 504-515. 10.18653/v1/p19-1048
12	武婷，曹春萍. 融合位置权重的基于注意力交叉注意力的长短期记忆方面情感分析模型［J］. 计算机应用， 2019， 39（8）： 2198-2203. 10.11772/j.issn.1001-9081.2018122565
	WU T， CAO C P. Aspect level sentiment analysis model with location weight and long short-term memory based on attention-over-attention ［J］. Journal of Computer Applications， 2019， 39（8）： 2198-2203. 10.11772/j.issn.1001-9081.2018122565
13	FAN Z F， WU Z， DAI X Y， et al. Target-oriented opinion words extraction with target-fused neural sequence labeling ［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg： ACL， 2019： 2509-2518. 10.18653/v1/n19-1259
14	WU Z， ZHAO F， DAI X-Y， et al. Latent opinions transfer network for target-oriented opinion words extraction ［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2020， 34（5）： 9298-9305. 10.1609/aaai.v34i05.6469
15	MITCHELL M， AGUILAR J， WILSON T， et al. Open domain targeted sentiment ［C］// Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2013： 1643-1654.
16	LI X， BING L， LI P， et al. A unified model for opinion target extraction and target sentiment prediction ［C］// Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto： AAAI Press， 2019： 6714-6721. 10.1609/aaai.v33i01.33016714
17	ZHAO H， HUANG L， ZHANG R， et al. SpanMlt： a span-based multi-task learning framework for pair-wise aspect and opinion terms extraction ［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020： 3239-3248. 10.18653/v1/2020.acl-main.296
18	XU L， LI H， LU W， et al. Position-aware tagging for aspect sentiment triplet extraction ［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2020： 2339-2349. 10.18653/v1/2020.emnlp-main.183
19	LIU S， LI K， LI Z. A robustly optimized BMRC for aspect sentiment triplet extraction ［C］// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg： ACL， 2022： 272-278. 10.18653/v1/2022.naacl-main.20
20	ZHANG C， LI Q， SONG D， et al. A multi-task learning framework for opinion triplet extraction ［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2020： 819-828. 10.18653/v1/2020.findings-emnlp.72
21	WU Z， YING C， ZHAO F， et al. Grid tagging scheme for aspect-oriented fine-grained opinion extraction ［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2020： 2576-2585. 10.18653/v1/2020.findings-emnlp.234
22	CHEN H， ZHAI Z. Enhanced multi-channel graph convolutional network for aspect sentiment triplet extraction ［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2022： 2974-2985. 10.18653/v1/2022.acl-long.212
23	GUPTA P， SCHÜTZE H， ANDRASSY B. Table filling multi-task recurrent neural network for joint entity and relation extraction ［C］// Proceedings of the 26th International Conference on Computational Linguistics： Technical Papers. Osaka， Japan： The COLING 2016 Organizing Committee， 2016： 2537-2547.
24	DEVLIN J， CHANG M-W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding ［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg： ACL， 2019： 4171-4186. 10.18653/v1/n18-2
25	CHANG T， XU Y， XU W， et al. Convolutions and self-attention： re-interpreting relative positions in pre-trained language models ［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2021： 4322-4333. 10.18653/v1/2021.acl-long.333
26	MA F， ZHANG C， SONG D W. Exploiting position bias for robust aspect sentiment classification ［C］// Findings of the Association for Computational Linguistics：ACL-IJCNLP 2021. Stroudsburg： ACL， 2021： 1352-1358. 10.18653/v1/2021.findings-acl.116
27	RAFFEL C， SHAZEER N， ROBERTS A， et al. Exploring the limits of transfer learning with a unified text-to-text transformer［J］. The Journal of Machine Learning Research， 2020， 21（1）： 140.
28	FAN F， FENG Y， ZHAO D. Multi-grained attention network for aspect-level sentiment classification ［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2018： 3433-3442. 10.18653/v1/d18-1380
29	LI L， LIU Y， ZHOU A Q. Hierarchical attention based position-aware network for aspect-level sentiment analysis ［C］// Proceedings of the 22nd Conference on Computational Natural Language Learning. Stroudsburg： ACL， 2018： 181-189. 10.18653/v1/k18-1018
30	DOZAT T， MANNING C D. Deep biaffine attention for neural dependency parsing ［C/OL］// Proceedings of the 5th International Conference on Learning Representations ［2023-04-01］. . 10.18653/v1/k17-3002
31	LI R， CHEN H， FENG F， et al. Dual graph convolutional networks for aspect-based sentiment analysis ［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2021： 6319-6329. 10.18653/v1/2021.acl-long.494
32	READ J， PFAHRINGER B， HOLMES G， et al. Classifier chains for multi-label classification ［J］. Machine Learning， 2011， 85（3）： 333-359. 10.1007/s10994-011-5256-5
33	LI C， GAO F， BU J， et al. SentiPrompt： sentiment knowledge enhanced promp-tuning for aspect-based sentiment analysis ［EB/OL］. ［2023-04-01］. .
34	CHEN Z， HUANG H， LIU B， et al. Semantic and syntactic enhanced aspect sentiment triplet extraction ［C］// Findings of the Association for Computational Linguistics： ACL-IJCNLP 2021. Stroudsburg： ACL， 2021： 1474-1483. 10.48550/arXiv.2106.03315

[1]	Baoshan YANG, Zhi YANG, Xingyuan CHEN, Bing HAN, Xuehui DU. Analysis of consistency between sensitive behavior and privacy policy of Android applications [J]. Journal of Computer Applications, 2024, 44(3): 788-796.
[2]	Lei GUO, Zhen JIA, Tianrui LI. Relational and interactive graph attention network for aspect-level sentiment analysis [J]. Journal of Computer Applications, 2024, 44(3): 696-701.
[3]	Kaitian WANG, Qing YE, Chunlei CHENG. Classification method for traditional Chinese medicine electronic medical records based on heterogeneous graph representation [J]. Journal of Computer Applications, 2024, 44(2): 411-417.
[4]	Li’an CHEN, Yi GUO. Text sentiment analysis model based on individual bias information [J]. Journal of Computer Applications, 2024, 44(1): 145-151.
[5]	Chenghao FENG, Zhenping XIE, Bowen DING. Selective generation method of test cases for Chinese text error correction software [J]. Journal of Computer Applications, 2024, 44(1): 101-112.
[6]	Hanxiao SHI, Leichun WANG. Short-term power load forecasting by graph convolutional network combining LSTM and self-attention mechanism [J]. Journal of Computer Applications, 2024, 44(1): 311-317.
[7]	Junhao LUO, Yan ZHU. Multi-dynamic aware network for unaligned multimodal language sequence sentiment analysis [J]. Journal of Computer Applications, 2024, 44(1): 79-85.
[8]	Xiaomin ZHOU, Fei TENG, Yi ZHANG. Automatic international classification of diseases coding model based on meta-network [J]. Journal of Computer Applications, 2023, 43(9): 2721-2726.
[9]	Xinyue ZHANG, Rong LIU, Chiyu WEI, Ke FANG. Aspect-based sentiment analysis method with integrating prompt knowledge [J]. Journal of Computer Applications, 2023, 43(9): 2753-2759.
[10]	Hongjun HENG, Dingcheng YANG. Knowledge enhanced aspect word interactive graph neural network [J]. Journal of Computer Applications, 2023, 43(8): 2412-2419.
[11]	Zexi JIN, Lei LI, Ji LIU. Transfer learning model based on improved domain separation network [J]. Journal of Computer Applications, 2023, 43(8): 2382-2389.
[12]	Yi ZHANG, Gangsheng CAI, Zhenmei WANG. Long non-coding RNA-disease association prediction model based on semantic and global dual attention mechanism [J]. Journal of Computer Applications, 2023, 43(7): 2125-2132.
[13]	Yuan WEI, Yan LIN, Shengnan GUO, Youfang LIN, Huaiyu WAN. Prediction of taxi demands between urban regions by fusing origin-destination spatial-temporal correlation [J]. Journal of Computer Applications, 2023, 43(7): 2100-2106.
[14]	Yao LIU, Xin TONG, Yifeng CHEN. Algorithm path self-assembling model for business requirements [J]. Journal of Computer Applications, 2023, 43(6): 1768-1778.
[15]	Zhixiong ZHENG, Jianhua LIU, Shuihua SUN, Ge XU, Honghui LIN. Aspect-based sentiment analysis model fused with multi-window local information [J]. Journal of Computer Applications, 2023, 43(6): 1796-1802.