Aspect-based sentiment analysis method with integrating prompt knowledge

doi:10.11772/j.issn.1001-9081.2022091347

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (9): 2753-2759.DOI: 10.11772/j.issn.1001-9081.2022091347

• Artificial intelligence • Previous Articles Next Articles

Aspect-based sentiment analysis method with integrating prompt knowledge

Xinyue ZHANG, Rong LIU(), Chiyu WEI, Ke FANG

College of Physical Science and Technology，Central China Normal University，Wuhan Hubei 430079，China

Received:2022-09-09 Revised:2022-11-11 Accepted:2022-11-15 Online:2023-02-14 Published:2023-09-10
Contact: Rong LIU
About author:ZHANG Xinyue， born in 1997， M. S. candidate. Her research interests include pattern recognition， aspect-based sentiment analysis.
WEI Chiyu， born in 1998， M. S. candidate. His research interests include deep learning， object detection.
FANG Ke， born in 1999， M. S. candidate. His research interests include deep learning， object detection.
Supported by:
Key Project of National Social Science Foundation of China(22ATQ004);Cross Science Research Project of Central China Normal University(CCNU22JC033)

融合提示知识的方面级情感分析方法

张心月, 刘蓉(), 魏驰宇, 方可

华中师范大学物理科学与技术学院，武汉 430079

通讯作者: 刘蓉
作者简介:张心月（1997—），女，河南周口人，硕士研究生，主要研究方向：模式识别、方面级情感分析
魏驰宇（1998—），男，河南周口人，硕士研究生，主要研究方向：深度学习、目标检测
方可（1999—），男，河南周口人，硕士研究生，主要研究方向：深度学习、目标检测。
基金资助:
国家社会科学基金重点项目(22ATQ004);华中师范大学交叉科学研究项目(CCNU22JC033)

Abstract

Abstract:

Aspect-based sentiment analysis based on pre-trained models generally uses end-to-end frameworks， has the problems of inconsistency between the upstream and downstream tasks， and is difficult to model the relationships between aspect words and context effectively. To address these problems， an aspect-based sentiment analysis method integrating prompt knowledge was proposed. First， in order to capture the semantic relation between aspect words and context effectively and enhance the model’s perception ability for sentiment analysis tasks， based on the Prompt mechanism， a prompt text was constructed and spliced with the original sentence and aspect words， and the obtained results were used as the input of the pre-trained model Bidirectional Encoder Representations from Transformers （BERT）. Then， a sentimental label vocabulary was built and integrated into the sentimental verbalizer layer， so as to reduce search space of the model， make the pre-trained model obtain rich semantic knowledge in the label vocabulary， and improve the learning ability of the model. Experimental results on Restaurant and Laptop field datasets of SemEval2014 Task4 dataset as well as ChnSentiCorp dataset show that the F1-score of the proposed method reaches 77.42%， 75.20% and 94.89% respectively， which is increased by 0.65 to 10.71， 1.02 to 9.58 and 0.83 to 6.40 percentage points compared with the mainstream aspect-based sentiment analysis methods such as Glove-TextCNN and P-tuning. The above verifies the effectiveness of the proposed method.

Key words: Natural Language Processing (NLP), aspect-based sentiment analysis, pre-trained model, prompt text, sentimental verbalizer

摘要：

针对基于预训练模型的方面级情感分析普遍使用端到端框架，存在上下游阶段任务不一致、难以有效建模方面词和上下文之间关系的问题，提出一种融合提示知识的方面级情感分析方法。首先基于Prompt机制构造提示文本，将该提示文本与原始句子和方面词进行拼接，并把得到的结果作为预训练模型BERT（Bidirectional Encoder Representation from Transformers）的输入，以有效捕获方面词和上下文之间的语义联系，同时提升模型对情感分析任务的感知能力；然后构建情感标签词表，并将它融入情感标签词映射层，以减小模型的搜索空间，使预训练模型获取标签词表中丰富的语义知识，并增强模型的学习能力。实验结果表明，所提方法在SemEval2014 Task4数据集的Restaurant、Laptop两个领域数据集和ChnSentiCorp数据集上的F1值分别达到了77.42%、75.20%、94.89%，与Glove-TextCNN、P-tuning等主流方面级情感分析方法相比提高了0.65~10.71、1.02~9.58与0.83~6.40个百分点，验证了所提方法对方面级情感分析的有效性。

关键词: 自然语言处理, 方面级情感分析, 预训练模型, 提示文本, 情感极性映射

CLC Number:

TP391.4

Xinyue ZHANG, Rong LIU, Chiyu WEI, Ke FANG. Aspect-based sentiment analysis method with integrating prompt knowledge[J]. Journal of Computer Applications, 2023, 43(9): 2753-2759.

张心月, 刘蓉, 魏驰宇, 方可. 融合提示知识的方面级情感分析方法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2753-2759.

Figures/Tables 15

Fig. 1 Structure of the proposed method

Fig. 2 Schematic diagram of MLM task

Fig. 3 Construction flow of prompt texts

Tab. 1 Some examples of prompt texts （partial）

数据集	$x p$
SemEval2014 Task4	Aspects， it was ［MASK］
ChnSentiCorp	是好评吗？［MASK］

Tab. 1 Some examples of prompt texts （partial）

数据集	$x p$
SemEval2014 Task4	Aspects， it was ［MASK］
ChnSentiCorp	是好评吗？［MASK］

Fig. 4 Example of input text for pre-trained text model

Fig. 5 Network structure of semantic encoding layer

Fig. 6 Construction flow of sentimental label word sets

Tab. 2 Examples of expanded label words

数据集	标签	标签词
SemEval2014 Task4	Positive	good，wonderful，great，…
	Negative	bad，upset，worse，…
	Neutral	indifferent， just ok，…
ChnSentiCorp	Positive	是，对，…
ChnSentiCorp	Negative	否，错，不，…

Tab. 3 SemEval2014 Task4 datasets

数据集		不同情感极性样本数
数据集		Positive	Neutral	Negative
Laptop	train	994	464	870
Laptop	test	341	169	128
Restaurant	train	2 164	637	807
Restaurant	test	728	196	196

Tab. 4 Examples of ChnSentiCorp dataset

评论文本	标签
很旧的设施，服务也不好，感觉一般，不能和大城市比。	0
第一感觉就是门童服务很到位，前台服务也面带微笑。房间宽敞明亮，上网速度也很快。很满意的一家酒店！	1
服务没有最坏只有更坏，先是早上没热水然后电梯也坏了。	0

Tab. 5 Experimental configuration

超参数	SemEval2014	ChnSentiCorp
预训练模型	BERT-base-uncased	BERT-base-chinese
最大文本长度	32	300
学习率	$1 × 10 - 5$	$1 × 10 - 5$
dropout	0.1	0.1
batch_size	8	8
epoch	10	10
分类类别	3	2

Tab. 5 Experimental configuration

超参数	SemEval2014	ChnSentiCorp
预训练模型	BERT-base-uncased	BERT-base-chinese
最大文本长度	32	300
学习率	$1 × 10 - 5$	$1 × 10 - 5$
dropout	0.1	0.1
batch_size	8	8
epoch	10	10
分类类别	3	2

Tab. 6 Experimental results on SemEval2014 dataset

方法	Laptop		Restaurant
方法	ACC	F1	ACC	F1
Glove-TextCNN	71.03	65.62	79.24	66.71
ELMo-Transformer	73.12	66.37	80.46	68.05
BERT-TextCNN^*	75.01	68.93	81.99	72.15
BERT-pair^*	74.66	68.64	81.92	71.97
BERT-BiLSTM	75.31	69.37	82.21	72.52
BMLA^*	76.73	71.50	83.54	74.91
P-tuning	76.95	74.18	83.98	76.77
本文方法	77.74	75.20	84.82	77.42

Tab. 7 Experimental results on ChnSentiCorp dataset

方法	ACC	F1
Glove-TextCNN	87.38	88.49
ELMo-Transformer	93.66	92.06
BERT-TextCNN	93.72	92.53
BERT-BiLSTM	94.05	94.06
P-tuning	87.12	89.82
本文方法	94.91	94.89

Tab. 8 Results of ablation experiment

组序	PT	SV	F1/%
组序	PT	SV	Laptop	Restaurant	ChnSentiCrop
1	×	×	68.89	71.58	91.56
2	×	√	74.05	75.03	94.08
3	√	×	74.97	77.26	94.15
4	√	√	75.20	77.42	94.89

Tab. 9 Average running time of ten iterations of different methods

方法	SemEval2014		ChnSentiCrop
方法	Laptop	Restaurant	ChnSentiCrop
P-tuning	1 700	2 140	840
BERT-BiLSTM	460	580	227
本文方法	220	282	110

References 28

1	ZHANG L， WANG S， LIU B. Deep learning for sentiment analysis： a survey［J］. WIREs Data Mining and Knowledge Discovery， 2018， 8（4）： No.e1253. 10.1002/widm.1253
2	LIN B， ZAMPETTI F， BAVOTA G， et al. Sentiment analysis for software engineering： how far can we go？［C］// Proceedings of the ACM/IEEE 40th International Conference on Software Engineering. New York： ACM， 2018： 94-104. 10.1145/3180155.3180195
3	QIU X P， SUN T X， XU Y G， et al. Pre-trained models for natural language processing： a survey［J］. Science China Technological Sciences， 2020， 63（10）： 1872-1897. 10.1007/s11431-020-1647-3
4	TANG D Y， QIN B， FENG X C， et al. Effective LSTMs for target-dependent sentiment classification［C］// Proceedings of the 26th International Conference on Computational Linguistics： Technical Papers. ［S.l.］： The COLING 2016 Organizing Committee， 2016： 3298-3307.
5	LIU M Z， ZHOU F Y， CHEN K， et al. Co-attention networks based on aspect and context for aspect-level sentiment analysis［J］. Knowledge-Based Systems， 2021， 217： No.106810. 10.1016/j.knosys.2021.106810
6	CHEN P， SUN Z Q， BING L D， et al. Recurrent attention network on memory for aspect sentiment analysis［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2017： 452-461. 10.18653/v1/d17-1047
7	CHEN Y Z， ZHUANG T H， GUO K. Memory network with hierarchical multi-head attention for aspect-based sentiment analysis［J］. Applied Intelligence， 2021， 51（7）： 4287-4304. 10.1007/s10489-020-02069-5
8	PENNINGTON J， SOCHER R， MANNING C D. GloVe： global vectors for word representation［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2014： 1532-1543. 10.3115/v1/d14-1162
9	MIKOLOV T， SUTSKEVER I， CHEN K， et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2. Red Hook， NY： Curran Associates Inc.， 2013： 3111-3119.
10	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg， PA： ACL， 2019： 4171-4186. 10.18653/v1/n18-2
11	PETERS M E， NEUMANN M， IYYER M， et al. Deep contextualized word representations［C］// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long Papers）. Stroudsburg， PA： ACL， 2018： 2227-2237. 10.18653/v1/n18-1202
12	SUN C， HUANG L Y， QIU X P. Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg， PA： ACL， 2019： 380-385.
13	XIA C Y， ZHANG C W， NGUYEN H， et al. CG-BERT： conditional text generation with BERT for generalized few-shot intent detection［EB/OL］. （2020-04-04）［2022-07-12］..
14	ZHANG K， ZHANG K， ZHANG M D， et al. Incorporating dynamic semantics into pre-trained language model for aspect-based sentiment analysis［EB/OL］. ［2022-05-25］. . 10.18653/v1/2022.findings-acl.285
15	BROWN T B， MANN B， RYDER N， et al. Language models are few-shot learners［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2020： 1877-1901. 10.18653/v1/2021.emnlp-main.734
16	LI C X， GAO F Y， BU J J， et al. SentiPrompt： sentiment knowledge enhanced prompt-tuning for aspect-based sentiment analysis［EB/OL］. （2021-09-17）［2022-07-12］..
17	JIANG Z B， XU F F， ARAKI J， et al. How can we know what language models know？［J］. Transactions of the Association for Computational Linguistics， 2020， 8：423-438. 10.1162/tacl_a_00324
18	GAO T Y， FISCH A， CHEN D Q. Making pre-trained language models better few-shot learners［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2021： 3816-3830. 10.18653/v1/2021.acl-long.295
19	LIU X， ZHENG Y N， DU Z X， et al. GPT understands， too［EB/OL］. （2021-03-18）［2022-07-12］.. 10.1016/j.aiopen.2023.08.012
20	SHIN T， RAZEGHI Y， LOGAN R L IV， et al. Autoprompt： eliciting knowledge from language models with automatically generated prompts［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2020： 4222-4235. 10.18653/v1/2020.emnlp-main.346
21	SCHICK T， SCHÜTZE H. Exploiting cloze-questions for few-shot text classification and natural language inference［C］// Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics： Main Volume. Stroudsburg， PA： ACL， 2021： 255-269. 10.18653/v1/2021.eacl-main.20
22	SCHICK T， SCHMID H， SCHUTZE H. Automatically identifying words that can serve as labels for few-shot text classification［C］// Proceedings of the 28th International Conference on Computational Linguistics. ［S.l.］： International Committee on Computational Linguistics， 2020： 5569-5578. 10.18653/v1/2020.coling-main.488
23	HU S D， DING N， WANG H D， et al. Knowledgeable prompt-tuning： incorporating knowledge into prompt verbalizer for text classification［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2022： 2225-2240. 10.18653/v1/2022.acl-long.158
24	赵亚欧，张家重，李贻斌，等. 基于ELMo和Transformer混合模型的情感分析［J］. 中文信息学报， 2021， 35（3）： 115-124. 10.3969/j.issn.1003-0077.2021.03.012
	ZHAO Y O， ZHANG J C， LI Y B， et al. Sentiment analysis based on hybrid model of ELMo and Transformer［J］. Journal of Chinese Information Processing， 2021， 35（3）： 115-124. 10.3969/j.issn.1003-0077.2021.03.012
25	NGUYEN Q T， NGUYEN T L， LUONG N H， et al. Fine-tuning bert for sentiment analysis of vietnamese reviews［C］// Proceedings of the 7th NAFOSTED Conference on Information and Computer Science. Piscataway： IEEE， 2020： 302-307. 10.1109/nics51282.2020.9335899
26	SHAHEEN M， NIGAM S. Plumeria at SemEval-2022 Task 6： sarcasm detection for english and arabic using transformers and data augmentation［C］// Proceedings of the 16th International Workshop on Semantic Evaluation. Stroudsburg， PA： ACL， 2022： 923-937. 10.18653/v1/2022.semeval-1.130
27	袁勋，刘蓉，刘明. 融合多层注意力的方面级情感分析模型［J］. 计算机工程与应用， 2021， 57（22）： 147-152.
	YUAN X， LIU R， LIU M. Aspect-level analysis model incorporating multi-layer attention［J］. Computer Engineering and Applications， 2021， 57（22）： 147-152.
28	SUN T X， LIU X Y， QIU X P， et al. Paradigm shift in natural language processing［J］. Machine Intelligence Research， 2022， 19（3）：169-183. 10.1007/s11633-022-1331-6

[1]	Xiaomin ZHOU, Fei TENG, Yi ZHANG. Automatic international classification of diseases coding model based on meta-network [J]. Journal of Computer Applications, 2023, 43(9): 2721-2726.
[2]	Yuelin TIAN, Ruizhang HUANG, Lina REN. Scholar fine-grained information extraction method fused with local semantic features [J]. Journal of Computer Applications, 2023, 43(9): 2707-2714.
[3]	Bihui YU, Xingye CAI, Jingxuan WEI. Few-shot text classification method based on prompt learning [J]. Journal of Computer Applications, 2023, 43(9): 2735-2740.
[4]	Hongjun HENG, Dingcheng YANG. Knowledge enhanced aspect word interactive graph neural network [J]. Journal of Computer Applications, 2023, 43(8): 2412-2419.
[5]	Xiaoyan ZHANG, Zhengyu DUAN. Cross-lingual zero-resource named entity recognition model based on sentence-level generative adversarial network [J]. Journal of Computer Applications, 2023, 43(8): 2406-2411.
[6]	Zexi JIN, Lei LI, Ji LIU. Transfer learning model based on improved domain separation network [J]. Journal of Computer Applications, 2023, 43(8): 2382-2389.
[7]	Yao LIU, Xin TONG, Yifeng CHEN. Algorithm path self-assembling model for business requirements [J]. Journal of Computer Applications, 2023, 43(6): 1768-1778.
[8]	Zhixiong ZHENG, Jianhua LIU, Shuihua SUN, Ge XU, Honghui LIN. Aspect-based sentiment analysis model fused with multi-window local information [J]. Journal of Computer Applications, 2023, 43(6): 1796-1802.
[9]	Lifeng SHI, Zhengwei NI. Dialogue state tracking model based on slot correlation information extraction [J]. Journal of Computer Applications, 2023, 43(5): 1430-1437.
[10]	Xingbin LIAO, Xiaolin QIN, Siqi ZHANG, Yangge QIAN. Review of interactive machine translation [J]. Journal of Computer Applications, 2023, 43(2): 329-334.
[11]	Ming XU, Linhao LI, Qiaoling QI, Liqin WANG. Abductive reasoning model based on attention balance list [J]. Journal of Computer Applications, 2023, 43(2): 349-355.
[12]	LIU Huan, DOU Quansheng. Aspect-based sentiment analysis model embedding different neighborhood representations [J]. Journal of Computer Applications, 2023, 43(1): 37-44.
[13]	LIU Hui, MA Xiang, ZHANG Linyu, HE Rujin. Aspect-based sentiment analysis model integrating match-LSTM network and grammatical distance [J]. Journal of Computer Applications, 2023, 43(1): 45-50.
[14]	Yingjie WANG, Jiuqi ZHU, Zumin WANG, Fengbo BAI, Jian GONG. Review of applications of natural language processing in text sentiment analysis [J]. Journal of Computer Applications, 2022, 42(4): 1011-1020.
[15]	Yaming LI, Kai XING, Hongwu DENG, Zhiyong WANG, Xuan HU. Derivative-free few-shot learning based performance optimization method of pre-trained models with convolution structure [J]. Journal of Computer Applications, 2022, 42(2): 365-374.

Aspect-based sentiment analysis method with integrating prompt knowledge

融合提示知识的方面级情感分析方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 28

Related Articles 15

Recommended Articles

Metrics