Text sentiment analysis model based on individual bias information

doi:10.11772/j.issn.1001-9081.2023010103

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (1): 145-151.DOI: 10.11772/j.issn.1001-9081.2023010103

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Text sentiment analysis model based on individual bias information

Li’an CHEN¹, Yi GUO¹^,²^,³()

^1.College of Information Science and Engineering，East China University of Science and Technology，Shanghai 200237，China
^2.National Engineering Laboratory for Big Data Distribution and Exchange Technologies （Shanghai Data Exchange），Shanghai 200436，China
^3.Shanghai Engineering Research Center of Big Data & Internet Audience，Shanghai 200072，China

Received:2023-02-09 Revised:2023-05-16 Accepted:2023-05-16 Online:2023-06-06 Published:2024-01-10
Contact: Yi GUO
About author:CHEN Li’an， born in 1998， M. S. candidate. Her research interests include text classification， sentiment analysis， aspect based sentiment analysis.
Supported by:
Science and Technology Program of Science and Technology Committee of Shanghai Municipality(22DZ1204903)

融合个体偏差信息的文本情感分析模型

陈丽安¹, 过弋¹^,²^,³()

^1.华东理工大学信息科学与工程学院, 上海 200237
^2.大数据流通与交易技术国家工程实验室(上海数据交易所), 上海 200436
^3.上海大数据与互联网受众工程技术研究中心, 上海 200072

通讯作者: 过弋
作者简介:陈丽安（1998—），女，安徽芜湖人，硕士研究生，主要研究方向：文本分类、情感分析、方面级情感分析；
第一联系人：过弋（1975—），男，陕西西安人，教授，博士生导师，博士，主要研究方向：文本挖掘、知识发现、商业智能。
基金资助:
上海市科学技术委员会科技计划项目(22DZ1204903)

Abstract

Abstract:

However，current text sentiment analysis often focus on the comment text itself， but ignore individual bias information between commenters and commentees， which has a considerable impact on the overall sentiment analysis. A text sentiment analysis model based on individual bias information， named UP-ATL （User and Product-Attention TranLSTM）， was proposed. In the model， self-attention mechanism and cross-attention mechanism were used to fuse the comment text and individual bias information in both directions. During the fusion process， a customized weight calculation method was used to alleviate the data sparsity problem caused by cold start in practical application scenarios. Finally， the feature fully fused comment text and bilateral representation information of the comment were obtained. Three real public datasets， Yelp2013， Yelp2014， and IMDB， were selected for effectiveness verification in the restaurant and film fields. The proposed model was compared with benchmark models such as UPNN （User Product Neural Network）， NSC （Neural Sentiment Classification）， CMA （Cascading Multiway Attention）and HUAPA （Hierarchical User And Product multi-head Attention）. The experimental results show that compared to the previous best performing HUAPA model， the accuracy of UP-ATL increases by 6.9 percentage points， 5.9 percentage points， and 1.6 percentage points， respectively on three datasets.

Key words: text sentiment analysis, self-attention mechanism, cross-attention mechanism, Transformer model, Long Short-Term Memory (LSTM) network

摘要：

目前情感分析任务经常只聚焦于评论文本本身，忽略了评论者与被评论者的个体偏差特征，会显著影响对文本的整体情感判断。针对上述问题，提出一种融合评论双边个体偏差信息的文本情感分析模型UP-ATL （User and Product-Attention TranLSTM）。该模型使用自注意力机制、交叉注意力机制对评论文本与个体偏差信息分别进行双向融合，在融合过程中采用定制化权重的计算方式，以缓解实际应用场景中冷启动带来的数据稀疏问题，最终得到特征充分融合的评论文本和评论双边的表示信息。选取餐饮领域、电影领域的三个真实公开数据集Yelp2013、Yelp2014、IMDB进行效果验证，与UPNN（User Product Neural Network）、NSC（Neural Sentiment Classification）、CMA（Cascading Multiway Attention）、HUAPA （Hierarchical User And Product multi-head Attention）等基准模型进行比较。实验结果表明，相较于比较模型中最好的HUAPA模型，UP-ATL的准确度在三个数据集上依次分别提高了6.9、5.9和1.6个百分点。

关键词: 文本情感分析, 自注意力机制, 交叉注意力机制, Transformer模型, 长短期记忆网络

CLC Number:

TP183

Li’an CHEN, Yi GUO. Text sentiment analysis model based on individual bias information[J]. Journal of Computer Applications, 2024, 44(1): 145-151.

陈丽安, 过弋. 融合个体偏差信息的文本情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(1): 145-151.

Figures/Tables 7

References 15

1	JELODAR H， WANG Y， ORJI R， et al. Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions： NLP using LSTM recurrent neural network approach ［J］. IEEE Journal of Biomedical and Health Informatics， 2020， 24（10）： 2733-2742. 10.1109/jbhi.2020.3001216
2	LI Q， JIN Z， WANG C， et al. Mining opinion summarizations using convolutional neural networks in Chinese microblogging systems ［J］. Knowledge-Based Systems， 2016， 107： 289-300. 10.1016/j.knosys.2016.06.017
3	韩建胜，陈杰，陈鹏，等.基于双向时间深度卷积网络的中文文本情感分类［J］.计算机应用与软件， 2019， 36（12）： 225-231. 10.3969/j.issn.1000-386x.2019.12.036
	HAN J S， CHEN J， CHEN P， et al. Chinese text sentiment classification based on bidirectional temporal deep convolutional network ［J］. Computer Applications and Software， 2019， 36（12）： 225-231. 10.3969/j.issn.1000-386x.2019.12.036
4	PERGOLA G， GUI L， HE Y. TDAM： a topic-dependent attention model for sentiment analysis ［J］. Information Processing & Management， 2019， 56（6）： 102084. 10.1016/j.ipm.2019.102084
5	赵容梅，熊熙，琚生根，等.基于混合神经网络的中文隐式情感分析［J］.四川大学学报（自然科学版）， 2020， 57（2）： 264-270.
	ZHAO R M， XIONG X， JU S G， et al. Implicit sentiment analysis for Chinese texts based on a hybrid neural network ［J］. Journal of Sichuan University （Natural Science Edition）， 2020， 57（2）： 264-270.
6	ARACI D. FinBERT： Financial sentiment analysis with pre-trained language models ［EB/OL］. ［2021-08-28］. . 10.1109/siu53274.2021.9477908
7	DELOBELLE P， WINTERS T， BERENDT B. RobBERT： a Dutch RoBERTa-based language model ［C］// Findings of the Association for Computational Linguistics： EMNLP 2020. Stroudsburg， PA： Association for Computational Linguistics， 2020： 3255-3265. 10.18653/v1/2020.findings-emnlp.292
8	ZHANG W， LI X， DENG Y， et al. A survey on aspect-based sentiment analysis： tasks， methods， and challenges ［J］. IEEE Transactions on Knowledge and Data Engineering， 2023， 35（11）： 11019-11038. 10.1109/tkde.2022.3230975
9	TANG D， QIN B， LIU T. Learning semantic representations of users and products for document level sentiment classification ［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2015： 1014-1023. 10.3115/v1/p15-1098
10	DOU Z-Y. Capturing user and product information for document level sentiment analysis with deep memory network ［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2017： 521-526. 10.18653/v1/d17-1054
11	CHEN H， SUN M， TU C， et al. Neural sentiment classification with user and product attention ［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2016： 1650-1659. 10.18653/v1/d16-1171
12	AMPLAYO R K， KIM J， SUNG S， et al. Cold-start aware user and product attention for sentiment classification ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 2535-2544. 10.18653/v1/p18-1236
13	WU Z， DAI X-Y， YIN C， et al. Improving review representations with user attention and product attention for sentiment classification ［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 5989-5996. 10.1609/aaai.v32i1.12054
14	LONG Y， MA M， LU Q， et al. Dual memory network model for biased product review classification ［C］// Proceedings of the 9th Workshop on Computational Approaches to Subjectivity， Sentiment and Social Media Analysis. Stroudsburg， PA： Association for Computational Linguistics， 2018： 140-148. 10.18653/v1/w18-6220
15	MA D， LI S， ZHANG X， et al. Cascading multiway attentions for document-level sentiment classification ［C］// Proceedings of the Eighth International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2017： 634-643.

实验环境	具体信息
操作系统	Ubuntu 18.03
内存	DDR4 64 GB
CPU	Intel Xeon Gold 5218
GPU	GeForce RTX 3090 （24 GB）
开发语言	Python 3.9.0
深度学习框架	PyTorch 1.9

实验环境	具体信息
操作系统	Ubuntu 18.03
内存	DDR4 64 GB
CPU	Intel Xeon Gold 5218
GPU	GeForce RTX 3090 （24 GB）
开发语言	Python 3.9.0
深度学习框架	PyTorch 1.9

真实情况	预测结果
真实情况	正例	反例
正例	TP	FN
反例	FP	TN

真实情况	预测结果
真实情况	正例	反例
正例	TP	FN
反例	FP	TN

模型	Yelp2013		Yelp2014		IMDB
模型	Acc/%	RMSE	Acc/%	RMSE	Acc/%	RMSE
UPNN^［9］	59.6	0.784	60.8	0.764	43.5	1.602
UPDMN^［10］	63.9	0.662	61.3	0.720	46.5	1.315
NSC^［11］	65.0	0.692	66.7	0.654	53.3	1.281
HCSC^［12］	65.7	0.660	67.6	0.639	54.2	1.213
HUAPA^［13］	68.3	0.628	68.6	0.626	55.0	1.185
DUPMN^［14］	66.2	0.667	67.6	0.639	53.9	1.279
CMA^［15］	66.4	0.677	67.6	0.637	54.0	1.191
UP-ATL	75.2	0.499	74.5	0.591	56.6	1.104

Text sentiment analysis model based on individual bias information

融合个体偏差信息的文本情感分析模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 15

Related Articles 15

Recommended Articles

Metrics

序号	Yelp2013		Yelp2014		IMDB
序号	Acc/%	RSME	Acc/%	RSME	Acc/%	RSME
Fold 1	74.6	0.503	74.3	0.608	56.5	1.121
Fold 2	74.9	0.526	73.2	0.635	55.9	1.167
Fold 3	75.6	0.491	73.8	0.603	56.3	1.134
Fold 4	74.1	0.548	74.7	0.589	56.7	1.109
Fold 5	74.8	0.535	74.2	0.597	56.6	1.104

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[3]	Zexin XU, Lei YANG, Kangshun LI. Shorter long-sequence time series forecasting model [J]. Journal of Computer Applications, 2024, 44(6): 1824-1831.
[4]	Yue LIU, Fang LIU, Aoyun WU, Qiuyue CHAI, Tianxiao WANG. 3D object detection network based on self-attention mechanism and graph convolution [J]. Journal of Computer Applications, 2024, 44(6): 1972-1977.
[5]	Rong HUANG, Junjie SONG, Shubo ZHOU, Hao LIU. Image aesthetic quality evaluation method based on self-supervised vision Transformer [J]. Journal of Computer Applications, 2024, 44(4): 1269-1276.
[6]	Xinran LUO, Tianrui LI, Zhen JIA. Chinese medical named entity recognition based on self-attention mechanism and lexicon enhancement [J]. Journal of Computer Applications, 2024, 44(2): 385-392.
[7]	Ziqi HUANG, Jianpeng HU. Entity category enhanced nested named entity recognition in automotive domain [J]. Journal of Computer Applications, 2024, 44(2): 377-384.
[8]	Liqing QIU, Xiaopan SU. Personalized multi-layer interest extraction click-through rate prediction model [J]. Journal of Computer Applications, 2024, 44(11): 3411-3418.
[9]	Xingyao YANG, Hongtao SHEN, Zulian ZHANG, Jiong YU, Jiaying CHEN, Dongxiao WANG. Sequential recommendation based on hierarchical filter and temporal convolution enhanced self-attention network [J]. Journal of Computer Applications, 2024, 44(10): 3090-3096.
[10]	Xiaoyu HUA, Dongfen LI, You FU, Kejun BI, Shi YING, Ruijin WANG. Industrial chain risk assessment and early warning model combining hierarchical graph neural network and long short-term memory [J]. Journal of Computer Applications, 2024, 44(10): 3223-3231.
[11]	Yanbo LI, Qing HE, Shunyi LU. Aspect sentiment triplet extraction integrating semantic and syntactic information [J]. Journal of Computer Applications, 2024, 44(10): 3275-3280.
[12]	Hanxiao SHI, Leichun WANG. Short-term power load forecasting by graph convolutional network combining LSTM and self-attention mechanism [J]. Journal of Computer Applications, 2024, 44(1): 311-317.
[13]	Zhiping ZHU, Yan YANG, Jie WANG. Scene graph-aware cross-modal image captioning model [J]. Journal of Computer Applications, 2024, 44(1): 58-64.
[14]	Jia CHEN, Hong ZHANG. Image text retrieval method based on feature enhancement and semantic correlation matching [J]. Journal of Computer Applications, 2024, 44(1): 16-23.
[15]	Guolong YUAN, Yujin ZHANG, Yang LIU. Image tampering forensics network based on residual feedback and self-attention [J]. Journal of Computer Applications, 2023, 43(9): 2925-2931.

模型	Yelp2013		Yelp2014		IMDB
模型	Acc/%	RMSE	Acc/%	RMSE	Acc/%	RMSE
UP-ATL	75.2	0.499	74.5	0.591	56.6	1.104
w/o Bi-LSTM	73.5（-1.7）	0.538	72.2（-2.3）	0.607	52.9（-3.7）	1.298
w/o Transformer	70.4（-4.8）	0.554	70.9（-3.6）	0.539	56.4（-0.2）	1.157
w/o Bi-LSTM+Transformer	67.8（-7.4）	0.638	69.5（-5.0）	0.606	55.0（-1.6）	1.206
w/o UP-Attention	72.1（-3.1）	0.542	71.7（-2.8）	0.583	55.5（-1.1）	1.185

模型	Yelp2013		Yelp2014		IMDB
模型	Acc/%	RMSE	Acc/%	RMSE	Acc/%	RMSE
UP-ATL	75.2	0.499	74.5	0.591	56.6	1.104
w/o Bi-LSTM	73.5（-1.7）	0.538	72.2（-2.3）	0.607	52.9（-3.7）	1.298
w/o Transformer	70.4（-4.8）	0.554	70.9（-3.6）	0.539	56.4（-0.2）	1.157
w/o Bi-LSTM+Transformer	67.8（-7.4）	0.638	69.5（-5.0）	0.606	55.0（-1.6）	1.206
w/o UP-Attention	72.1（-3.1）	0.542	71.7（-2.8）	0.583	55.5（-1.1）	1.185