Text sentiment analysis model based on individual bias information

doi:10.11772/j.issn.1001-9081.2023010103

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (1): 145-151.DOI: 10.11772/j.issn.1001-9081.2023010103

• Artificial intelligence • Previous Articles

Text sentiment analysis model based on individual bias information

Li’an CHEN¹, Yi GUO¹^,²^,³()

^1.College of Information Science and Engineering，East China University of Science and Technology，Shanghai 200237，China
^2.National Engineering Laboratory for Big Data Distribution and Exchange Technologies （Shanghai Data Exchange），Shanghai 200436，China
^3.Shanghai Engineering Research Center of Big Data & Internet Audience，Shanghai 200072，China

Received:2023-02-09 Revised:2023-05-16 Accepted:2023-05-16 Online:2023-06-06 Published:2024-01-10
Contact: Yi GUO
About author:CHEN Li’an， born in 1998， M. S. candidate. Her research interests include text classification， sentiment analysis， aspect based sentiment analysis.
Supported by:
Science and Technology Program of Science and Technology Committee of Shanghai Municipality(22DZ1204903)

融合个体偏差信息的文本情感分析模型

陈丽安¹, 过弋¹^,²^,³()

^1.华东理工大学信息科学与工程学院, 上海 200237
^2.大数据流通与交易技术国家工程实验室(上海数据交易所), 上海 200436
^3.上海大数据与互联网受众工程技术研究中心, 上海 200072

通讯作者: 过弋
作者简介:陈丽安（1998—），女，安徽芜湖人，硕士研究生，主要研究方向：文本分类、情感分析、方面级情感分析；
第一联系人：过弋（1975—），男，陕西西安人，教授，博士生导师，博士，主要研究方向：文本挖掘、知识发现、商业智能。
基金资助:
上海市科学技术委员会科技计划项目(22DZ1204903)

Abstract

Abstract:

However，current text sentiment analysis often focus on the comment text itself， but ignore individual bias information between commenters and commentees， which has a considerable impact on the overall sentiment analysis. A text sentiment analysis model based on individual bias information， named UP-ATL （User and Product-Attention TranLSTM）， was proposed. In the model， self-attention mechanism and cross-attention mechanism were used to fuse the comment text and individual bias information in both directions. During the fusion process， a customized weight calculation method was used to alleviate the data sparsity problem caused by cold start in practical application scenarios. Finally， the feature fully fused comment text and bilateral representation information of the comment were obtained. Three real public datasets， Yelp2013， Yelp2014， and IMDB， were selected for effectiveness verification in the restaurant and film fields. The proposed model was compared with benchmark models such as UPNN （User Product Neural Network）， NSC （Neural Sentiment Classification）， CMA （Cascading Multiway Attention）and HUAPA （Hierarchical User And Product multi-head Attention）. The experimental results show that compared to the previous best performing HUAPA model， the accuracy of UP-ATL increases by 6.9 percentage points， 5.9 percentage points， and 1.6 percentage points， respectively on three datasets.

Key words: text sentiment analysis, self-attention mechanism, cross-attention mechanism, Transformer model, Long Short-Term Memory (LSTM) network

摘要：

目前情感分析任务经常只聚焦于评论文本本身，忽略了评论者与被评论者的个体偏差特征，会显著影响对文本的整体情感判断。针对上述问题，提出一种融合评论双边个体偏差信息的文本情感分析模型UP-ATL （User and Product-Attention TranLSTM）。该模型使用自注意力机制、交叉注意力机制对评论文本与个体偏差信息分别进行双向融合，在融合过程中采用定制化权重的计算方式，以缓解实际应用场景中冷启动带来的数据稀疏问题，最终得到特征充分融合的评论文本和评论双边的表示信息。选取餐饮领域、电影领域的三个真实公开数据集Yelp2013、Yelp2014、IMDB进行效果验证，与UPNN（User Product Neural Network）、NSC（Neural Sentiment Classification）、CMA（Cascading Multiway Attention）、HUAPA （Hierarchical User And Product multi-head Attention）等基准模型进行比较。实验结果表明，相较于比较模型中最好的HUAPA模型，UP-ATL的准确度在三个数据集上依次分别提高了6.9、5.9和1.6个百分点。

关键词: 文本情感分析, 自注意力机制, 交叉注意力机制, Transformer模型, 长短期记忆网络

CLC Number:

TP183

Li’an CHEN, Yi GUO. Text sentiment analysis model based on individual bias information[J]. Journal of Computer Applications, 2024, 44(1): 145-151.

陈丽安, 过弋. 融合个体偏差信息的文本情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(1): 145-151.

Figures/Tables 7

References 15

1	JELODAR H， WANG Y， ORJI R， et al. Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions： NLP using LSTM recurrent neural network approach ［J］. IEEE Journal of Biomedical and Health Informatics， 2020， 24（10）： 2733-2742. 10.1109/jbhi.2020.3001216
2	LI Q， JIN Z， WANG C， et al. Mining opinion summarizations using convolutional neural networks in Chinese microblogging systems ［J］. Knowledge-Based Systems， 2016， 107： 289-300. 10.1016/j.knosys.2016.06.017
3	韩建胜，陈杰，陈鹏，等.基于双向时间深度卷积网络的中文文本情感分类［J］.计算机应用与软件， 2019， 36（12）： 225-231. 10.3969/j.issn.1000-386x.2019.12.036
	HAN J S， CHEN J， CHEN P， et al. Chinese text sentiment classification based on bidirectional temporal deep convolutional network ［J］. Computer Applications and Software， 2019， 36（12）： 225-231. 10.3969/j.issn.1000-386x.2019.12.036
4	PERGOLA G， GUI L， HE Y. TDAM： a topic-dependent attention model for sentiment analysis ［J］. Information Processing & Management， 2019， 56（6）： 102084. 10.1016/j.ipm.2019.102084
5	赵容梅，熊熙，琚生根，等.基于混合神经网络的中文隐式情感分析［J］.四川大学学报（自然科学版）， 2020， 57（2）： 264-270.
	ZHAO R M， XIONG X， JU S G， et al. Implicit sentiment analysis for Chinese texts based on a hybrid neural network ［J］. Journal of Sichuan University （Natural Science Edition）， 2020， 57（2）： 264-270.
6	ARACI D. FinBERT： Financial sentiment analysis with pre-trained language models ［EB/OL］. ［2021-08-28］. . 10.1109/siu53274.2021.9477908
7	DELOBELLE P， WINTERS T， BERENDT B. RobBERT： a Dutch RoBERTa-based language model ［C］// Findings of the Association for Computational Linguistics： EMNLP 2020. Stroudsburg， PA： Association for Computational Linguistics， 2020： 3255-3265. 10.18653/v1/2020.findings-emnlp.292
8	ZHANG W， LI X， DENG Y， et al. A survey on aspect-based sentiment analysis： tasks， methods， and challenges ［J］. IEEE Transactions on Knowledge and Data Engineering， 2023， 35（11）： 11019-11038. 10.1109/tkde.2022.3230975
9	TANG D， QIN B， LIU T. Learning semantic representations of users and products for document level sentiment classification ［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2015： 1014-1023. 10.3115/v1/p15-1098
10	DOU Z-Y. Capturing user and product information for document level sentiment analysis with deep memory network ［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2017： 521-526. 10.18653/v1/d17-1054
11	CHEN H， SUN M， TU C， et al. Neural sentiment classification with user and product attention ［C］// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2016： 1650-1659. 10.18653/v1/d16-1171
12	AMPLAYO R K， KIM J， SUNG S， et al. Cold-start aware user and product attention for sentiment classification ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2018： 2535-2544. 10.18653/v1/p18-1236
13	WU Z， DAI X-Y， YIN C， et al. Improving review representations with user attention and product attention for sentiment classification ［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 5989-5996. 10.1609/aaai.v32i1.12054
14	LONG Y， MA M， LU Q， et al. Dual memory network model for biased product review classification ［C］// Proceedings of the 9th Workshop on Computational Approaches to Subjectivity， Sentiment and Social Media Analysis. Stroudsburg， PA： Association for Computational Linguistics， 2018： 140-148. 10.18653/v1/w18-6220
15	MA D， LI S， ZHANG X， et al. Cascading multiway attentions for document-level sentiment classification ［C］// Proceedings of the Eighth International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： Association for Computational Linguistics， 2017： 634-643.

实验环境	具体信息
操作系统	Ubuntu 18.03
内存	DDR4 64 GB
CPU	Intel Xeon Gold 5218
GPU	GeForce RTX 3090 （24 GB）
开发语言	Python 3.9.0
深度学习框架	PyTorch 1.9

实验环境	具体信息
操作系统	Ubuntu 18.03
内存	DDR4 64 GB
CPU	Intel Xeon Gold 5218
GPU	GeForce RTX 3090 （24 GB）
开发语言	Python 3.9.0
深度学习框架	PyTorch 1.9

真实情况	预测结果
真实情况	正例	反例
正例	TP	FN
反例	FP	TN

真实情况	预测结果
真实情况	正例	反例
正例	TP	FN
反例	FP	TN

模型	Yelp2013		Yelp2014		IMDB
模型	Acc/%	RMSE	Acc/%	RMSE	Acc/%	RMSE
UPNN^［9］	59.6	0.784	60.8	0.764	43.5	1.602
UPDMN^［10］	63.9	0.662	61.3	0.720	46.5	1.315
NSC^［11］	65.0	0.692	66.7	0.654	53.3	1.281
HCSC^［12］	65.7	0.660	67.6	0.639	54.2	1.213
HUAPA^［13］	68.3	0.628	68.6	0.626	55.0	1.185
DUPMN^［14］	66.2	0.667	67.6	0.639	53.9	1.279
CMA^［15］	66.4	0.677	67.6	0.637	54.0	1.191
UP-ATL	75.2	0.499	74.5	0.591	56.6	1.104

Text sentiment analysis model based on individual bias information

融合个体偏差信息的文本情感分析模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 15

Related Articles 15

Recommended Articles

Metrics

序号	Yelp2013		Yelp2014		IMDB
序号	Acc/%	RSME	Acc/%	RSME	Acc/%	RSME
Fold 1	74.6	0.503	74.3	0.608	56.5	1.121
Fold 2	74.9	0.526	73.2	0.635	55.9	1.167
Fold 3	75.6	0.491	73.8	0.603	56.3	1.134
Fold 4	74.1	0.548	74.7	0.589	56.7	1.109
Fold 5	74.8	0.535	74.2	0.597	56.6	1.104

[1]	Guolong YUAN, Yujin ZHANG, Yang LIU. Image tampering forensics network based on residual feedback and self-attention [J]. Journal of Computer Applications, 2023, 43(9): 2925-2931.
[2]	Yi ZHANG, Zhenmei WANG. circRNA-disease association prediction by two-stage fusion on graph auto-encoder [J]. Journal of Computer Applications, 2023, 43(6): 1979-1986.
[3]	Jiagao WU, Shiwen ZHANG, Yudong JIANG, Linfeng LIU. Social-interaction GAN for pedestrian trajectory prediction based on state-refinement long short-term memory and attention mechanism [J]. Journal of Computer Applications, 2023, 43(5): 1565-1570.
[4]	Hao SUN, Jian CAO, Haisheng LI, Dianhui MAO. Session-based recommendation model based on enhanced capsule network [J]. Journal of Computer Applications, 2023, 43(4): 1043-1049.
[5]	Haiyu YANG, Wenpu GUO, Kai KANG. Signal modulation recognition method based on convolutional long short-term deep neural network [J]. Journal of Computer Applications, 2023, 43(4): 1318-1322.
[6]	Chunyong YIN, Liwen ZHOU. Unsupervised time series anomaly detection model based on re-encoding [J]. Journal of Computer Applications, 2023, 43(3): 804-811.
[7]	Xiaofei JI, Kexin ZHANG, Lirong TANG. Book spine segmentation algorithm based on improved DeepLabv3+ network [J]. Journal of Computer Applications, 2023, 43(12): 3927-3932.
[8]	Bennian YU, Yongzhao ZHAN, Qirong MAO, Wenlong DONG, Honglin LIU. Double complex convolution and attention aggregating recurrent network for speech enhancement [J]. Journal of Computer Applications, 2023, 43(10): 3217-3224.
[9]	LIU Hui, MA Xiang, ZHANG Linyu, HE Rujin. Aspect-based sentiment analysis model integrating match-LSTM network and grammatical distance [J]. Journal of Computer Applications, 2023, 43(1): 45-50.
[10]	ZHANG Jun, WU Pengli, SHI Lukui, SHI Jin, PAN Bin. Deep learning model for multi-station temperature prediction combined with MOD11A1 and surface meteorological station data [J]. Journal of Computer Applications, 2023, 43(1): 321-328.
[11]	Yinglü XUAN, Yuan WAN, Jiahui CHEN. Time series classification by LSTM based on multi-scale convolution and attention mechanism [J]. Journal of Computer Applications, 2022, 42(8): 2343-2352.
[12]	Yayao ZUO, Haoyu CHEN, Zhiran CHEN, Jiawei HONG, Kun CHEN. Named entity recognition method combining multiple semantic features [J]. Journal of Computer Applications, 2022, 42(7): 2001-2008.
[13]	Wei REN, Hexiang BAI. Multi-label image classification method based on global and local label relationship [J]. Journal of Computer Applications, 2022, 42(5): 1383-1390.
[14]	Yifei WANG, Lei YU, Fei TENG, Jiayu SONG, Yue YUAN. Resource load prediction model based on long-short time series feature fusion [J]. Journal of Computer Applications, 2022, 42(5): 1508-1515.
[15]	Zhihua LIU, Wenjie CHEN, Aibin CHEN. Homologous spectrogram feature fusion with self-attention mechanism for bird sound classification [J]. Journal of Computer Applications, 2022, 42(4): 1260-1268.

模型	Yelp2013		Yelp2014		IMDB
模型	Acc/%	RMSE	Acc/%	RMSE	Acc/%	RMSE
UP-ATL	75.2	0.499	74.5	0.591	56.6	1.104
w/o Bi-LSTM	73.5（-1.7）	0.538	72.2（-2.3）	0.607	52.9（-3.7）	1.298
w/o Transformer	70.4（-4.8）	0.554	70.9（-3.6）	0.539	56.4（-0.2）	1.157
w/o Bi-LSTM+Transformer	67.8（-7.4）	0.638	69.5（-5.0）	0.606	55.0（-1.6）	1.206
w/o UP-Attention	72.1（-3.1）	0.542	71.7（-2.8）	0.583	55.5（-1.1）	1.185

模型	Yelp2013		Yelp2014		IMDB
模型	Acc/%	RMSE	Acc/%	RMSE	Acc/%	RMSE
UP-ATL	75.2	0.499	74.5	0.591	56.6	1.104
w/o Bi-LSTM	73.5（-1.7）	0.538	72.2（-2.3）	0.607	52.9（-3.7）	1.298
w/o Transformer	70.4（-4.8）	0.554	70.9（-3.6）	0.539	56.4（-0.2）	1.157
w/o Bi-LSTM+Transformer	67.8（-7.4）	0.638	69.5（-5.0）	0.606	55.0（-1.6）	1.206
w/o UP-Attention	72.1（-3.1）	0.542	71.7（-2.8）	0.583	55.5（-1.1）	1.185