Sentiment boosting model for emotion recognition in conversation text

doi:10.11772/j.issn.1001-9081.2022010044

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (3): 706-712.DOI: 10.11772/j.issn.1001-9081.2022010044

• Artificial intelligence • Previous Articles

Sentiment boosting model for emotion recognition in conversation text

Yu WANG¹(), Yubo YUAN¹^,², Yi GUO¹^,²^,³, Jiajie ZHANG¹

^1.School of Computer Science and Engineering，East China University of Science and Technology，Shanghai 200237，China
^2.Shanghai Engineering Research Center of Big Data and Internet Audience，Shanghai 200072，China
^3.National Engineering Laboratory for Big Data Distribution and Exchange Technologies （Shanghai Data Exchange），Shanghai 200436，China

Received:2022-01-14 Revised:2022-05-11 Accepted:2022-05-12 Online:2022-09-17 Published:2023-03-10
Contact: Yu WANG
About author:YUAN Yubo， born in 1976， Ph. D.， associate professor. His research interests include machine learning.
GUO Yi， born in 1975， Ph. D.， professor. His research interests include text mining.
ZHANG Jiajie， born in 1999， M. S. candidate. His research interests include graph neural network.
Supported by:
Shanghai Engineering Research Technology Center Project(18DZ2252300)

情感增强的对话文本情绪识别模型

王雨¹(), 袁玉波¹^,², 过弋¹^,²^,³, 张嘉杰¹

^1.华东理工大学信息科学与工程学院, 上海 200237
^2.上海大数据与互联网受众工程技术研究中心, 上海 200072
^3.大数据流通与交易技术国家工程实验室(上海数据交易所), 上海 200436

通讯作者: 王雨
作者简介:王雨（1998—），女，山东泰安人，硕士研究生，主要研究方向：对话情绪识别
袁玉波（1976—），男，云南宣威人，副教授，博士，主要研究方向：机器学习
过弋（1975—），男，江苏无锡人，教授，博士，CCF会员，主要研究方向：文本挖掘
张嘉杰（1999—），男，河南焦作人，硕士研究生，主要研究方向：图神经网络。
基金资助:
上海市工程技术中心项目(18DZ2252300)

Abstract

Abstract:

To address the problems that many existing studies ignore the correlation between interlocutors’ emotions and sentiments， a sentiment boosting model for emotion recognition in conversation text was proposed， namely Sentiment Boosting Graph Neural network （SBGN）. Firstly， themes and dialogue intent were integrated into the text， and the reconstructed text features were extracted by fine-tuning the pre-trained language model. Secondly， a symmetric learning structure for emotion analysis was given， with the reconstructed features fed into a Graph Neural Network （GNN） emotion analysis model and a Bi-directional Long Short-Term Memory （Bi-LSTM） sentiment classification model. Finally， by fusing emotion analysis and sentiment classification models， a new loss function was constructed with sentiment classification loss function as a penalty， and the optimal penalty factor was adjusted and obtained by learning. Experimental results on public dataset DailyDialog show that SBGN model improves 16.62 percentage points compared with Dialogue Graph Convolutional Network （DialogueGCN） model， and improves 14.81 percentage points compared with the state-of-art model Directed Acyclic Graph-Emotion Recognition from Conversation （DAG-ERC） in micro-average F1. It can be seen that SBGN model can effectively improve the performance of emotion analysis in dialogue system.

Key words: Emotion Recognition in Conversations (ERC), sentiment classification, theme induction, Graph Neural Network (GNN), Bi-directional Long Short-Term Memory (Bi-LSTM)

摘要：

针对现有的许多研究忽略了说话人的情绪和情感的相关性的问题，提出一种情感增强的图网络对话文本情绪识别模型——SBGN。首先，将主题和对话意图融入文本，并微调预训练语言模型RoBERTa以提取重构的文本特征；其次，给出情绪分析的对称学习结构，将重构特征分别输入图神经网络（GNN）情绪分析模型和双向长短时记忆（Bi-LSTM）情感分类模型；最后，融合情绪分析和情感分类模型，将情感分类的损失函数作为惩罚以构建新的损失函数，并通过学习调节得到最优的惩罚因子。在公开数据集DailyDialog上的实验结果表明，相较于DialogueGCN模型与目前最先进的DAG-ERC模型，SBGN模型的微平均F1分别提高16.62与14.81个百分点。可见，SBGN模型能有效提高对话系统情绪分析的性能。

关键词: 对话情绪识别, 情感分类, 主题诱导, 图神经网络, 双向长短时记忆

CLC Number:

TP391.1

Yu WANG, Yubo YUAN, Yi GUO, Jiajie ZHANG. Sentiment boosting model for emotion recognition in conversation text[J]. Journal of Computer Applications, 2023, 43(3): 706-712.

王雨, 袁玉波, 过弋, 张嘉杰. 情感增强的对话文本情绪识别模型[J]. 《计算机应用》唯一官方网站, 2023, 43(3): 706-712.

Figures/Tables 7

References 18

1	AKHTAR M S， CHAUHAN D S， GHOSAL D， et al. Multi-task learning for multi-modal emotion recognition and sentiment analysis［C］// Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： ACL， 2019： 370-379. 10.18653/v1/n19-1034
2	KUMAR A， EKBAL A， KAWAHR D， et al. Emotion helps sentiment： a multi-task model for sentiment and emotion recognition ［C］// Proceedings of the 2019 International Joint Conference on Neural Networks. Piscataway： IEEE， 2019： 1-8. 10.1109/ijcnn.2019.8852352
3	LIU Y H， OTT M， GOYAL N， et al. RoBERTa： A robustly optimized bert pretraining approach ［EB/OL］（2019-07-26）［2021-01-05］. .
4	LI Y R， SU H， SHEN X Y， et al. DailyDialog： a manually labelled multi-turn dialogue dataset ［C］// Proceedings of the 8th International Joint Conference on Natural Language Processing. ［S.l.］： Asian Federation of Natural Language Processing， 2017： 986-995.
5	HERZIG J， FEIGENBLAT G， SHMUELI-SCHEUER M， et al. Classifying emotions in customer support dialogues in social media［C］//Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg， PA： ACL， 2016： 64-73. 10.18653/v1/w16-3609
6	KIM Y. Convolutional neural networks for sentence classification ［C］// Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2014： 1746-1751. 10.3115/v1/d14-1181
7	郝志峰，黄浩，蔡瑞初，等.基于多特征融合与双向RNN的细粒度意见分析［J］.计算机工程，2018，44（7）：199-204，211.
	HAO Z F， HUANG H， CAI R C， et al. Fine-grained opinion analysis based on multi-feature fusion and bidirectional RNN［J］. Computer Engineering， 2018， 44（7）： 199-204， 211.
8	HAZARIKA D， PORIA S， ZADEH A， et al. Conversational memory network for emotion recognition in dyadic dialogue videos［C］// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long Papers）. Stroudsburg， PA： ACL， 2018： 2122-2132. 10.18653/v1/n18-1193
9	HAZARIKA D， PORIA S， MIHALCEA R， et al. ICON： interactive conversational memory network for multimodal emotion detection ［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2018： 2594-2604. 10.18653/v1/d18-1280
10	MAJUMDER N， PORIA S， HAZARIKA D， et al. DialogueRNN： an attentive RNN for emotion detection in conversations ［C］// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2019， 33： 6818-6825. 10.1609/aaai.v33i01.33016818
11	SHEN W Z， CHEN J Q， QUAN X J， et al. DialogXL： all-in-one XLNet for multi-party conversation emotion recognition ［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2021， 35（15）： 13789-13797. 10.1609/aaai.v35i15.17625
12	ZHONG P X， WANG D， MIAO C. Knowledge-enriched transformer for emotion detection in textual conversations［C］//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2019： 165-176. 10.18653/v1/d19-1016
13	BHAT V， YADAV A， YADAV S， et al. AdCOFE： Advanced contextual feature extraction in conversations for emotion classification［J］. PeerJ Computer Science， 2021， 7： No.786. 10.7717/peerj-cs.786
14	GHOSAL D， MAJUMDER N， PORIA S， et al. DialogueGCN： a graph convolutional neural network for emotion recognition in conversation［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2019： 154-164. 10.18653/v1/d19-1015
15	ISHIWATARI T， YASUDA Y， MIYAZAKI T， et al. Relation-aware graph attention networks with relational position encodings for emotion recognition in conversations［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2020： 7360-7370. 10.18653/v1/2020.emnlp-main.597
16	彭韬，杨亮，桑钟屹，等.基于异构二部图的对话情感分析［J］.中文信息学报，2021，35（11）：135-142. 10.3969/j.issn.1003-0077.2021.11.014
	PENG T， YANG L， SANG Z Y， et al. Conversation sentiment analysis based on heterogeneous bipartite graphs［J］. Journal of Chinese Information Processing， 2021，35（11）：135-142. 10.3969/j.issn.1003-0077.2021.11.014
17	SHEN W Z， WU S Y， YANG Y Y， et al. Directed acyclic graph network for conversational emotion recognition ［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2020： 1551-1560. 10.18653/v1/2021.acl-long.123
18	QIN L B， CHE W X， LI Y M， et al. DCR-Net： a deep co-interactive relation network for joint dialog act recognition and sentiment classification ［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 8665-8672. 10.1609/aaai.v34i05.6391

说话人	话语	情绪标签	情感标签
A	你喜欢它吗？你真的喜欢吗？	惊讶	积极
B	哦，耶！	高兴	积极
A	袋鼠那一幕怎么样？	中性	中性
B	我很惊讶地在一部世界大战史诗中看到了袋鼠。	惊讶	积极
A	你睡着了！	生气	消极
B	别走，对不起。	悲伤	消极

说话人	话语	情绪标签	情感标签
A	你喜欢它吗？你真的喜欢吗？	惊讶	积极
B	哦，耶！	高兴	积极
A	袋鼠那一幕怎么样？	中性	中性
B	我很惊讶地在一部世界大战史诗中看到了袋鼠。	惊讶	积极
A	你睡着了！	生气	消极
B	别走，对不起。	悲伤	消极

模型	调节因子	微平均F1	宏平均F1
MLP	—	58.35	61.49
Bi-LSTM	—	58.57	61.39
Joint MLP	0.9	58.88	66.94
Joint LSTM	0.8	58.99	67.41
Joint Bi-LSTM	0.8	59.22	67.51
DialogueGCN	—	57.52	—
RGAT	—	59.02	—
DAG-ERC	—	59.33	—
SBGN	0.4	74.14	79.55

模型	调节因子	微平均F1	宏平均F1
MLP	—	58.35	61.49
Bi-LSTM	—	58.57	61.39
Joint MLP	0.9	58.88	66.94
Joint LSTM	0.8	58.99	67.41
Joint Bi-LSTM	0.8	59.22	67.51
DialogueGCN	—	57.52	—
RGAT	—	59.02	—
DAG-ERC	—	59.33	—
SBGN	0.4	74.14	79.55

模型名称	微平均F1
模型名称	基础特征	增强特征
MLP	58.35	71.94
LSTM	58.45	70.83
Bi-LSTM	58.57	72.34
Joint MLP	58.88	72.71
Joint LSTM	58.99	72.56
Joint Bi-LSTM	59.22	72.94
DAG-ERC	59.33	73.65
SBGN	59.45	74.14

Sentiment boosting model for emotion recognition in conversation text

情感增强的对话文本情绪识别模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 18

Related Articles 15

Recommended Articles

Metrics

[1]	Lubao LI, Tian CHEN, Fuji REN, Beibei LUO. Bimodal emotion recognition method based on graph neural network and attention [J]. Journal of Computer Applications, 2023, 43(3): 700-705.
[2]	Jun ZHANG, Pengli WU, Lukui SHI, Jin SHI, Bin PAN. Deep learning model for multi-station temperature prediction combined with MOD11A1 and surface meteorological station data [J]. Journal of Computer Applications, 2023, 43(1): 321-328.
[3]	Meng YU, Wentao HE, Xuchuan ZHOU, Mengtian CUI, Keqi WU, Wenjie ZHOU. Review of recommendation system [J]. Journal of Computer Applications, 2022, 42(6): 1898-1913.
[4]	Jiafan ZHOU, Yuefeng DU, Baoyan SONG, Xiaoguang LI, Azhu ZHAO, Xujie XIAO. MOOC video recommendation method based on meta-path attention mechanism [J]. Journal of Computer Applications, 2022, 42(6): 1808-1813.
[5]	Xiaohan LI, Huading JIA, Xue CHENG, Taiyong LI. Stock market volatility prediction method based on improved genetic algorithm and graph neural network [J]. Journal of Computer Applications, 2022, 42(5): 1624-1633.
[6]	Weichao DANG, Zhiyu YAO, Shangwang BAI, Gaimei GAO, Chunxia LIU. Session recommendation method based on graph model and attention model [J]. Journal of Computer Applications, 2022, 42(11): 3610-3616.
[7]	ZHANG Rong, ZHANG Xianguo. Opinion spam detection based on hierarchical heterogeneous graph attention network [J]. Journal of Computer Applications, 2021, 41(5): 1275-1281.
[8]	Yu DENG, Xiaoyu LI, Jian CUI, Qi LIU. Multi-head attention memory network for short text sentiment classification [J]. Journal of Computer Applications, 2021, 41(11): 3132-3138.
[9]	Junwei REN, Cheng ZENG, Siyu XIAO, Jinxia QIAO, Peng HE. Session-based recommendation model of multi-granular graph neural network [J]. Journal of Computer Applications, 2021, 41(11): 3164-3170.
[10]	YUAN Jingling, DING Yuanyuan, PAN Donghang, LI Lin. Chinese implicit sentiment classification model based on sequence and contextual features [J]. Journal of Computer Applications, 2021, 41(10): 2820-2828.
[11]	LUO Jun, CHEN Lifei. Sentiment classification of incomplete data based on bidirectional encoder representations from transformers [J]. Journal of Computer Applications, 2021, 41(1): 139-144.
[12]	YIN Chunyong, ZHANG Sun. End-to-end adversarial variational Bayes method for short text sentiment classification [J]. Journal of Computer Applications, 2020, 40(9): 2536-2542.
[13]	LIN Zhixing, WANG Like. Network situation prediction method based on deep feature and Seq2Seq model [J]. Journal of Computer Applications, 2020, 40(8): 2241-2247.
[14]	WANG Minrui, GAO Shu, YUAN Ziyong, YUAN Lei. Sequence generation model with dynamic routing for multi-label text classification [J]. Journal of Computer Applications, 2020, 40(7): 1884-1890.
[15]	HU Tiantian, DAN Yabo, HU Jie, LI Xiang, LI Shaobo. News named entity recognition and sentiment classification based on attention-based bi-directional long short-term memory neural network and conditional random field [J]. Journal of Computer Applications, 2020, 40(7): 1879-1883.

模型	运行时间	模型	运行时间
DAG-ERC	49.34	SBGN_Bi-LSTM	64.80
SBGN_MLP	52.86	SBGN	65.86

模型	运行时间	模型	运行时间
DAG-ERC	49.34	SBGN_Bi-LSTM	64.80
SBGN_MLP	52.86	SBGN	65.86