Empathy prediction from texts based on transfer learning

doi:10.11772/j.issn.1001-9081.2021091632

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (11): 3603-3609.DOI: 10.11772/j.issn.1001-9081.2021091632

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Empathy prediction from texts based on transfer learning

Chenguang LI¹, Bo ZHANG², Qian ZHAO², Xiaoping CHEN¹, Xingfu WANG¹()

^1.College of Computer Science and Technology，University of Science and Technology of China，Hefei Anhui 230026，China
^2.State Grid Anhui Electric Power Company Limited，Hefei Anhui 230022，China

Received:2021-09-15 Revised:2022-01-17 Accepted:2022-01-28 Online:2022-11-14 Published:2022-11-10
Contact: Xingfu WANG
About author:LI Chenguang， born in 1999， M. S. candidate. His research interests include emotion recognition， natural language processing.
ZHANG Bo， born in 1966， M. S.， senior engineer. His research interests include power marketing service management.
ZHAO Qian， born in 1976， M. S.， senior engineer. His research interests include power marketing service management.
CHEN Xiaoping， born in 1955， Ph. D.， professor. His research interests include agent formal modeling， multi‑robot system.
WANG Xingfu， born in 1965， Ph. D.， associate professor. His research interests include natural language processing， emotional analysis.
Supported by:
National Natural Science Foundation of China(92048301);Science and Technology Project of Anhui Electric Power Company Limited(52120018004x)

基于迁移学习的文本共情预测

李晨光¹, 张波², 赵骞², 陈小平¹, 王行甫¹()

^1.中国科学技术大学计算机科学与技术学院，合肥 230026
^2.国网安徽省电力有限公司，合肥 230022

通讯作者: 王行甫
作者简介:李晨光（1999—），男，河南许昌人，硕士研究生，主要研究方向：情感识别、自然语言处理
张波（1966—），男，安徽淮南人，高级工程师，硕士，主要研究方向：电力营销服务管理
赵骞（1976—），男，安徽合肥人，高级工程师，硕士，主要研究方向：电力营销服务管理
陈小平（1955—），男，重庆人，教授，博士，主要研究方向：智能体形式化建模、多机器人系统
王行甫（1965—），男，安徽合肥人，副教授，博士，主要研究方向：自然语言处理、情感分析。cg0808@mail.ustc.edu.cn
基金资助:
国家自然科学基金资助项目(92048301);安徽省电力有限公司科技项目(52120018004x)

Abstract

Abstract:

Empathy prediction from texts achieves little progress due to the lack of sufficient labeled data， while the related task of text sentiment polarity classification has a large number of labeled samples. Since there is a strong correlation between empathy prediction and polarity classification， a transfer learning?based text empathy prediction method was proposed. Transferable public features were learned from the sentiment polarity classification task to assist text empathy prediction task. Firstly， a dynamic weighted fusion of public and private features between two tasks was performed through an attention mechanism. Secondly， in order to eliminate domain differences in datasets between two tasks， an adversarial learning strategy was used to distinguish the domain?unique features from the domain?public features between two tasks. Finally， a Hinge?loss constraint strategy was proposed to make common features be generic for different target labels and private features be unique to different target labels. Experimental results on two benchmark datasets show that compared to the comparison transfer learning methods， the proposed method has higher Pearson Correlation Coefficient （PCC） and coefficient of determination （R²）， and has lower Mean?Square Error （MSE）， which fully demonstrates the effectiveness of the proposed method.

Key words: transfer learning, text empathy prediction, text sentiment polarity classification, Nature Language Processing (NLP), deep learning

摘要：

由于缺乏足够的训练数据，文本共情预测的进展一直都较为缓慢；而与之相关的文本情感极性分类任务则存在大量有标签的训练样本。由于文本共情预测与文本情感极性分类两个任务间存在较大相关性，因此提出了一种基于迁移学习的文本共情预测方法，该方法可从情感极性分类任务中学习到可迁移的公共特征，并通过学习到的公共特征辅助文本共情预测任务。首先通过一个注意力机制对两个任务间的公私有特征进行动态加权融合；其次为了消除两个任务间的数据集领域差异，通过一种对抗学习策略来区分两个任务间的领域独有特征与领域公共特征；最后提出了一种Hinge?loss约束策略，使共同特征对不同的目标标签具有通用性，而私有特征对不同的目标标签具有独有性。在两个基准数据集上的实验结果表明，相较于对比的迁移学习方法，所提方法的皮尔逊相关系数（PCC）和决定系数（R²）更高，均方误差（MSE）更小，充分说明了所提方法的有效性。

关键词: 迁移学习, 文本共情预测, 文本情感极性分类, 自然语言处理, 深度学习

CLC Number:

TP391.1

Chenguang LI, Bo ZHANG, Qian ZHAO, Xiaoping CHEN, Xingfu WANG. Empathy prediction from texts based on transfer learning[J]. Journal of Computer Applications, 2022, 42(11): 3603-3609.

李晨光, 张波, 赵骞, 陈小平, 王行甫. 基于迁移学习的文本共情预测[J]. 《计算机应用》唯一官方网站, 2022, 42(11): 3603-3609.

Figures/Tables 7

References 27

1	BELLET P S， MALONEY M J. The importance of empathy as an interviewing skill in medicine［J］. Journal of the American Medical Association， 1991， 266（13）： 1831-1832. 10.1001/jama.266.13.1831
2	BATSON C D， FULTZ J， SCHOENRADE P A. Distress and empathy： two qualitatively distinct vicarious emotions with different motivational consequences［J］. Journal of Personality， 1987， 55（1）： 19-39. 10.1111/j.1467-6494.1987.tb00426.x
3	BASCH M F. Empathic understanding： a review of the concept and some theoretical considerations［J］. Journal of the American Psychoanalytic Association， 1983， 31（1）： 101-126. 10.1177/000306518303100104
4	SOBER E， WILSON D S. Summary of： ‘Unto others： the evolution and psychology of unselfish behavior’［J］. Journal of Consciousness Studies， 2000， 7（1/2）： 185-206.
5	FUNG P， DEY A， SIDDIQUE F B， et al. Zara the supergirl： an empathetic personality recognition system［C］// Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics： Demonstrations. Stroudsburg， PA： Association for Computational Linguistics， 2016： 87-91. 10.18653/v1/n16-3018
6	ALAM F， DANIELI M， RICCARDI G. Annotating and modeling empathy in spoken conversations［J］. Computer Speech & Language， 2018， 50： 40-61. 10.1016/j.csl.2017.12.003
7	MAJUMDER N， HONG P， PENG S， et al. MIME： MIMicking Emotions for empathetic response generation ［EB/OL］. ［2021-04-28］. . 10.18653/v1/2020.emnlp-main.721
8	BUECHEL S， BUFFONE A， SLAFF B， et al. Modeling empathy and distress in reaction to news stories［EB/OL］. ［2021-06-15］. . 10.18653/v1/d18-1507
9	ZHOU N， JURGENS D. Condolences and empathy in online communities［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2020： 609-626. 10.18653/v1/2020.emnlp-main.45
10	SHARMA A， MINER A S， ATKINS D C， et al. A computational approach to understanding empathy expressed in text‑based mental health support. ［EB/OL］. ［2021-05-09］. . 10.18653/v1/2020.emnlp-main.425
11	PANG B， LEE L. Seeing stars： exploiting class relationships for sentiment categorization with respect to rating scales［C］// Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2005： 115-124. 10.3115/1219840.1219855
12	ROSENTHAL S， FARRA N， NAKOV P. SemEval‑2017 task 4： sentiment analysis in Twitter［C］// Proceedings of the 11th International Workshop on Semantic Evaluation. Stroudsburg， PA： Association for Computational Linguistics， 2017： 502-518. 10.18653/v1/s17-2088
13	BHATT H S， ROY S， RAJKUMAR A， et al. Learning transferable feature representations using neural networks［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2019： 4124-4134. 10.18653/v1/p19-1404
14	MAAS A， DALY R E， PHAM P T， et al. Learning word vectors for sentiment analysis［C］// Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： Association for Computational Linguistics， 2011： 142-150.
15	XIAO B， CAN D， GEORGIOU P G， et al. Analyzing the language of therapist empathy in motivational interview based psychotherapy［C］// Proceedings of the 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference. ［S.l.］： PMC， 2012： 6411762. 10.1109/apsipa31516.2013
16	KHANPOUR H， CARAGEA C， BIYANI P. Identifying empathetic messages in online health communities［C］// Proceedings of the Eighth International Joint Conference on Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2017， 2： 246-251. 10.1609/aaai.v32i1.12170
17	ZHOU K， AIELLO L M， SCEPANOVIC S， et al. The language of situational empathy［J］. Proceedings of the ACM on Human‑ Computer Interaction， 2021， 5（CSCW1）： Article No. 13. 10.1145/3449087
18	DREDZE M， KULESZA A， CRAMMER K. Multi‑domain learning by confidence‑weighted parameter combination［J］. Machine Learning， 2010， 79（1）： 123-149. 10.1007/s10994-009-5148-0
19	PAN S J， YANG Q. A survey on transfer learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2009， 22（10）： 1345-1359. 10.1109/tkde.2009.191
20	HUANG J， GRETTON A， BORGWARDT K， et al. Correcting sample selection bias by unlabeled data［C］// Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge， MA： MIT Press， 2006： 601-608. 10.7551/mitpress/7503.003.0080
21	SUGIYAMA M， SUZUKI T， NAKAJIMA S， et al. Direct importance estimation for covariate shift adaptation［J］. Annals of the Institute of Statistical Mathematics， 2008， 60（4）： 699-746. 10.1007/s10463-008-0197-x
22	MALMI E， SEVERYN A， ROTHE S. Unsupervised text style transfer with padded masked language models［EB/OL］.［2021-06-28］. . 10.18653/v1/2020.emnlp-main.699
23	ZHOU J T， ZHANG H， JIN D， et al. Dual adversarial neural transfer for low‑resource named entity recognition［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2019： 3461-3471. 10.18653/v1/p19-1336
24	CAO P， CHEN Y， LIU K， et al. Adversarial transfer learning for Chinese named entity recognition with self‑attention mechanism［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2018： 182-192. 10.18653/v1/d18-1017
25	GRAVES A， FERNÁNDEZ S， SCHMIDHUBER J. Bidirectional LSTM networks for improved phoneme classification and recognition［C］// Proceedings of the 2005 International Conference on Artificial Neural Networks， LNTCS 3697. Berlin： Springer， 2005： 799-804.
26	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre‑training of deep bidirectional transformers for language understanding. ［EB/OL］. ［2021-09-01］. . 10.18653/v1/n18-2
27	KINGMA D P， BA J AND. Adam： a method for stochastic optimization. ［EB/OL］. ［2021-06-08］. .

样例	共情	极性
I am so happy that more people will undergo the procedure that can save their lives	高	正
I really hate ISIS. They must be destroyed so that they won’t hurt another soul.	高	负
This sounds worrying， but nothing critical， everyone has their own misfortunes.	低	负

样例	共情	极性
I am so happy that more people will undergo the procedure that can save their lives	高	正
I really hate ISIS. They must be destroyed so that they won’t hurt another soul.	高	负
This sounds worrying， but nothing critical， everyone has their own misfortunes.	低	负

模型		Buechel共情数据集				Zhou共情数据集
		SemEval		IMDB		SemEval		IMDB
		PCC‑EC	PCC‑PD	PCC‑EC	PCC‑PD	MSE	R²	MSE	R²
BiLSTM+AL+HN+AT	基线	0.441	0.474	0.431	0.454	0.475	0.169	0.484	0.136
	-AL	0.434	0.460	0.421	0.442	0.508	0.115	0.497	0.115
	-HN	0.435	0.463	0.427	0.452	0.496	0.134	0.489	0.128
	-AT	0.434	0.469	0.428	0.451	0.488	0.152	0.491	0.124
BERT+AL+HN+AT	-AL	0.512	0.523	0.503	0.497	0.423	0.310	0.436	0.283
	-HN	0.488	0.481	0.479	0.474	0.442	0.282	0.459	0.247
	-AT	0.498	0.502	0.483	0.492	0.437	0.280	0.448	0.258
	-AL	0.503	0.510	0.487	0.482	0.430	0.294	0.442	0.271

模型		Buechel共情数据集				Zhou共情数据集
		SemEval		IMDB		SemEval		IMDB
		PCC‑EC	PCC‑PD	PCC‑EC	PCC‑PD	MSE	R²	MSE	R²
BiLSTM+AL+HN+AT	基线	0.441	0.474	0.431	0.454	0.475	0.169	0.484	0.136
	-AL	0.434	0.460	0.421	0.442	0.508	0.115	0.497	0.115
	-HN	0.435	0.463	0.427	0.452	0.496	0.134	0.489	0.128
	-AT	0.434	0.469	0.428	0.451	0.488	0.152	0.491	0.124
BERT+AL+HN+AT	-AL	0.512	0.523	0.503	0.497	0.423	0.310	0.436	0.283
	-HN	0.488	0.481	0.479	0.474	0.442	0.282	0.459	0.247
	-AT	0.498	0.502	0.483	0.492	0.437	0.280	0.448	0.258
	-AL	0.503	0.510	0.487	0.482	0.430	0.294	0.442	0.271

极性数据量/10⁴	Buechel共情数据集		Zhou共情数据集
极性数据量/10⁴	PCC‑EC	PCC‑PD	MSE	R²
0	0.443	0.462	0.461	0.259
1	0.499	0.518	0.434	0.276
2	0.508	0.489	0.430	0.277
3	0.504	0.480	0.439	0.287
4	0.503	0.492	0.442	0.285
5	0.503	0.497	0.436	0.283

Empathy prediction from texts based on transfer learning

基于迁移学习的文本共情预测

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 27

Related Articles 15

Recommended Articles

Metrics

网络	Buechel 共情数据集		网络	Zhou 共情数据集
网络	PCC‑EC	PCC‑PD	网络	MSE	R²
Ridge	0.385	0.410	Random Forest	0.492	0.128
FNN	0.379	0.401	RoBERTa	0.429	0.297
CNN	0.404	0.444	Bi‑LSTM	0.553	0.004
Bi‑LSTM	0.407	0.426	BERT	0.461	0.259
BERT	0.443	0.462

模型	示例1		示例2
模型	EC	PD	EC	PD
Baseline	4.82	5.24	1.85	2.21
本文方法	6.32	6.24	1.65	2.02
Ground Truth	7.00	6.75	1.00	1.00

[1]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[2]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[3]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[4]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[5]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[6]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[7]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[8]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[9]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[10]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[11]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.
[12]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[13]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[14]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.
[15]	Yajuan ZHAO, Fanjun MENG, Xingjian XU. Review of online education learner knowledge tracing [J]. Journal of Computer Applications, 2024, 44(6): 1683-1698.