Sentiment classification model of psychological counseling text based on attention over attention mechanism

doi:10.11772/j.issn.1001-9081.2023081168

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (8): 2393-2399.DOI: 10.11772/j.issn.1001-9081.2023081168

• Artificial intelligence • Previous Articles Next Articles

Sentiment classification model of psychological counseling text based on attention over attention mechanism

Yuqing WANG¹^,², Guangli ZHU¹^,²(), Wenjie DUAN¹^,², Shuyu LI¹^,², Ruotong ZHOU¹^,²

^1.School of Computer Science and Engineering，Anhui University of Science and Technology，Huainan Anhui 232001，China
^2.Institute of Artificial Intelligence，Hefei Comprehensive National Science Center，Hefei Anhui 230088，China

Received:2023-08-31 Revised:2023-09-12 Accepted:2023-10-08 Online:2024-08-22 Published:2024-08-10
Contact: Guangli ZHU
About author:WANG Yuqing， born in 2000， M. S. candidate. Her researchinterests include sentiment analysis， data mining.
ZHU Guangli， born in 1971， M. S.， associate professor. Herresearch interests include intelligent information processing， affectivecomputing.
DUAN Wenjie ， born in 2000， M. S. candidate. His researchinterests include sentiment analysis.
LI Shuyu， born in 1999， M. S. candidate. Her research interestsinclude data mining.
ZHOU Ruotong ， born in 2000， M. S. candidate. Her researchinterests include data mining.
Supported by:
This work is partially supported by National Natural ScienceFoundation of China （62076006）； University Synergy InnovationProgram of Anhui Province （GXXT-2021-008）； Graduate InnovationFund Project of Anhui University of Science and Technology（2023cx2124）.

基于交互注意力机制的心理咨询文本情感分类模型

汪雨晴¹^,², 朱广丽¹^,²(), 段文杰¹^,², 李书羽¹^,², 周若彤¹^,²

^1.安徽理工大学计算机科学与工程学院，安徽淮南 232001
^2.合肥综合性国家科学中心人工智能研究院，合肥 230088

通讯作者: 朱广丽
作者简介:汪雨晴（2000—），女，安徽蚌埠人，硕士研究生，CCF会员，主要研究方向：情感分析、数据挖掘
朱广丽（1971—），女，安徽淮南人，副教授，硕士，主要研究方向：智能信息处理、情感计算 glzhu@aust.edu.cn
段文杰（2000—），男，安徽宿州人，硕士研究生，CCF会员，主要研究方向：情感分析
李书羽（1999—），女，安徽铜陵人，硕士研究生，主要研究方向：数据挖掘
周若彤（2000—），女，安徽合肥人，硕士研究生，CCF会员，主要研究方向：数据挖掘。
基金资助:
国家自然科学基金面上项目(62076006);安徽高校协同创新项目(GXXT?2021?008);安徽理工大学研究生创新基金资助项目(2023cx2124)

Abstract

Abstract:

Sentiment classification in psychological counseling scenes aims to obtain the sentiment polarity of the inquirer’s utterance， which can provide support for establishing psychological counseling Artificial Intelligence （AI） assistants. Existing methods obtain the sentiment polarity of text through contextual information， failing to consider the sentiment transmission between the current sentence and the forward neighbor sentences in the dialogue record. To address the issue， a model for sentiment classification of psychological counseling text was proposed based on Attention Over Attention （AOA） mechanism. Historical sentiment words were assigned weights by temporal sequence， which improved the accuracy of sentiment classification for psychological counseling text. In a dialogue， historical sentiment word sequences of both sides were extracted by constructed sentiment lexicon of mental health. Subsequently， the current sentence and two sequences of historical sentiment words were input into the Bidirectional Long Short-Term Memory （BiLSTM） network to get corresponding feature vectors. The Ebbinghaus forgetting curve was used to allocate internal weights to the sequences of historical sentiment words. Both inertia features and interaction features were captured by AOA mechanism. Then， the above two features along with the text features were input into the classification layer， calculating the probability of sentiment polarity. Experimental results on public dataset Emotional First Aid Dataset show that the proposed model improves F1 value by 1.55% compared with Capsule network and Directional Graph Convolutional Network （Caps-DGCN） model. Hence the proposed model can effectively improve the sentiment classification effect of psychological counseling text.

Key words: psychological counseling, sentiment lexicon of mental health, Ebbinghaus forgetting curve, Attention Over Attention (AOA) mechanism, Bidirectional Long Short-Term Memory (BiLSTM) network

摘要：

心理咨询场景下的情感分类旨在获得咨询者话语的情感倾向，为建立心理咨询AI助手提供支持。现有的方法利用语境信息获取文本情感倾向，但未考虑对话记录中当前句与前向近邻句之间的情感传递。针对这一问题，提出一种基于交互注意力（AOA）机制的心理咨询文本情感分类模型，根据时序对历史情感词分配权重，进而提高分类准确率。利用构建的心理健康情感词典分别提取对话双方的历史情感词序列，再将当前句和历史情感词序列输入到双向长短期记忆（BiLSTM）网络获取对应的特征向量，并利用艾宾浩斯遗忘曲线对历史情感词序列分配权重。通过AOA机制获得惯性特征和交互特征，并结合文本特征输入到分类层计算情感倾向概率。在公开数据集Emotional First Aid Dataset上的实验结果表明，相较于Caps-DGCN（Capsule network and Directional Graph Convolutional Network）模型，所提模型的F1值提高了1.55%。可见，所提模型可以有效提升心理咨询文本的情感分类效果。

关键词: 心理咨询, 心理健康情感词典, 艾宾浩斯遗忘曲线, 交互注意力机制, 双向长短期记忆网络

CLC Number:

TP391

Yuqing WANG, Guangli ZHU, Wenjie DUAN, Shuyu LI, Ruotong ZHOU. Sentiment classification model of psychological counseling text based on attention over attention mechanism[J]. Journal of Computer Applications, 2024, 44(8): 2393-2399.

汪雨晴, 朱广丽, 段文杰, 李书羽, 周若彤. 基于交互注意力机制的心理咨询文本情感分类模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2393-2399.

Figures/Tables 9

Fig. 1 Sentiment classification framework of psychological counseling text based on AOA mechanism

Fig. 2 Flow of constructing sentiment lexicon of mental health

Tab. 1 Part of sentiment seed words

情感极性	情感种子词
积极	喜欢、爱、开心、自信、幸福、努力
消极	恶劣、恐惧、困扰、难过、焦虑

Tab. 2 Part of sentiment lexicon of mental health vocabulary

情感

极性

心理健康情感词

积极

帮助、支持、进步、幸福、满足、兴奋、安心、感恩、乐观、自信

消极

沮丧、不安、焦虑、失败、担忧、绝望、孤独、压力、羞愧、憎恨

中性

任务、交流、分享、冷静、理智、平淡、无所谓、自我控制、随便

Fig. 3 Sentiment classification model of psychological counseling text

Fig. 4 Schematic diagram of extracting features based on AOA

Tab. 3 Statistical information of experiment datasets

数据集	心理咨询文本数	咨询者对话总数
训练集	7 462	28 356
验证集	1 066	4 584
测试集	2 132	8 954

Tab. 4 Comparison experiment results

领域	模型	$P$	$R$	$F 1$
文本情感分析领域	BiLSTM	63.18	61.43	62.85
	BiERU-lc	64.67	63.86	64.38
	BE-Att-BiLSTM	65.45	63.94	65.17
	CS-GCN	68.57	65.75	67.91
心理咨询领域	MHA-BCNN	68.13	66.21	67.26
	Caps-DGCN	70.21	68.61	69.48
	本文模型	71.62	68.83	70.56

Tab. 4 Comparison experiment results

领域	模型	$P$	$R$	$F 1$
文本情感分析领域	BiLSTM	63.18	61.43	62.85
	BiERU-lc	64.67	63.86	64.38
	BE-Att-BiLSTM	65.45	63.94	65.17
	CS-GCN	68.57	65.75	67.91
心理咨询领域	MHA-BCNN	68.13	66.21	67.26
	Caps-DGCN	70.21	68.61	69.48
	本文模型	71.62	68.83	70.56

Tab. 5 Ablation experiment results

模型	$P$	$R$	$F 1$
BiLSTM-AOA	64.18	61.64	63.21
BiLSTM-AOA-Lex	66.67	64.86	65.42
BiLSTM-AOA-TEFC	66.89	65.38	66.23
本文模型	71.62	68.83	70.56

Tab. 5 Ablation experiment results

模型	$P$	$R$	$F 1$
BiLSTM-AOA	64.18	61.64	63.21
BiLSTM-AOA-Lex	66.67	64.86	65.42
BiLSTM-AOA-TEFC	66.89	65.38	66.23
本文模型	71.62	68.83	70.56

References 29

1	DHEERAJ K， RAMAKRISHNUDU T. Negative emotions detection on online mental-health related patients texts using the deep learning with MHA-BCNN model［J］. Expert Systems with Applications， 2021， 182： 115265.
2	RATHNAYAKA P， MILLS N， BURNETT D， et al. A mental health chatbot with cognitive skills for personalised behavioural activation and remote health monitoring［J］. Sensors， 2022， 22（10）： 3653.
3	ZHANG Y， TIWARI P， SONG D， et al. Learning interaction dynamics with an interactive LSTM for conversational sentiment analysis［J］. Neural Networks， 2021， 133： 40-56.
4	TANG S， WANG C， XU K， et al. An emotion evolution network for emotion recognition in conversation［C］// Proceedings of the 2022 IEEE 34th International Conference on Tools with Artificial Intelligence. Piscataway： IEEE， 2022： 1231-1238.
5	HAZARIKA D， PORIA S， ZADEH A， et al. Conversational memory network for emotion recognition in dyadic dialogue videos［C］ // Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long Papers）. Stroudsburg： ACL， 2018： 2122-2132.
6	SAXENA P， HUANG Y J， KUROHASHI S. Static and dynamic speaker modeling based on graph neural network for emotion recognition in conversation［C］// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies： Student Research Workshop. Stroudsburg： ACL， 2022： 247-253.
7	ZHANG S， WEI Z， WANG Y， et al. Sentiment analysis of Chinese micro-blog text based on extended sentiment dictionary［J］. Future Generation Computer Systems， 2018， 81： 395-403.
8	GAO Y， SU P， ZHAO H， et al. Research on sentiment dictionary based on sentiment analysis in news domain［C］// Proceedings of the 2021 7th IEEE International Conference on Big Data Security on Cloud， IEEE International Conference on High Performance and Smart Computing， and IEEE International Conference on Intelligent Data and Security. Piscataway： IEEE， 2021： 117-122.
9	LIU H， CHEN X， LIU X. A study of the application of weight distributing method combining sentiment dictionary and TF-IDF for text sentiment analysis［J］. IEEE Access， 2022， 10： 32280-32289.
10	WONGKAR M， ANGDRESEY A. Sentiment analysis using Naive Bayes algorithm of the data crawler： Twitter［C］// Proceedings of the 2019 Fourth International Conference on Informatics and Computing. Piscataway： IEEE， 2019： 1-5.
11	LU K， WU J. Sentiment analysis of film review texts based on sentiment dictionary and SVM［C］// Proceedings of the 2019 3rd International Conference on Innovation in Artificial Intelligence. New York： ACM， 2019： 73-77.
12	HERZIG J， FEIGENBLAT G， SHMUELI-SCHEUER M， et al. Classifying emotions in customer support dialogues in social media［C］// Proceedings of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg： ACL， 2016： 64-73.
13	衡红军，徐天宝.基于多尺度卷积和门控机制的注意力情感分析模型［J］.计算机应用， 2022， 42（9）： 2674-2679.
	HENG H J， XU T B. Attention sentiment analysis model based on multi-scale convolution and gating mechanism［J］. Journal of Computer Applications， 2021， 41（9）： 999-1005.
14	BASIRI M E， NEMATI S， ABDAR M， et al. ABCDM： an attention-based bidirectional CNN-RNN deep model for sentiment analysis［J］. Future Generation Computer Systems， 2021， 115： 279-294.
15	BEHERA R K， JENA M， RATH S K， et al. Co-LSTM： convolutional LSTM model for sentiment analysis in social big data［J］. Information Processing & Management， 2021， 58（1）： 102435.
16	陈晨，周夏冰，王中卿，等.基于多方注意力建模的对话情感分类［J］.中文信息学报，2022， 36（12）： 173-181.
	CHEN C， ZHOU X B， WANG Z Q， et al. Dialog sentiment analysis with multi-party attention ［J］. Journal of Chinese Information Processing， 2022， 36（12）： 173-181.
17	张军，张丽，沈凡凡，等.RoBERTa融合BiLSTM及注意力机制的隐式情感分析［J］.计算机工程与应用，2022，58（23）：142-150.
	ZHANG J， ZHANG L， SHEN F F， et al. lmplicit sentiment analysis based on RoBERTa fused with BiLSTM and attention mechanism［J］. Computer Engineering and Applications， 2022，58（23）：142-150.
18	YANG J， YANG J. Aspect based sentiment analysis with self-attention and gated convolutional networks［C］// Proceedings of the 2020 IEEE 11th International Conference on Software Engineering and Service Science. Piscataway： IEEE， 2020： 146-149.
19	廖健，张楷，王素格，等. 基于异构用户知识融合的隐式情感分析研究［C］// 第二十一届中国计算语言大会论文集. 北京：中国中文信息学会， 2022： 523-534.
	LIAO J， ZHANG K， WANG S G， et al. Research on implicit sentiment analysis based on heterogeneous user knowledge fusion ［C］// Proceedings of the 21st Chinese National Conference on Computational Linguistics. Beijing： Chinese Information Processing Society of China， 2022： 523-534.
20	KIM K， PARK S. AOBERT： all-modalities-in-one BERT for multimodal sentiment analysis［J］. Information Fusion， 2023， 92： 37-45.
21	LI W， SHAO W， JI S， et al. BiERU： bidirectional emotional recurrent unit for conversational sentiment analysis［J］. Neurocomputing， 2022， 467： 73-82.
22	孟洁，李妍，赵迪，等.基于图卷积网络的客服对话情感分析［J］.计算机系统应用， 2022， 31（5）： 147-156.
	MENG J， LI Y， ZHAO D， et al. Sentiment polarity analysis of customer dialogues based on graph convolutional network［J］. Computer Systems & Applications， 2022， 31（5）： 147-156.
23	ZHANG Y， TIWARI P， SONG D， et al. Learning interaction dynamics with an interactive LSTM for conversational sentiment analysis［J］. Neural Networks， 2021， 133： 40-56.
24	HU T， ZHANG D， WANG J， et al. Relation between emotion regulation and mental health： a meta-analysis review［J］. Psychological Reports， 2014， 114（2）： 341-362.
25	STEBNICKI M A. Empathy fatigue： healing the mind， body， and spirit of professional counselors［J］. American Journal of Psychiatric Rehabilitation， 2007， 10（4）： 317-338.
26	İME Y， ÜMMET D. The effects of cognitive behavioral psychological group counseling program on the psychological resilience and emotional flexibility of adolescents［J］. Current Psychology， 2023， 43： 8885-8895.
27	SHEN Y， HUANG Y， JIA R， et al. Research on user’s mental health based on comment text［C］// Proceedings of the 17th CCF Conference on Computer Supported Cooperative Work and Social Computing. Singapore： Springer， 2022： 298-310.
28	PRIYA P， FIRDAUS M， EKBAL A. A multi-task learning framework for politeness and emotion detection in dialogues for mental health counselling and legal aid［J］. Expert Systems with Applications， 2023， 224： 120025.
29	JBENE M， RAIF M， TIGANI S， et al. User sentiment analysis in conversational systems based on augmentation and attention-based BiLSTM［J］. Procedia Computer Science， 2022， 207： 4106-4112.

[1]	Xudong HOU, Fei TENG, Yi ZHANG. Medical named entity recognition model based on deep auto-encoding [J]. Journal of Computer Applications, 2022, 42(9): 2686-2692.
[2]	Jie HU, Yan HU, Mengchi LIU, Yan ZHANG. Chinese named entity recognition based on knowledge base entity enhanced BERT model [J]. Journal of Computer Applications, 2022, 42(9): 2680-2685.
[3]	Yi ZHANG, Shuangsheng WANG, Bin HE, Peiming YE, Keqiang LI. Named entity recognition method of elementary mathematical text based on BERT [J]. Journal of Computer Applications, 2022, 42(2): 433-439.
[4]	Lanlan ZENG, Yisong WANG, Panfeng CHEN. Named entity recognition based on BERT and joint learning for judgment documents [J]. Journal of Computer Applications, 2022, 42(10): 3011-3017.
[5]	XU Li, LI Jianhua. Biomedical named entity recognition with graph network based on syntactic dependency parsing [J]. Journal of Computer Applications, 2021, 41(2): 357-362.
[6]	DONG Yongfeng, LIU Chao, WANG Liqin, LI Yingshuang. Relationship reasoning method combining multi-hop relationship path information [J]. Journal of Computer Applications, 2021, 41(10): 2799-2805.
[7]	. Computational trust evaluation model based on temporal sequential marker [J]. Journal of Computer Applications, 2010, 30(3): 663-667.

Sentiment classification model of psychological counseling text based on attention over attention mechanism

基于交互注意力机制的心理咨询文本情感分类模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 29

Related Articles 7

Recommended Articles

Metrics