基于注意力机制的主题扩展情感对话生成

doi:10.11772/j.issn.1001-9081.2020071063

计算机应用 ›› 2021, Vol. 41 ›› Issue (4): 1078-1083.DOI: 10.11772/j.issn.1001-9081.2020071063

所属专题：人工智能

基于注意力机制的主题扩展情感对话生成

杨丰瑞^1,2,3, 霍娜^1,2, 张许红^1,2, 韦巍^1,2

1. 重庆邮电大学通信与信息工程学院, 重庆 400065;
2. 重庆邮电大学通信新技术应用研究中心, 重庆 400065;
3. 重庆重邮信科(集团)股份有限公司, 重庆 401121

收稿日期:2020-07-20 修回日期:2020-09-13 出版日期:2021-04-10 发布日期:2020-10-19
通讯作者: 霍娜
作者简介:杨丰瑞（1963—），男，重庆人，教授，博士，主要研究方向：移动通信、大数据；霍娜（1996—），女，河南信阳人，硕士研究生，主要研究方向：大数据、对话系统；张许红（1994—），女，河南洛阳人，硕士研究生，主要研究方向：大数据、情感分析；韦巍（1996—），男，江苏苏州人，硕士研究生，主要研究方向：移动通信、无线传感。

Topic-expanded emotional conversation generation based on attention mechanism

YANG Fengrui^1,2,3, HUO Na^1,2, ZHANG Xuhong^1,2, WEI Wei^1,2

1. School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
2. Research Center of New Telecommunication Technology Applications, Chongqing University of Posts and Telecommunications, Chongqing 400065, China;
3. Chongqing Chongyou Information Technology (Group) Company Limited, Chongqing 401121, China

Received:2020-07-20 Revised:2020-09-13 Online:2021-04-10 Published:2020-10-19

摘要/Abstract

摘要： 越来越多的研究开始聚焦于情感对话生成，然而现有的研究往往只关注情感因素，却忽视了对话中主题的相关性和多样性以及与主题密切相关的情感倾向，这可能导致生成响应的质量下降。因此提出一种融合主题信息和情感因素的主题扩展情感对话生成模型。该模型首先将对话上下文进行全局编码，引入主题模型以获得全局主题词，并使用外部情感词典获得全局情感词；其次在融合模块里利用语义相似度扩展主题词，并利用依存句法分析提取与主题相关的情感词；最后将上下文、主题词和情感词输入到一个基于注意力机制的解码器中，促使解码器生成主题相关的情感响应。实验结果表明，该模型能生成内容丰富且情感相关的回答。相较于主题增强情感对话生成模型（TE-ECG），所提出的模型在unigram多样性（distinct-1）和bigram多样性（distinct-2）上平均提高了16.3%和15.4%；相较于基于注意力机制的序列到序列模型（Seq2SeqA），所提出的模型在unigram多样性（distinct-1）和bigram多样性（distinct-2）上平均提高了26.7%和28.7%。

关键词: 情感对话生成, 融合模块, 主题模型, 序列到序列, 注意力机制

Abstract: More and more studies begin to focus on emotional conversation generation. However, the existing studies tend to focus only on emotional factors and ignore the relevance and diversity of topics in dialogues, as well as the emotional tendency closely related to topics, which may lead to the quality decline of generated responses. Therefore, a topic-expanded emotional conversation generation model that integrated topic information and emotional factors was proposed. Firstly, the conversation context was globally-encoded, the topic model was introduced to obtain the global topic words, and the external affective dictionary was used to obtain the global affective words in this model. Secondly, the topic words were expanded by semantic similarity and the topic-related affective words were extracted by dependency syntax analysis in the fusion module. Finally, the context, topic words and affective words were input into a decoder based on the attention mechanism to prompt the decoder to generate topic-related emotional responses. Experimental results show that the model can generate rich and emotion-related responses. Compared with the model Topic-Enhanced Emotional Conversation Generation(TE-ECG), the proposed model has an average increase of 16.3% and 15.4% in unigram diversity(distinct-1) and bigram diversity(distinct-2); and compared with Seq2SeqA(Sequence to Sequence model with Attention), the proposed model has an average increase of 26.7% and 28.7% in unigram diversity(distinct-1) and bigram diversity(distinct-2).

Key words: emotional conversation generation, fusion module, topic model, Sequence to Sequence, (Seq2Seq) attention mechanism

中图分类号:

TP183

杨丰瑞, 霍娜, 张许红, 韦巍. 基于注意力机制的主题扩展情感对话生成[J]. 计算机应用, 2021, 41(4): 1078-1083.

YANG Fengrui, HUO Na, ZHANG Xuhong, WEI Wei. Topic-expanded emotional conversation generation based on attention mechanism[J]. Journal of Computer Applications, 2021, 41(4): 1078-1083.

参考文献

[1] MARTINOVSKI B,TRAUM D. The error is the clue:breakdown in human-machine interaction[C]//Proceedings of the 2003 ISCA Tutorial and Research Workshop on Error Handling in Dialogue Systems. Chateau d' Oex,Vaud,Switzerland:International Speech Communication Association,2003:11-16.
[2] PRENDINGER H, MORI J, ISHIZUKA M. Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game[J]. International Journal of Human-Computer Studies,2005,62(2):231-245.
[3] SHANG L,LU Z,LI H. Neural responding machine for short-text conversation[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics/7th International Joint Conference on Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2015:1577-1586.
[4] MOU L,SONG Y,YAN R,et al. Sequence to backward and forward sequences:a content-introducing approach to generative short text conversation[C]//Proceedings of the 26th International Conference on Computational Linguistics. Osaka:COLING 2016 Organizing Committee,2016:3349-3358.
[5] XING C,WU W,WU Y,et al. Topic aware neural response generation[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto,CA:AAAI Press,2017:3351-3357.
[6] ZHOU H, HUANG M, ZHANG T, et al. Emotional chatting machine:emotional conversation generation with internal and external memory[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press, 2018:730-738.
[7] 彭叶红. 基于主题模型与变分自编码的情感对话生成技术研究[D]. 武汉:华中师范大学,2019:22-24.(PENG Y H. Research on emotional conversation generation technology based on topic model and variational auto-encoder[D]. Wuhan:Central China Normal University,2019:22-24.)
[8] WANG W,HUANG M,XU X,et al. Chat more:deepening and widening the chatting topic via a deep model[C]//Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM,2018:255-264.
[9] ZHOU X,LI L,DONG D,et al. Multi-turn response selection for chatbots with deep attention matching network[C]//Proceedings of the 56th Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics,2018:1118-1127.
[10] WU Y,LI Z,WU W,ZHOU M. Response selection with topic clues for retrieval-based chatbots[J]. Neurocomputing,2018, 316:251-261.
[11] YANG L,QIU M,QU C,et al. Response ranking with deep matching networks and external knowledge in information-seeking conversation systems[C]//Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM,2018:245-254.
[12] SHAO Y,GOUWS S,BRITZ D,et al. Generating high-quality and informative conversation responses with sequence-to-sequence models[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics,2017:2210-2219.
[13] YAO L,ZHNAG Y,FENG Y,et al. Towards implicit contentintroducing for generative short-text conversation systems[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2017:2190-2199.
[14] TAO C,GAO S,SHANG M,et al. Get the point of my utterance! Learning towards effective responses with multi-head attention mechanism[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press,2018:4418-4424.
[15] LIU B,XU Z,SUN C,et al. Content-oriented user modeling for personalized response ranking in chatbots[J]. IEEE/ACM Transactions on Audio,Speech,and Language Processing,2018, 26(1):122-133.
[16] 陈晨, 朱晴晴, 严睿, 等. 基于深度学习的开放领域对话系统研究综述[J]. 计算机学报,2019,42(7):1439-1466.(CHEN C, ZHU Q Q,YAN R,et al. Survey on deep learning based open domain dialogue system[J]. Chinese Journal of Computers,2019, 42(7):1439-1466.)
[17] GHOSH S,CHOLLET M,LAKSANA E,et al. Affect-LM:a neural Language Model for customizable Affective text generation[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg,PA:Association for Computational Linguistics,2017:634-642.
[18] ZHANG R,WANG Z. Learning to converse emotionally like humans:a conditional variational approach[C]//Proceedings of the 7th CCF International Conference on Natural Language Processing and Chinese Computing, LNCS 11108. Cham:Springer,2018:98-109.
[19] CHO K,VAN MERRIËNBOER B,BAHDANAU D,et al. On the properties of neural machine translation:encoder-decoder approaches[C]//Proceedings of the 8th Workshop on Syntax, Semantics and Structure in Statistical Translation. Stroudsburg, PA:Association for Computational Linguistics,2014:103-111.
[20] ZNAO W X,JIANG J,WENG J,et al. Comparing Twitter and traditional media using topic models[C]//Proceedings of the 33rd European Conference on Information Retrieval, LNCS 6611. Berlin:Springer,2011:338-349.
[21] 徐琳宏, 林鸿飞, 潘宇, 等. 情感词汇本体的构造[J]. 情报学报,2008,27(2):180-185.(XU L H,LIN H F,PAN Y,et al. Constructing the affective lexicon ontology[J]. Journal of the China Society for Scientific and Technical Information,2008,27(2):180-185.)
[22] KINGMA D P, BA J L. Adam:a method for stochastic optimization[EB/OL].[2020-03-20]. https://arxiv.org/pdf/1412.6980v9.pdf.
[23] BAHDANAU D,CHO K,BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL].[2020-03-20]. https://arxiv.org/pdf/1409.0473.pdf.
[24] LIU C W,LOWE R,SERBAN L,et al. How NOT to evaluate your dialogue system:an empirical study of unsupervised evaluation metrics for dialogue response generation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg,PA:Association for Computational Linguistics,2016:2122-2132.
[25] LI J,GALLEY M,BROCKETT C,et al. A diversity-promoting objective function for neural conversation models[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg,PA:Association for Computational Linguistics,2016:110-119.

基于注意力机制的主题扩展情感对话生成

Topic-expanded emotional conversation generation based on attention mechanism

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[2]	刘雅璇, 钟勇. 基于头实体注意力的实体关系联合抽取方法[J]. 计算机应用, 2021, 41(9): 2517-2522.
[3]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[4]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[5]	党伟超, 李涛, 白尚旺, 高改梅, 刘春霞. 基于自注意力长短期记忆网络的Web软件系统实时剩余寿命预测方法[J]. 计算机应用, 2021, 41(8): 2346-2351.
[6]	李朝, 兰海, 魏宪. 基于注意力的毫米波-激光雷达融合目标检测[J]. 计算机应用, 2021, 41(7): 2137-2144.
[7]	李扬志, 袁家政, 刘宏哲. 基于时空注意力图卷积网络模型的人体骨架动作识别算法[J]. 计算机应用, 2021, 41(7): 1915-1921.
[8]	张洋, 江铭虎. 基于注意力机制的文本作者识别[J]. 计算机应用, 2021, 41(7): 1897-1901.
[9]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.
[10]	武维, 李泽平, 杨华蔚, 林川, 王忠德. 融合内容特征和时序信息的深度注意力视频流行度预测模型[J]. 计算机应用, 2021, 41(7): 1878-1884.
[11]	李想, 王卫兵, 尚学达. 指针生成网络和覆盖损失优化的Transformer在生成式文本摘要领域的应用[J]. 计算机应用, 2021, 41(6): 1647-1651.
[12]	沈雪雯, 王晓东, 姚宇. 基于空间分频的超声图像分割注意力网络[J]. 计算机应用, 2021, 41(6): 1828-1835.
[13]	刘世泽, 朱奕达, 陈润泽, 罗海勇, 赵方, 孙艺, 王宝会. 基于残差时域注意力神经网络的交通模式识别算法[J]. 计算机应用, 2021, 41(6): 1557-1565.
[14]	张蓉, 张献国. 基于层次异构图注意力网络的虚假评论检测[J]. 计算机应用, 2021, 41(5): 1275-1281.
[15]	李文惠, 曾上游, 王金金. 基于改进注意力机制的图像描述生成算法[J]. 计算机应用, 2021, 41(5): 1262-1267.