计算机应用 ›› 2019, Vol. 39 ›› Issue (4): 972-976.DOI: 10.11772/j.issn.1001-9081.2018081691

• 人工智能 • 上一篇    下一篇

基于注意力机制的改进CLSM检索式匹配问答方法

于重重1, 曹帅1, 潘博1, 张青川1, 徐世璇2   

  1. 1. 北京工商大学 计算机与信息工程学院, 北京 100048;
    2. 中国社会科学院 民族学与人类学研究所, 北京 100081
  • 收稿日期:2018-08-15 修回日期:2018-11-12 发布日期:2019-04-10 出版日期:2019-04-10
  • 通讯作者: 于重重
  • 作者简介:于重重(1971-),女,北京人,教授,博士,CCF会员,主要研究方向:人工智能、模式识别、机器学习;曹帅(1993-),女,甘肃庆阳人,硕士研究生,主要研究方向:人工智能、自然语言处理;潘博(1992-),男,广西南宁人,硕士研究生,主要研究方向:人工智能、自然语言处理;张青川(1982-),男,北京人,讲师,博士研究生,主要研究方向:自然语言处理、机器学习、数据挖掘;徐世璇(1954-),女,浙江宁波人,研究员,博士,主要研究方向:少数民族语言。
  • 基金资助:
    教育部人文社会科学研究规划基金资助项目(16YJAZH072);国家社会科学基金重大项目(14ZDB156)。

Retrieval matching question and answer method based on improved CLSM with attention mechanism

YU Chongchong1, CAO Shuai1, PAN Bo1, ZHANG Qingchuan1, XU Shixuan2   

  1. 1. School of Computer and Information Engineering, Beijing Technology and Business University, Beijing 100048, China;
    2. Institute of Ethnology and Anthropology, Chinese Academy of Social Sciences, Beijing 100081, China
  • Received:2018-08-15 Revised:2018-11-12 Online:2019-04-10 Published:2019-04-10
  • Supported by:
    This work is partially supported by the Ministry of Education Humanities and Social Sciences Research Planning Fund Project (16YJAZH072), the National Social Science Foundation major Project (14ZDB156).

摘要: 针对检索式匹配问答模型对中文语料适应性弱和句子语义信息被忽略的问题,提出一种基于卷积神经网络潜在语义模型(CLSM)的中文文本语义匹配模型。首先,在传统CLSM基础上进行改进,去掉单词和字母的N元模型层,以增强模型对中文语料的适应性;其次,采用注意力机制算法,针对输入的中文词向量信息建立实体关注层模型,以加强句中核心词的权重信息;最后,通过卷积神经网络(CNN)有效地捕获输入句子上下文结构方面信息,并通过池化层对获取的语义信息进行降维。基于医疗问答对数据集,将改进模型与传统语义模型、传统翻译模型、深度神经网络模型进行对比,实验结果显示所提模型在归一化折现累积增益(NDCG)方面有4~10个百分点的提升,优于对比模型。

关键词: 潜在语义模型, 注意力机制, 检索式匹配问答

Abstract: Focusing on the problem that the Retrieval Matching Question and Answer (RMQA) model has weak adaptability to Chinese corpus and the neglection of semantic information of the sentence, a Chinese text semantic matching model based on Convolutional neural network Latent Semantic Model (CLSM) was proposed. Firstly, the word-N-gram layer and letter-N-gram layer of CLSM were removed to enhance the adaptability of the model to Chinese corpus. Secondly, with the focus on vector information of input Chinese words, an entity attention layer model was established based on the attention mechanism algorithm to strengthen the weight information of the core words in sentence. Finally, Convolutional Neural Network (CNN) was used to capture the input sentence context structure information effectively and the pool layer was used to reduce the dimension of semantic information. In the experiments based on a medical question and answer dataset, compared with the traditional semantic models, traditional translation models and deep neural network models, the proposed model has 4-10 percentage points improvement in Normalized Discount Cumulative Gain (NDCG).

Key words: Convolutional Latent Semantic Model (CLSM), attention mechanism, Retrieval Matching Question and Answer (RMQA)

中图分类号: