• •    

基于注意力机制的改进CLSM检索式匹配问答方法研究

于重重1,曹帅2,潘博2,张青川3,徐世璇4   

  1. 1. 北京工商大学 计算机与信息工程学院,北京100048
    2. 北京工商大学
    3. 北京科技大学 计算机与通信工程学院
    4. 中国社会科学院
  • 收稿日期:2018-08-15 修回日期:2018-11-02 发布日期:2018-11-02
  • 通讯作者: 于重重

Retrieval Matching Question-and-Answer method based on improved CLSM

  • Received:2018-08-15 Revised:2018-11-02 Online:2018-11-02
  • Contact: YU Chongchong

摘要: 传统检索式匹配问答系统在对问句和候选答案进行特征抽取时使用了一些较为粗糙的方法, 忽略了句子的语义信息,直接影响了问答结果的准确性。近年来,随着人工智能大潮的又一次兴起,基于卷积神经网络的潜在语义模型(Convolutional Latent Semantic Model, CLSM)在提取句子语义特征方面取得了不错的效果。针对中文匹配问答任务,该文在传统CLSM模型上进行改进,去掉了N元模型层,设计了基于CLSM的中文文本语义特征提取模型,改进了传统CLSM模型对中文文本适应性弱的缺点;并引入了实体关注层,对句子中的核心词汇的语义信息进行加强。同时设计了三组对比实验,实验结果显示改进模型较传统翻译模型在NDCG方面有4%-10%的提升。验证了所建模型能够通过实体关注层加强核心词的信息,同时利用卷积神经网络有效地捕获语义匹配有用的上下文结构方面信息,从而提升检索式匹配问答的准确率。

关键词: CLSM, 注意力机制, 检索式匹配问答

Abstract: The traditional Retrieval Matching Question and Answer model (RMQA) uses some old methods in the feature extraction of questions and candidate answers, which ignores the semantic information of sentences and affects the accuracy of question and answer results substantially. Recently, the Convolutional Latent Semantic Model (CLSM) has got good results in extracting sentence semantic features. As for RMQA task, we drop the word-n-gram layer and letter-n-gram layer of the CLSM to build a CLSM-based semantic feature extraction model. And we also propose an entity_attention layer to balance the matrix’s value of important entity. We design three groups of contrast experiments, the experimental results showed that the improved model had a 4%-10% improvement over the traditional translation model in NDCG. It is proved that the model can strengthen the information of the core word through the entity concern layer, and use the convolution neural network to capture the useful contextual structure information of semantic matching effectively so as to improve the accuracy of the retrieval matching question and answer.

Key words: CLSM, Attention, RMQA