《计算机应用》唯一官方网站 ›› 2021, Vol. 41 ›› Issue (11): 3132-3138.DOI: 10.11772/j.issn.1001-9081.2021010040

• 人工智能 • 上一篇    下一篇

用于短文本情感分类的多头注意力记忆网络

邓钰1, 李晓瑜1(), 崔建2, 刘齐3   

  1. 1.电子科技大学 信息与软件工程学院,成都 610054
    2.解放军93246部队,长春 130000
    3.解放军95486部队,成都 610041
  • 收稿日期:2021-01-11 修回日期:2021-03-03 接受日期:2021-03-30 发布日期:2021-04-26 出版日期:2021-11-10
  • 通讯作者: 李晓瑜
  • 作者简介:邓钰(1983—),男,江西景德镇人,博士研究生,主要研究方向:自然语言处理、深度学习; 李晓瑜(1985—),女,山东菏泽人,副教授,博士,主要研究方向:数据挖掘、量子机器学习、大数据; 崔建(1981—),男,辽宁营口人,博士,主要研究方向:信息融合、数据挖掘
    刘齐(1982—),女,湖南长沙人,硕士,主要研究方向:网络安全、数据挖掘。
  • 基金资助:
    四川省科技计划项目(重点研发项目)(19ZDYF0794)

Multi-head attention memory network for short text sentiment classification

Yu DENG1, Xiaoyu LI1(), Jian CUI2, Qi LIU3   

  1. 1.School of Information and Software Engineering,University of Electronic Science and Technology of China,Chengdu Sichuan 610054,China
    2.Unit 93246 of PLA,Changchun Jilin 130000,China
    3.Unit 95486 of PLA,Chengdu Sichuan 610041,China
  • Received:2021-01-11 Revised:2021-03-03 Accepted:2021-03-30 Online:2021-04-26 Published:2021-11-10
  • Contact: Xiaoyu LI
  • About author:DENG Yu,born in 1983,Ph. D. candidate. His research interests include natural language processing,deep learning.
    LI Xiaoyu,born in 1985,Ph. D,associate professor. Her research interests include data mining,quantum machine learning,big data.
    CUI Jian,born in 1981,Ph. D. His research interests include information fusion,data mining.
    LIU Qi,born in 1982,M. S. Her research interests include network security,data mining.
  • Supported by:
    the Science and Technology Program of Sichuan Province (Key Research and Development Program)(19ZDYF0794)

摘要:

随着社交网络的发展,对其包含的海量文本进行情感分析具有重要的社会价值。不同于普通文本分类,短文本情感分类需要挖掘隐含的情感语义特征,具有极大的难度和挑战性。为了能在更高的层次上得到短文本的情感语义特征,提出了一种多头注意力记忆网络(MAMN)用于短文本情感分类。首先,利用n元语法特征信息和有序神经元长短时记忆(ON-LSTM)网络对多头自注意力机制进行改进,以对文本上下文内联关系进行充分提取,使模型可以获得更丰富的文本特征信息。然后,利用多头注意力机制对多跳记忆网络的结构进行优化,使得在拓展模型深度的同时,挖掘更高层次的上下文内联情感语义关系。在电影评论集(MR)、斯坦福情感树(SST)-1和SST-2这三个不同的数据集上进行了大量实验。实验结果表明,与基于循环神经网络(RNN)和卷积神经网络(CNN)结构的基线模型以及一些最新成果相比,所提MAMN取得了较优的分类效果,验证了多跳结构对于性能改善的重要作用。

关键词: 短文本, 情感分类, 情感语义特征, 多头注意力, 记忆网络

Abstract:

With the development of social networks, it has important social value to analyze the sentiments of massive texts in the social networks. Different from ordinary text classification, short text sentiment classification needs to mine the implicit sentiment semantic features, so it is very difficult and challenging. In order to obtain short text sentiment semantic features at a higher level, a new Multi-head Attention Memory Network (MAMN) was proposed for sentiment classification of short texts. Firstly, n-gram feature information and Ordered Neurons Long Short-Term Memory (ON-LSTM) network were used to improve the multi-head self-attention mechanism to fully extract the internal relationship of the text context, so that the model was able obtain richer text feature information. Secondly, multi-head attention mechanism was adopted to optimize the multi-hop memory network structure, so as to expand the depth of the model and mine higher level contextual internal semantic relations at the same time. A large number of experiments were carried out on Movie Review dataset (MR), Stanford Sentiment Treebank (SST)-1 and SST-2 datasets. The experimental results show that compared with the baseline models based on Recurrent Neural Network (RNN) and Convolutional Neural Network (CNN) structure and some latest works, the proposed MAMN achieves the better classification results, and the importance of multi-hop structure in performance improvement is verified.

Key words: short text, sentiment classification, sentiment semantic feature, multi-head attention, memory network

中图分类号: