Journal of Computer Applications ›› 2018, Vol. 38 ›› Issue (11): 3053-3056.DOI: 10.11772/j.issn.1001-9081.2018041363

    Next Articles

Transfer learning based hierarchical attention neural network for sentiment analysis

QU Zhaowei1, WANG Yuan1, WANG Xiaoru2   

  1. 1. Institute of Network Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China;
    2. College of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • Received:2018-04-29 Revised:2018-06-28 Online:2018-11-10 Published:2018-11-10
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61672108).

基于迁移学习的分层注意力网络情感分析算法

曲昭伟1, 王源1, 王晓茹2   

  1. 1. 北京邮电大学 网络技术研究院, 北京 100876;
    2. 北京邮电大学 计算机学院, 北京 100876
  • 通讯作者: 王源
  • 作者简介:曲昭伟(1970-),男,吉林辽源人,教授,博士,主要研究方向:数据挖掘、人工智能;王源(1994-),女,吉林长春人,硕士研究生,主要研究方向:人工智能、深度学习;王晓茹(1980-),女,北京人,副教授,博士,主要研究方向:计算机视觉、人工智能。
  • 基金资助:
    国家自然科学基金资助项目(61672108)。

Abstract: The purpose of document-level sentiment analysis is to predict users' sentiment expressed in the document. Traditional neural network-based methods rely on unsupervised word vectors. However, the unsupervised word vectors cannot exactly represent the contextual relationship of context and understand the context. Recurrent Neural Network (RNN) generally used to process sentiment analysis problems has complex structure and numerous model parameters. To address the above issues, a Transfer Learning based Hierarchical Attention Neural Network (TLHANN) was proposed. Firstly, an encoder was trained to understand the context with machine translation task for generating hidden vectors. Then, the encoder was transferred to sentiment analysis task by concatenating the hidden vector generated by the encoder with the corresponding unsupervised vector. The contextual relationship of context could be better represented by distributed representation. Finally, a two-level hierarchical network was applied to sentiment analysis task. A simplified RNN unit called Minimal Gate Unit (MGU) was arranged at each level leading to fewer parameters. The attention mechanism was used in the model for extracting important information. The experimental results show that, the accuracy of the proposed algorithm is increased by an avervage of 8.7% and 23.4% compared with the traditional neural network algorithm and Support Vector Machine (SVM).

Key words: sentiment analysis, Recurrent Neural Network (RNN), transfer learning, distributed representation, attention mechanism

摘要: 文本情感分析的目的是判断文本的情感类型。传统的基于神经网络的研究方法主要依赖于无监督训练的词向量,但这些词向量无法准确体现上下文语境关系;常用于处理情感分析问题的循环神经网络(RNN),模型参数众多,训练难度较大。为解决上述问题,提出了基于迁移学习的分层注意力神经网络(TLHANN)的情感分析算法。首先利用机器翻译任务训练一个用于在上下文中理解词语的编码器;然后,将这个编码器迁移到情感分析任务中,并将编码器输出的隐藏向量与无监督训练的词向量结合。在情感分析任务中,使用双层神经网络,每层均采用简化的循环神经网络结构——最小门单元(MGU),有效减少了参数个数,并引入了注意力机制提取重要信息。实验结果证明,所提算法的分类准确率与传统循环神经网络算法、支持向量机(SVM)算法相比分别平均提升了8.7%及23.4%。

关键词: 情感分析, 循环神经网络, 迁移学习, 分布式表示, 注意力机制

CLC Number: