Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (8): 2186-2191.DOI: 10.11772/j.issn.1001-9081.2019010096

• Artificial intelligence • Previous Articles     Next Articles

Cross-domain sentiment classification method of convolution-bi-directional long short-term memory based on attention mechanism

GONG Qin, LEI Man, WANG Jichao, WANG Baoqun   

  1. School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China
  • Received:2019-01-14 Revised:2019-03-19 Online:2019-08-10 Published:2019-04-15
  • Supported by:
    This work is partially supported by the Program for Changjiang Scholars and Innovative Research Teams in Universities (IRT_16R72).

基于注意力机制的卷积双向长短期记忆模型跨领域情感分类方法

龚琴, 雷曼, 王纪超, 王保群   

  1. 重庆邮电大学 通信与信息工程学院, 重庆 400065
  • 通讯作者: 龚琴
  • 作者简介:龚琴(1993-),女,四川成都人,硕士研究生,主要研究方向:自然语言处理、深度学习;雷曼(1992-),女,重庆人,硕士研究生,主要研究方向:推荐系统、社交网络;王纪超(1993-),男,河南郑州人,硕士研究生,主要研究方向:机器学习、数据挖掘;王保群(1993-),男,山东聊城人,硕士研究生,主要研究方向:机器学习、数据挖掘。
  • 基金资助:
    长江学者和创新团队发展计划项目(IRT_16R72)。

Abstract: Concerning the problems that the text representation features in the existing cross-domain sentiment classification method ignore the sentiment information of important words and there is negative transfer during transfer process, a Convolution-Bi-directional Long Short-Term Memory based on Attention mechanism (AC-BiLSTM) model was proposed to realize knowledge transfer. Firstly, the vector representation of text was obtained by low-dimensional dense word vectors. Secondly, after local context features being obtained by convolution operation, the long dependence relationship between the features was fully considered by Bi-directional Long Short-Term Memory (BiLSTM) network. Then, the contribution degrees of different words to the text were considered by introducing attention mechanism, and a regular term constraint was introduced into the objective function in order to avoid the negative transfer phenomenon in transfer process. Finally, the model parameters trained on source domain product reviews were transferred to target domain product reviews, and the labeled data in a small number of target domains were fine-tuned. Experimental results show that compared with AE-SCL-SR (AutoEncoder Structural Correspondence Learning with Similarity Regularization) method and Adversarial Memory Network (AMN) method, AC-BiLSTM method has average accuracy increased by 6.5% and 2.2% respectively, which demonstrates that AC-BiLSTM method can effectively improve cross-domain sentiment classification performance.

Key words: sentiment classification, cross-domain, transfer learning, attention mechanism, Long Short-Term Memory (LSTM) network

摘要: 针对现有跨领域情感分类方法中文本表示特征忽略了重要单词的情感信息,且在迁移过程中存在负迁移的问题,提出一种基于注意力机制的卷积-双向长短期记忆(AC-BiLSTM)模型的知识迁移方法。首先,利用低维稠密的词向量对文本进行向量表示;其次,采用卷积操作获取局部上下文特征之后,通过双向长短期记忆(BiLSTM)网络充分考虑特征之间的长期依赖关系;然后,通过引入注意力机制考虑不同词汇对文本的贡献程度,同时为了避免迁移过程中出现负迁移现象,在目标函数中引入正则项约束;最后,将在源领域产品评论训练得到的模型参数迁移到目标领域产品评论中,并在少量目标领域有标注数据上进行微调。实验结果表明,与AE-SCL-SR方法和对抗记忆网络(AMN)方法相比,AC-BiLSTM方法的平均准确率分别提高了6.5%和2.2%,AC-BiLSTM方法可以有效地提高跨领域情感分类性能。

关键词: 情感分类, 跨领域, 迁移学习, 注意力机制, 长短期记忆网络

CLC Number: