Cross-domain sentiment classification method of convolution-bi-directional long short-term memory based on attention mechanism

doi:10.11772/j.issn.1001-9081.2019010096

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (8): 2186-2191.DOI: 10.11772/j.issn.1001-9081.2019010096

• Artificial intelligence • Previous Articles Next Articles

Cross-domain sentiment classification method of convolution-bi-directional long short-term memory based on attention mechanism

GONG Qin, LEI Man, WANG Jichao, WANG Baoqun

School of Communication and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, China

Received:2019-01-14 Revised:2019-03-19 Online:2019-08-10 Published:2019-04-15
Supported by:
This work is partially supported by the Program for Changjiang Scholars and Innovative Research Teams in Universities (IRT_16R72).

基于注意力机制的卷积双向长短期记忆模型跨领域情感分类方法

龚琴, 雷曼, 王纪超, 王保群

重庆邮电大学通信与信息工程学院, 重庆 400065

通讯作者: 龚琴
作者简介:龚琴(1993-),女,四川成都人,硕士研究生,主要研究方向:自然语言处理、深度学习;雷曼(1992-),女,重庆人,硕士研究生,主要研究方向:推荐系统、社交网络;王纪超(1993-),男,河南郑州人,硕士研究生,主要研究方向:机器学习、数据挖掘;王保群(1993-),男,山东聊城人,硕士研究生,主要研究方向:机器学习、数据挖掘。
基金资助:
长江学者和创新团队发展计划项目（IRT_16R72）。

Abstract

Abstract: Concerning the problems that the text representation features in the existing cross-domain sentiment classification method ignore the sentiment information of important words and there is negative transfer during transfer process, a Convolution-Bi-directional Long Short-Term Memory based on Attention mechanism (AC-BiLSTM) model was proposed to realize knowledge transfer. Firstly, the vector representation of text was obtained by low-dimensional dense word vectors. Secondly, after local context features being obtained by convolution operation, the long dependence relationship between the features was fully considered by Bi-directional Long Short-Term Memory (BiLSTM) network. Then, the contribution degrees of different words to the text were considered by introducing attention mechanism, and a regular term constraint was introduced into the objective function in order to avoid the negative transfer phenomenon in transfer process. Finally, the model parameters trained on source domain product reviews were transferred to target domain product reviews, and the labeled data in a small number of target domains were fine-tuned. Experimental results show that compared with AE-SCL-SR (AutoEncoder Structural Correspondence Learning with Similarity Regularization) method and Adversarial Memory Network (AMN) method, AC-BiLSTM method has average accuracy increased by 6.5% and 2.2% respectively, which demonstrates that AC-BiLSTM method can effectively improve cross-domain sentiment classification performance.

Key words: sentiment classification, cross-domain, transfer learning, attention mechanism, Long Short-Term Memory (LSTM) network

摘要： 针对现有跨领域情感分类方法中文本表示特征忽略了重要单词的情感信息，且在迁移过程中存在负迁移的问题，提出一种基于注意力机制的卷积-双向长短期记忆（AC-BiLSTM）模型的知识迁移方法。首先，利用低维稠密的词向量对文本进行向量表示；其次，采用卷积操作获取局部上下文特征之后，通过双向长短期记忆（BiLSTM）网络充分考虑特征之间的长期依赖关系；然后，通过引入注意力机制考虑不同词汇对文本的贡献程度，同时为了避免迁移过程中出现负迁移现象，在目标函数中引入正则项约束；最后，将在源领域产品评论训练得到的模型参数迁移到目标领域产品评论中，并在少量目标领域有标注数据上进行微调。实验结果表明，与AE-SCL-SR方法和对抗记忆网络（AMN）方法相比，AC-BiLSTM方法的平均准确率分别提高了6.5%和2.2%，AC-BiLSTM方法可以有效地提高跨领域情感分类性能。

关键词: 情感分类, 跨领域, 迁移学习, 注意力机制, 长短期记忆网络

CLC Number:

GONG Qin, LEI Man, WANG Jichao, WANG Baoqun. Cross-domain sentiment classification method of convolution-bi-directional long short-term memory based on attention mechanism[J]. Journal of Computer Applications, 2019, 39(8): 2186-2191.

龚琴, 雷曼, 王纪超, 王保群. 基于注意力机制的卷积双向长短期记忆模型跨领域情感分类方法[J]. 计算机应用, 2019, 39(8): 2186-2191.

References

[1] AGARWAL B, MITTAL N, BANSAL P, et al. Sentiment analysis using common-sense and context information[J]. Computational Intelligence and Neuroscience, 2015, 2015(78):Article No. 30.
[2] 陈龙,管子玉,何金红,等.情感分类研究进展[J].计算机研究与发展,2017,54(6):1150-1170. (CHEN L, GUAN Z Y, HE J H, et al. A survey on sentiment classification[J]. Journal of Computer Research and Development, 2017, 54(6):1150-1170.)
[3] GAMA J, ŽLIOBAITE I, BIFET A, et al. A survey on concept drift adaptation[J]. ACM Computing Surveys, 2014, 46(4):Article No. 44.
[4] BARDDAL J P, GOMES H M, ENEMBRECK F, et al. A survey on feature drift adaptation:definition, benchmark, challenges and future directions[J]. Journal of Systems and Software, 2017, 127(52):278-294.
[5] MOHAMED HUSSEIN D M E D. A survey on sentiment analysis challenges[J]. Journal of King Saud University-Engineering Sciences, 2018, 30(4):330-338.
[6] RAVI K, RAVI V. A survey on opinion mining and sentiment analysis:tasks, approaches and applications[J]. Knowledge-Based Systems, 2015, 89(17):14-46.
[7] BLIZER J, DREDZE M, PEREIRA F. Biographies, bollywood, boom-boxes and blenders:domain adaptation for sentiment classification[C]//Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. Stroudsburg, PA:ACL, 2007:440-447.
[8] 庄福振,罗平,何清,等.迁移学习研究进展[J].软件学报,2015,26(1):26-39. (ZHUANG F Z, LUO P, HE Q, et al. Survey on transfer learning research[J]. Journal of Software, 2015, 26(1):26-39.)
[9] BLITZER J, McDONALD R, PEREIRA F. Domain adaptation with structural correspondence learning[C]//Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:ACL, 2006:120-128.
[10] PAN S J, NI X, SUN J T, et al. Cross-domain sentiment classification via spectral feature alignment[C]//Proceedings of the 19th International Conference on World Wide Web. New York:ACM, 2010:751-760.
[11] XIA R, ZONG C, HU X, et al. Feature ensemble plus sample selection:domain adaptation for sentiment classification[J]. IEEE Intelligent Systems, 2013, 28(3):10-18.
[12] DESHMUKH J S, TRIPATHY A K. Entropy based classifier for cross-domain opinion mining[J]. Applied Computing and Informatics, 2018, 14(1):55-64.
[13] 吴斌,吉佳,孟琳,等.基于迁移学习的唐诗宋词情感分析[J].电子学报,2016,44(11):2780-2787. (WU B, JI J, MENG L, et al. Transfer learning based sentiment analysis for poetry of the Tang dynasty and Song dynasty[J]. Acta Electronica Sinica, 2016, 44(11):2780-2787.
[14] QIN B, LIU T, TANG D. Deep learning for sentiment analysis:successful approaches and future challenges[J]. Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery, 2015, 5(6):292-303.
[15] SCHMIDHUBER J. Deep learning in neural networks:an overview[J]. Neural Networks, 2015, 61(27):85-117.
[16] GLOROT X, BORDES A, BENGIO Y. Domain adaptation for large-scale sentiment classification:a deep learning approach[C]//Proceedings of the 28th International Conference on Machine Learning. New York:ACM, 2011:513-520.
[17] CHEN M, XU Z, WEINBERGER K Q, et al. Marginalized denoising autoencoders for domain adaptation[C]//Proceedings of the 29th International Conference on Machine Learning. New York:ACM, 2012:1627-1634.
[18] CLINCHANT S, CSURKA G, CHIDLOVSKⅡ B. A domain adaptation regularization for denoising autoencoders[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:ACL, 2016:26-31.
[19] PENG Y, WANG S, LU B L. Marginalized denoising autoencoder via graph regularization for domain adaptation[C]//Proceedings of the 2013 International Conference on Neural Information Processing, LNCS 8227. Berlin:Springer, 2013:156-163.
[20] GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. Journal of Machine Learning Research, 2015, 17(1):2096-2030.
[21] ZISER Y, REICHART R. Neural structural correspondence learning for domain adaptation[C]//Proceedings of the 21st Conference on Computational Natural Language Learning. Stroudsburg, PA:ACL, 2017:400-410.
[22] LI Z, ZHANG Y, WEI Y, et al. End-to-end adversarial memory network for cross-domain sentiment classification[C]//Proceedings of the 2017 International Joint Conference on Artificial Intelligence. Menlo Park, CA:AAAI Press, 2017:2237-2243.
[23] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[C]//Proceedings of the 2013 International Conference on Learning Representations. Stroudsburg, PA:ACL, 2013:1-12.
[24] BAHDANA D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego, CA:[s.n.], 2015:1-15.

Cross-domain sentiment classification method of convolution-bi-directional long short-term memory based on attention mechanism

基于注意力机制的卷积双向长短期记忆模型跨领域情感分类方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[2]	LIU Yaxuan, ZHONG Yong. Joint extraction method of entities and relations based on subject attention [J]. Journal of Computer Applications, 2021, 41(9): 2517-2522.
[3]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[4]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[5]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[6]	DANG Weichao, LI Tao, BAI Shangwang, GAO Gaimei, LIU Chunxia. Real-time remaining life prediction method of Web software system based on self-attention-long short-term memory network [J]. Journal of Computer Applications, 2021, 41(8): 2346-2351.
[7]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[8]	LI Yangzhi, YUAN Jiazheng, LIU Hongzhe. Human skeleton-based action recognition algorithm based on spatiotemporal attention graph convolutional network model [J]. Journal of Computer Applications, 2021, 41(7): 1915-1921.
[9]	ZHANG Yang, JIANG Minghu. Authorship identification of text based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(7): 1897-1901.
[10]	ZHANG Yuanjun, ZHANG Xihuang. Dynamic network representation learning model based on graph convolutional network and long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1857-1864.
[11]	WU Wei, LI Zeping, YANG Huawei, LIN Chuan, WANG Zhongde. Deep attention video popularity prediction model fusing content features and temporal information [J]. Journal of Computer Applications, 2021, 41(7): 1878-1884.
[12]	LI Chao, LAN Hai, WEI Xian. Attention-based object detection with millimeter wave radar-lidar fusion [J]. Journal of Computer Applications, 2021, 41(7): 2137-2144.
[13]	LIU Shize, ZHU Yida, CHEN Runze, LUO Haiyong, ZHAO Fang, SUN Yi, WANG Baohui. Traffic mode recognition algorithm based on residual temporal attention neural network [J]. Journal of Computer Applications, 2021, 41(6): 1557-1565.
[14]	LI Xiang, WANG Weibing, SHANG Xueda. Application of Transformer optimized by pointer generator network and coverage loss in field of abstractive text summarization [J]. Journal of Computer Applications, 2021, 41(6): 1647-1651.
[15]	SHEN Xuewen, WANG Xiaodong, YAO Yu. Spatial frequency divided attention network for ultrasound image segmentation [J]. Journal of Computer Applications, 2021, 41(6): 1828-1835.