Aspect-level cross-domain sentiment analysis based on capsule network

doi:10.11772/j.issn.1001-9081.2021101779

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (12): 3700-3707.DOI: 10.11772/j.issn.1001-9081.2021101779

• Artificial intelligence • Previous Articles

Aspect-level cross-domain sentiment analysis based on capsule network

Jiana MENG¹, Pin LYU¹, Yuhai YU¹(), Shichang SUN¹, Hongfei LIN²

^1.College of Computer Science and Engineering，Dalian Minzu University，Dalian Liaoning 116600，China
^2.School of Computer Science and Technology，Dalian University of Technology，Dalian Liaoning 116024，China

Received:2021-10-18 Revised:2021-12-29 Accepted:2022-01-14 Online:2022-01-24 Published:2022-12-10
Contact: Yuhai YU
About author:MENG Jiana，born in 1972， Ph. D.， professor. Her research interests include machine learning， text mining.
LYU Pin，born in 1996， M. S. candidate. Her research interests include sentiment tendency analysis.
SUN Shichang， born in 1979， Ph. D.， associate professor. His research interests include transfer learning.
LIN Hongfei，born in 1962， Ph. D.， professor. His research interests include text mining， information retrieval.
Supported by:
National Natural Science Foundation of China(61876031);2019 Scientific Research Funded Project of Liaoning Provincial Department of Education(LJYT201906)

基于胶囊网络的方面级跨领域情感分析

孟佳娜¹, 吕品¹, 于玉海¹(), 孙世昶¹, 林鸿飞²

^1.大连民族大学计算机科学与工程学院，辽宁大连 116600
^2.大连理工大学计算机科学与技术学院，辽宁大连 116024

通讯作者: 于玉海
作者简介:孟佳娜（1972—），女，吉林四平人，教授，博士，CCF会员，主要研究方向：机器学习、文本挖掘
吕品（1996—），女，内蒙古呼和浩特人，硕士研究生，CCF会员，主要研究方向：情感倾向性分析
孙世昶（1979—），男，辽宁大连人，副教授，博士，CCF会员，主要研究方向：迁移学习
林鸿飞（1962—），男，吉林辽源人，教授，博士，CCF会员，主要研究方向：文本挖掘、信息检索。
基金资助:
国家自然科学基金资助项目(61876031);辽宁省教育厅2019年度科学研究经费项目(LJYT201906)

Abstract

Abstract:

In the cross-domain sentiment analysis， the labeled samples in the target domain are seriously insufficient， the distributions of features in different domains are very different， and the emotional polarities expressed by features in one domain differ a lot from the emotional polarities in another domain， all of these problems lead to low classification accuracy. To deal with the above problems， an aspect-level cross-domain sentiment analysis method based on capsule network was proposed. Firstly， the feature representations of text were obtained by BERT （Bidirectional Encoder Representation from Transformers） pre-training model. Secondly， for the fine-grained aspect-level sentiment features， Recurrent Neural Network （RNN） was used to fuse the context features and aspect features. Thirdly， capsule network and dynamic routing were used to distinguish overlapping features， and the sentiment classification model was constructed on the basis of capsule network. Finally， a small amount of data in the target domain was used to fine-tune the model to realize cross-domain transfer learning. The optimal F1 score of the proposed method is 95.7% on Chinese dataset and 91.8% on English dataset， which effectively solves the low accuracy problem of insufficient training samples.

Key words: aspect-level sentiment analysis, cross-domain, capsule network, Recurrent Neural Network (RNN), pre-training

摘要：

在跨领域情感分析任务中，目标领域带标签样本严重不足，并且不同领域间的特征分布差异较大，特征所表达的情感极性也有很大差别，这些问题都导致了分类准确率较低。针对以上问题，提出一种基于胶囊网络的方面级跨领域情感分析方法。首先，通过BERT预训练模型获取文本的特征表示；其次，针对细粒度的方面级情感特征，采用循环神经网络（RNN）将上下文特征与方面特征进行融合；然后，使用胶囊网络配合动态路由来区分重叠特征，并构建基于胶囊网络的情感分类模型；最后，利用目标领域的少量数据对模型进行微调来实现跨领域迁移学习。所提方法在中文数据集上的最优的F1值达到95.7%，英文数据集上的最优的F1值达到了91.8%，有效解决了训练样本不足造成的准确率低的问题。

关键词: 方面级情感分析, 跨领域, 胶囊网络, 循环神经网络, 预训练

CLC Number:

TP389.1

Jiana MENG, Pin LYU, Yuhai YU, Shichang SUN, Hongfei LIN. Aspect-level cross-domain sentiment analysis based on capsule network[J]. Journal of Computer Applications, 2022, 42(12): 3700-3707.

孟佳娜, 吕品, 于玉海, 孙世昶, 林鸿飞. 基于胶囊网络的方面级跨领域情感分析[J]. 《计算机应用》唯一官方网站, 2022, 42(12): 3700-3707.

Figures/Tables 15

References 25

1	PANG B， LEE L. Opinion mining and sentiment analysis［J］. Foundations and Trends in Information Retrieval， 2008， 2（1/2）： 1-135. 10.1561/1500000011
2	ISHAQ A， ASGHAR S， GILLANI S A. Aspect-based sentiment analysis using a hybridized approach based on CNN and GA［J］. IEEE Access， 2020， 8： 135499-135512. 10.1109/access.2020.3011802
3	LI X， BING L D， ZHANG W X， et al. Exploiting BERT for end-to-end aspect-based sentiment analysis［C］// Proceedings of the 5th Workshop on Noisy User-generated Text. Stroudsburg， PA： Association for Computational Linguistics， 2019： 34-41. 10.18653/v1/d19-5505
4	贺萌.基于域适应的跨领域情感分类方法研究［D］. 北京：北京工业大学， 2020：45-64.
	HE M. Research on domain adaption based cross-domain sentiment classification method［D］. Beijing： Beijing University of Technology， 2020：45-64.
5	赵光耀，吕成国，付国宏，等. 基于领域特有情感词注意力模型的跨领域属性情感分析［J］. 中文信息学报， 2021， 35（6）： 93-102. 10.3969/j.issn.1003-0077.2021.06.010
	ZHAO G Y， LYU C G， FU G H， et al. Domain specific sentiment words based attention model for cross-domain attribute-oriented sentiment analysis［J］. Journal of Chinese Information Processing， 2021， 35（6）： 93-102. 10.3969/j.issn.1003-0077.2021.06.010
6	孟佳娜，吕品，于玉海，等. 基于CNN的方面级跨领域情感分析研究［J］. 计算机工程与应用， 2022， 58（16）：175-183. 10.3778/j.issn.1002-8331.2101-0316
	MENG J N， LYU P， YU Y H， et al. Aspect-level cross-domain sentiment analysis based on CNN［J］. Computer Engineering and Applications， 2022， 58（16）：175-183. 10.3778/j.issn.1002-8331.2101-0316
7	MA D H， LI S J， ZHANG X D， et al. Interactive attention networks for aspect-level sentiment classification［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 4068-4074. 10.24963/ijcai.2017/568
8	王昆，郑毅，方书雅，等. 基于文本筛选和改进BERT的长文本方面级情感分析［J］. 计算机应用， 2020， 40（10）：2838-2844. 10.11772/j.issn.1001-9081.2020020164
	WANG K， ZHENG Y， FANG S Y， et al. Long text aspect-level sentiment analysis based on text filtering and improved BERT［J］. Journal of Computer Applications， 2020， 40（10）：2838-2844. 10.11772/j.issn.1001-9081.2020020164
9	DU C N， SUN H F， WANG J Y， et al. Capsule network with interactive attention for aspect-level sentiment classification［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2019： 5489-5498. 10.18653/v1/d19-1551
10	支淑婷，李晓戈，王京博，等. 基于多注意力长短时记忆的实体属性情感分析［J］. 计算机应用， 2019， 39（1）：160-167. 10.11772/j.issn.1001-9081.2018061232
	ZHI S T， LI X G， WANG J B， et al. Sentiment analysis of entity aspects based on multi-attention long short-term memory［J］. Journal of Computer Applications， 2019， 39（1）：160-167. 10.11772/j.issn.1001-9081.2018061232
11	ZHU P S， QIAN T Y. Enhanced aspect level sentiment classification with auxiliary memory［C］// Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2018： 1077-1087.
12	武婷，曹春萍. 融合位置权重的基于注意力交叉注意力的长短期记忆方面情感分析模型［J］. 计算机应用， 2019， 39（8）：2198-2203. 10.11772/j.issn.1001-9081.2018122565
	WU T， CAO C P. Aspect level sentiment classification model with location weight and long-short term memory based on attention-over-attention［J］. Journal of Computer Applications， 2019， 39（8）：2198-2203. 10.11772/j.issn.1001-9081.2018122565
13	YOSINSKI J， CLUNE J， BENGIOY， et al. How transferable are features in deep neural networks？［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2014： 3320-3328.
14	LONG M S， CAO Y， WANG J M， et al. Learning transferable features with deep adaptation networks［C］// Proceedings of the 32nd Conference on International Conference on Machine Learning. New York： JMLR.org， 2015： 97-105.
15	GANIN Y， USTINOVA E， AJAKAN H， et al. Domain-adversarial training of neural networks［J］. Journal of Machine Learning Research， 2016， 17： 1-35. 10.1007/978-3-319-58347-1_10
16	XU R F， XU J， WANG X L. Instance level transfer learning for cross lingual opinion analysis［C］// Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis. Stroudsburg， PA： Association for Computational Linguistics， 2011： 182-188.
17	LI Z， ZHANG Y， WEI Y， et al. End-to-end adversarial memory network for cross-domain sentiment classification［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 2237-2243. 10.24963/ijcai.2017/311
18	HINTON G， SABOUR S， FROSST N. Matrix capsules with EM routing［EB/OL］. （2018-02-16）［2021-10-18］..
19	ZHAO W， YE J B， YANG M， et al. Investigating capsule networks with dynamic routing for text classification［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2018： 3110-3119. 10.18653/v1/d18-1350
20	WU Q， TAN S B. A two-stage framework for cross-domain sentiment classification［J］. Expert Systems with Applications， 2011， 38（11）： 14269-14275.
21	BLITZER J， DREDZE M， PEREIRA F. Biographies， Bollywood， boom-boxes and blenders： domain adaptation for sentiment classification［C］// Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2007： 440-447. 10.3115/1557690.1557765
22	SHARMA R， BHATTACHARYYA P， DANDAPAT S， et al. Identifying transferable information across domains for cross-domain sentiment classification［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2018： 968-978. 10.18653/v1/p18-1089
23	SUN K， ZHANG R C， MENSAH S， et al. Aspect-level sentiment analysis via convolution over dependency tree［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2019： 5679-5688. 10.18653/v1/d19-1569
24	ZHANG C， LI Q C， SONG D W. Aspect-based sentiment classification with aspect-specific graph convolutional networks［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2019： 4568-4578. 10.18653/v1/d19-1464
25	CHEN P， SUN Z Q， BING L D， et al. Recurrent attention network on memory for aspect sentiment analysis［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2017： 452-461. 10.18653/v1/d17-1047

超参数	值
Bert_size	768
capsule_size	300
句子长度	80
词向量维度	300
Batch_size	32
Dropout比率	0.2
epoch	5
微调数据比例（m）	0、0.05、0.1、0.15、0.2

超参数	值
Bert_size	768
capsule_size	300
句子长度	80
词向量维度	300
Batch_size	32
Dropout比率	0.2
epoch	5
微调数据比例（m）	0、0.05、0.1、0.15、0.2

源域 → 目标域	准确率		F1值
源域 → 目标域	句子级	方面级	句子级	方面级
平均值	0.781	0.883	0.707	0.886
C→B	0.725	0.869	0.870	0.875
C→H	0.683	0.863	0.641	0.858
H→B	0.780	0.840	0.785	0.864
H→C	0.835	0.934	0.882	0.913
B→C	0.848	0.932	0.854	0.945
B→H	0.812	0.862	0.813	0.860

源域 → 目标域	准确率		F1值
源域 → 目标域	句子级	方面级	句子级	方面级
平均值	0.781	0.883	0.707	0.886
C→B	0.725	0.869	0.870	0.875
C→H	0.683	0.863	0.641	0.858
H→B	0.780	0.840	0.785	0.864
H→C	0.835	0.934	0.882	0.913
B→C	0.848	0.932	0.854	0.945
B→H	0.812	0.862	0.813	0.860

源域→ 目标域	准确率		F1值
源域→ 目标域	句子级	方面级	句子级	方面级
平均值	0.792	0.858	0.802	0.861
B→D	0.804	0.867	0.841	0.876
B→E	0.743	0.854	0.820	0.852
B→K	0.785	0.848	0.793	0.868
D→B	0.824	0.877	0.813	0.866
D→E	0.780	0.826	0.762	0.837
D→K	0.813	0.890	0.825	0.870
E→D	0.760	0.857	0.796	0.848
E→K	0.831	0.890	0.836	0.865
E→B	0.772	0.854	0.766	0.855
K→D	0.812	0.858	0.785	0.846
K→E	0.816	0.864	0.804	0.878
K→B	0.759	0.805	0.781	0.876

Aspect-level cross-domain sentiment analysis based on capsule network

基于胶囊网络的方面级跨领域情感分析

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 25

Related Articles 15

Recommended Articles

Metrics

源域→目标域	SCL-MI	ITIAD	DANN	GCAE-TL	本文方法
平均值	0.743	0.742	0.748	0.751	0.795
B→D	0.788	0.805	0.737	0.774	0.756
B→E	0.719	0.730	0.680	0.763	0.812
B→K	0.772	0.720	0.788	0.764	0.748
D→B	0.732	0.670	0.750	0.752	0.796
D→E	0.715	0.740	0.745	0.746	0.783
D→K	0.740	0.710	0.776	0.747	0.786
E→B	0.685	0.683	0.700	0.734	0.797
E→D	0.704	0.775	0.710	0.794	0.856
E→K	0.829	0.857	0.845	0.713	0.848
K→B	0.693	0.679	0.712	0.723	0.806
K→D	0.720	0.740	0.714	0.788	0.759
K→E	0.822	0.800	0.821	0.718	0.788

源域→目标域	CDT	ASGCN	RAM	本文方法
平均值	0.778	0.769	0.736	0.795
B→D	0.745	0.742	0.726	0.756
B→E	0.782	0.805	0.647	0.812
B→K	0.764	0.774	0.751	0.748
D→B	0.738	0.727	0.739	0.796
D→E	0.781	0.741	0.775	0.783
D→K	0.752	0.735	0.706	0.786
E→B	0.728	0.764	0.713	0.797
E→D	0.836	0.782	0.750	0.856
E→K	0.832	0.830	0.784	0.848
K→B	0.773	0.781	0.732	0.806
K→D	0.751	0.784	0.730	0.759
K→E	0.820	0.763	0.781	0.788

源域→目标域	SCL-MI	ITIAD	DANN	GCAE-TL	本文方法
平均值	0.694	0.714	0.732	0.707	0.779
C→B	0.640	0.725	0.702	0.667	0.742
C→H	0.703	0.740	0.735	0.704	0.751
H→B	0.682	0.694	0.750	0.529	0.736
H→C	0.754	0.746	0.723	0.857	0.884
B→C	0.793	0.769	0.775	0.758	0.805
B→H	0.591	0.607	0.706	0.724	0.760

[1]	Jie HU, Yan HU, Mengchi LIU, Yan ZHANG. Chinese named entity recognition based on knowledge base entity enhanced BERT model [J]. Journal of Computer Applications, 2022, 42(9): 2680-2685.
[2]	Heng CHEN, Siyi WANG, Zhengguang LI, Guanyu LI, Xin LIU. Capsule network knowledge graph embedding model based on relational memory [J]. Journal of Computer Applications, 2022, 42(7): 1985-1992.
[3]	Jing JIANG, Yu CHEN, Jieping SUN, Shenggen JU. Integrating posterior probability calibration training into text classification algorithm [J]. Journal of Computer Applications, 2022, 42(6): 1789-1795.
[4]	Meng YU, Wentao HE, Xuchuan ZHOU, Mengtian CUI, Keqi WU, Wenjie ZHOU. Review of recommendation system [J]. Journal of Computer Applications, 2022, 42(6): 1898-1913.
[5]	Xueqin CHEN, Tao TAO, Zhongwang ZHANG, Yilei WANG. Long short-term session-based recommendation algorithm combining paired coding scheme and two-dimensional conventional neural network [J]. Journal of Computer Applications, 2022, 42(5): 1347-1354.
[6]	Xinrong HU, Junyu ZHANG, Tao PENG, Junping LIU, Ruhan HE, Kai HE. Cascaded cross-domain feature fusion for virtual try-on [J]. Journal of Computer Applications, 2022, 42(4): 1269-1274.
[7]	Jialiang HAN, Yudong HAN, Xuanzhe LIU, Yaoshuai ZHAO, Di FENG. Personalized recommendation service system based on cloud-client-convergence [J]. Journal of Computer Applications, 2022, 42(11): 3506-3512.
[8]	Xiaopeng WANG, Yuanyuan SUN, Hongfei LIN. Encoding-decoding relationship extraction model based on criminal Electra [J]. Journal of Computer Applications, 2022, 42(1): 87-93.
[9]	LIU Zichen, LI Xiaojuan, WEI Wei. Automatic patent price evaluation based on recurrent neural network [J]. Journal of Computer Applications, 2021, 41(9): 2532-2538.
[10]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[11]	ZHAO Xiaohu, LI Xiao. Image captioning algorithm based on multi-feature extraction [J]. Journal of Computer Applications, 2021, 41(6): 1640-1646.
[12]	MA Jianhong, CAO Wenbin, LIU Yuangang, XIA Shuang. Patent clustering method based on functional effect [J]. Journal of Computer Applications, 2021, 41(5): 1361-1366.
[13]	NI Shuiping, LI Huifang. Battery state-of-charge prediction method based on one-dimensional convolutional neural network combined with long short-term memory network [J]. Journal of Computer Applications, 2021, 41(5): 1514-1521.
[14]	CHE Bingqian, ZHOU Dong. Tag recommendation method combining network structure information and text content [J]. Journal of Computer Applications, 2021, 41(4): 976-983.
[15]	Qian LIU, Hongyuan WANG, Liang CAO, Boyan SUN, Yu XIAO, Ji ZHANG. Cloth-changing person re-identification based on joint loss capsule network [J]. Journal of Computer Applications, 2021, 41(12): 3596-3601.