基于自注意力门控图卷积网络的特定目标情感分析

doi:10.11772/j.issn.1001-9081.2019122154

计算机应用 ›› 2020, Vol. 40 ›› Issue (8): 2202-2206.DOI: 10.11772/j.issn.1001-9081.2019122154

基于自注意力门控图卷积网络的特定目标情感分析

陈佳伟, 韩芳, 王直杰

东华大学信息科学与技术学院, 上海 201620

收稿日期:2019-12-24 修回日期:2020-02-27 出版日期:2020-08-10 发布日期:2020-05-13
通讯作者: 韩芳(1981-),女,山西朔州人,教授,博士,主要研究方向:智能系统、神经动力学,yadiahan@dhu.edu.cn
作者简介:陈佳伟(1994-),男,江苏南通人,硕士研究生,主要研究方向:自然语言处理、机器学习;王直杰(1969-),男,浙江台州人,教授,博士,主要研究方向:神经网络、数字信号处理、工业控制软件。
基金资助:
国家自然科学基金资助项目（11972115，11572084）。

Aspect-based sentiment analysis with self-attention gated graph convolutional network

CHEN Jiawei, HAN Fang, WANG Zhijie

College of Information Science and Technology, Donghua University, Shanghai 201620, China

Received:2019-12-24 Revised:2020-02-27 Online:2020-08-10 Published:2020-05-13
Supported by:
This work is partially supported by the National Natural Science Foundation of China (11972115, 11572084).

摘要/Abstract

摘要： 基于特定目标的情感分析旨在预测句子中不同方面表达的不同情感倾向。针对之前利用循环神经网络（RNN）结合注意力机制的网络模型所带来的训练参数多且缺少对相关句法约束和长距离词依赖机制解释的问题，提出自注意力门控图卷积网络MSAGCN。首先，模型采用多头自注意力机制编码上下文词和目标，捕获句子内部的语义关联；然后，采用在句子的依存树上建立图卷积网络的方法获取句法信息以及词的依存关系；最后，通过带有目标嵌入的门控单元（GTRU）获取特定目标的情感。与基线模型相比，所提模型的准确率和调和平均值F1分别提高了1%~3.3%和1.4%~6.3%；同时，预训练的BERT模型也被应用到当前任务中，使模型效果获得了新的提升。实验结果表明所提出的模型能更好掌握用户评论的情感倾向。

关键词: 特定目标情感分析, 自注意力机制, 图卷积网络, 门控机制, BERT

Abstract: Aspect-based sentiment analysis tries to estimate different emotional tendencies expressed in different aspects of a sentence. Aiming at the problem that the existing network model based on Recurrent Neural Network (RNN) combined with attention mechanism has too many training parameters and lacks explanation of related syntax constraints and long distance word dependence mechanism, a self-attention gated graph convolutional network was proposed, namely MSAGCN. First, the multi-headed self-attention mechanism was used to encode context words and targets, thus capturing semantic associations within the sentence. Then, a graph convolutional network was established on the sentence's dependency tree to obtain syntactic information and word dependencies. Finally, the sentiment of the specific target was obtained through the GTRU (Gated Tanh-ReLU Unit). Compared with the baseline model, the proposed model has the accuracy and F1 improved by 1%-3.3% and 1.4%-6.3% respectively. At the same time, the pre-trained Bidirectional Encoder Representations from Transformers (BERT) model was also applied to the current task to further improve the model effect. Experimental results verify that the proposed model can better grasp the emotional tendencies of user reviews.

Key words: aspect-based sentiment analysis, self-attention mechanism, Graph Convolutional Network (GCN), gating mechanism, Bidirectional Encoder Representations from Transformers (BERT)

中图分类号:

TP389.1

陈佳伟, 韩芳, 王直杰. 基于自注意力门控图卷积网络的特定目标情感分析[J]. 计算机应用, 2020, 40(8): 2202-2206.

CHEN Jiawei, HAN Fang, WANG Zhijie. Aspect-based sentiment analysis with self-attention gated graph convolutional network[J]. Journal of Computer Applications, 2020, 40(8): 2202-2206.

参考文献

[1] 周立柱,贺宇凯,王建勇. 情感分析研究综述[J]. 计算机应用, 2008, 28(11):2725-2728. (ZHOU L Z, HE Y K, WANG J Y. Survey on research of sentiment analysis[J]. Journal of Computer Applications, 2008, 28(11):2725-2728.)
[2] KIRITCHENKO S, ZHU X, CHERRY C, et al. NRC-Canada-2014:Detecting aspects and sentiment in customer reviews[C]//Proceedings of the 8th International Workshop on Semantic Evaluation. Stroudsburg:Association for Computational Linguistics, 2014:437-442.
[3] VO D T, ZHANG Y. Target-dependent twitter sentiment classification with rich automatic features[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press, 2015:1347-1353.
[4] BENGIO Y, DUCHARME R, VINCENT P, et al. A neural probabilistic language model[J]. The Journal of Machine Learning Research, 2003, 3:1137-1155.
[5] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc., 2013:3111-3119.
[6] LE Q, MIKOLOV T. Distributed representations of sentences and documents[C]//Proceedings of the 31st International Conference on Machine Learning. New York:JMLR.org, 2014:1188-1196.
[7] PENNINGTON J, SOCHER R, MANNING C. Glove:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2014:1532-1543.
[8] TANG D, QIN B, FENG X, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of the 26th International Conference on Computational Linguistic. Stroudsburg:Association for Computational Linguistics, 2016:3298-3307.
[9] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL].[2019-11-24]. https://arxiv.org/pdf/1409.0473.pdf.
[10] WANG Y, HUANG M, ZHU X, et al. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg:Association for Computational Linguistics, 2016:606-615.
[11] MA D, LI S, ZHANG X, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Palo Alto, CA:AAAI Press, 2017:4068-4074.
[12] 李丽双,周安桥,刘阳,等. 基于动态注意力GRU的特定目标情感分类[J]. 中国科学:信息科学, 2019(8):1019-1030. (LI L S, ZHOU A Q, LIU Y, et al. Aspect-based sentiment analysis based on dynamic attention GRU[J]. SCIENTIA SINICA Informationis, 2019(8):1019-1030.)
[13] LI X, BING L, LAM Y, et al. Transformation networks for target-oriented sentiment classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2018:946-956.
[14] 粱斌,刘全,徐进,等. 基于多注意力卷积神经网络的特定目标情感分析[J]. 计算机研究与发展, 2017, 54(8):1724-1735. (LIANG B, LIU Q, XU J, et al. Aspect-based sentiment analysis based on multi-attention CNN[J]. Journal of Computer Research and Development, 2017, 54(8):1724-1735.)
[15] XUE W, LI T. Aspect based sentiment analysis with gated convolutional networks[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2018:2514-2523.
[16] HE R, WEE S L, NG H T, et al. Effective attention modeling for aspect-level sentiment classification[C]//Proceedings of the 27th International Conference on Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2018:1121-1131.
[17] DEVLIN J, CHANG M W, LEE K, et al. BERT:pre-training of deep bidirectional transformers for language understanding[EB/OL].[2019-10-24].https://arxiv.org/pdf/1810.04805.pdf.
[18] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Cambridge:MIT Press, 2017:6000-6010.
[19] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL].[2019-10-25].https://arxiv.org/pdf/1609.02907.pdf.
[20] PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval-2014 task4:aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation. Stroudsburg:Association for Computational Linguistics, 2014:27-35.
[21] DONG L, WEI F, TAN C, et al. Adaptive recursive neural network for target-dependent twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg:Association for Computational Linguistics, 2014:49-54.
[22] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[23] ZHANG C, LI Q, SONG D. Aspect-based sentiment classification with aspect-specific graph convolutional networks[EB/OL].[2019-11-13].https://arxiv.org/pdf/1909.03477.pdf.
[24] SONG Y, WANG J, JIANG T, et al. Attentional encoder network for targeted sentiment classification[EB/OL].[2019-11-15].https://arxiv.org/pdf/1902.09314.pdf.

基于自注意力门控图卷积网络的特定目标情感分析

Aspect-based sentiment analysis with self-attention gated graph convolutional network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	党伟超, 李涛, 白尚旺, 高改梅, 刘春霞. 基于自注意力长短期记忆网络的Web软件系统实时剩余寿命预测方法[J]. 计算机应用, 2021, 41(8): 2346-2351.
[2]	张元钧, 张曦煌. 基于图卷积与长短期记忆网络的动态网络表示学习模型[J]. 计算机应用, 2021, 41(7): 1857-1864.
[3]	李扬志, 袁家政, 刘宏哲. 基于时空注意力图卷积网络模型的人体骨架动作识别算法[J]. 计算机应用, 2021, 41(7): 1915-1921.
[4]	温超东, 曾诚, 任俊伟, 张. 结合ALBERT和双向门控循环单元的专利文本分类[J]. 计算机应用, 2021, 41(2): 407-412.
[5]	王晓霞, 钱雪忠, 宋威. 基于注意力与图卷积网络的关系抽取模型[J]. 计算机应用, 2021, 41(2): 350-356.
[6]	许力, 李建华. 基于句法依存分析的图网络生物医学命名实体识别[J]. 计算机应用, 2021, 41(2): 357-362.
[7]	沈子懿, 王卫亚, 蒋东华, 荣宪伟. 基于Hopfield混沌神经网络和压缩感知的可视化图像加密算法[J]. 计算机应用, 2021, 41(10): 2893-2899.
[8]	杨璐, 何明祥. 基于门控机制和卷积神经网络的中文文本情感分析模型[J]. 计算机应用, 2021, 41(10): 2842-2848.
[9]	罗俊, 陈黎飞. 基于BERT的不完全数据情感分类[J]. 计算机应用, 2021, 41(1): 139-144.
[10]	李生武, 张选德. 基于自注意力机制的多域卷积神经网络的视觉追踪[J]. 计算机应用, 2020, 40(8): 2219-2224.
[11]	杨国峰, 杨勇. 基于BERT的常见作物病害问答系统问句分类[J]. 计算机应用, 2020, 40(6): 1580-1586.
[12]	许一宁, 何小海, 张津, 卿粼波. 基于多层次分辨率递进生成对抗网络的文本生成图像方法[J]. 计算机应用, 2020, 40(12): 3612-3617.
[13]	张小川, 戴旭尧, 刘璐, 冯天硕. 融合多头自注意力机制的中文短文本分类模型[J]. 计算机应用, 2020, 40(12): 3485-3489.
[14]	黄伟建, 李丹阳, 黄远. 面向空气质量的时空混合预测模型[J]. 计算机应用, 2020, 40(11): 3385-3392.
[15]	陈可佳, 杨泽宇, 刘峥, 鲁浩. 基于邻域选择策略的图卷积网络模型[J]. 计算机应用, 2019, 39(12): 3415-3419.