基于深度学习的文本自动摘要方案

doi:10.11772/j.issn.1001-9081.2018081958

计算机应用 ›› 2019, Vol. 39 ›› Issue (2): 311-315.DOI: 10.11772/j.issn.1001-9081.2018081958

• 人工智能 • 下一篇

基于深度学习的文本自动摘要方案

张克君^1,2, 李伟男², 钱榕¹, 史泰猛¹, 焦萌¹

1. 北京电子科技学院计算机科学与技术系, 北京 100070;
2. 西安电子科技大学计算机科学与技术学院, 西安 710071

收稿日期:2018-09-20 修回日期:2018-11-14 出版日期:2019-02-10 发布日期:2019-02-15
通讯作者: 李伟男
作者简介:张克君(1972-),男,山东临沂人,副教授,博士,CCF会员,主要研究方向:信息安全、智能信息处理;李伟男(1994-),男,陕西西安人,硕士研究生,主要研究方向:自动摘要;钱榕(1970-),男,山东济南人,副教授,博士,CCF会员,主要研究方向:复杂网络、数据挖掘;史泰猛(1995-),男,河北衡水人,硕士研究生,主要研究方向:文本分类;焦萌(1994-),女,河北石家庄人,硕士研究生,主要研究方向:文本主题挖掘。
基金资助:
国家重点研发计划项目（2018YFB1004101）。

Automatic text summarization scheme based on deep learning

ZHANG Kejun^1,2, LI Weinan², QIAN Rong¹, SHI Taimeng¹, JIAO Meng¹

1. Department of Computer Science and Technology, Beijing Electronic Science and Technology Institute, Beijing 100070, China;
2. School of Computer Science and Technology, Xidian University, Xi'an Shaanxi 710071, China

Received:2018-09-20 Revised:2018-11-14 Online:2019-02-10 Published:2019-02-15
Supported by:
This work is partially supported by the National Key R&D Program of China (2018YFB1004101).

摘要/Abstract

摘要： 针对自然语言处理（NLP）生成式自动摘要领域的语义理解不充分、摘要语句不通顺和摘要准确度不够高的问题，提出了一种新的生成式自动摘要解决方案，包括一种改进的词向量生成技术和一个生成式自动摘要模型。改进的词向量生成技术以Skip-Gram方法生成的词向量为基础，结合摘要的特点，引入词性、词频和逆文本频率三个词特征，有效地提高了词语的理解；而提出的Bi-MulRnn+生成式自动摘要模型以序列映射（seq2seq）与自编码器结构为基础，引入注意力机制、门控循环单元（GRU）结构、双向循环神经网络（BiRnn）、多层循环神经网络（MultiRnn）和集束搜索，提高了生成式摘要准确性与语句流畅度。基于大规模中文短文本摘要（LCSTS）数据集的实验结果表明，该方案能够有效地解决短文本生成式摘要问题，并在Rouge标准评价体系中表现良好，提高了摘要准确性与语句流畅度。

关键词: 自然语言处理, 生成式文本自动摘要, 序列映射, 自编码器, 词向量, 循环神经网络

Abstract: Aiming at the problems of inadequate semantic understanding, improper summary sentences and inaccurate summary in the field of Natural Language Processing (NLP) abstractive automatic summarization, a new automatic summary solution was proposed, including an improved word vector generation technique and an abstractive automatic summarization model. The improved word vector generation technology was based on the word vector generated by the skip-gram method. Combining with the characteristics of abstract, three word features including part of speech, word frequency and inverse text frequency were introduced, which effectively improved the understanding of words. The proposed Bi-MulRnn+ abstractive automatic summarization model was based on sequence-to-sequence (seq2seq) framework and self-encoder structure. By introducing attention mechanism, Gated Recurrent Unit (GRU) gate structure, Bi-directional Recurrent Neural Network (BiRnn) and Multi-layer Recurrent Neural Network (MultiRnn), the model improved the summary accuracy and sentence fluency of abstractive summarization. The experimental results of Large-Scale Chinese Short Text Summarization (LCSTS) dataset show that the proposed scheme can effectively solve the problem of abstractive summarization of short text, and has good performance in Rouge standard evaluation system, improving summary accuracy and sentence fluency.

Key words: Natural Language Processing (NLP), abstractive automatic text summarization, sequence to sequence (seq2seq), self-encoder, word vector, Recurrent Neural Network (RNN)

中图分类号:

张克君, 李伟男, 钱榕, 史泰猛, 焦萌. 基于深度学习的文本自动摘要方案[J]. 计算机应用, 2019, 39(2): 311-315.

ZHANG Kejun, LI Weinan, QIAN Rong, SHI Taimeng, JIAO Meng. Automatic text summarization scheme based on deep learning[J]. Journal of Computer Applications, 2019, 39(2): 311-315.

参考文献

[1] BAHDANAU D, CHO K H, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL].[2018-03-20]. https://arxiv.org/pdf/1409.0473v7.pdf.
[2] BAHDANAU D, CHOROWSKI J, SERDYUK D, et al. End-to-end attention-based large vocabulary speech recognition[C]//Proceedings of the 2016 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway, NJ:IEEE, 2016:4945-4949.
[3] VENUGOPALAN S, ROHRBACH M, DONAHUE J, et al. Sequence to sequence-video to text[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:4534-4542.
[4] RUSH A M, CHOPRA S, WESTON J. A neural attention model for abstractive sentence summarization[EB/OL].[2018-02-23]. https://arxiv.org/pdf/1509.00685.pdf.
[5] CHOPRA S, AULI M, RUSH A M. Abstractive sentence summarization with attentive recurrent neural networks[EB/OL].[2018-03-21] http://aclweb.org/anthology/N/N16/N16-1012.pdf.
[6] NALLAPATI R, ZHOU B W, dos SANTOS C N, et al. Abstractive text summarization using sequence-to-sequence RNNs and beyond[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. Stroudsburg, PA:ACL, 2016:280-290.
[7] ABADI M, BARHAM P, CHEN J M, et al. Tensor flow:a system for large-scale machine learning[C]//Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation. Berkeley, CA:USENIX, 2016:265-283.
[8] BRITZ D,GOLDIE A, LUONG M-T, et al. Massive exploration of neural machine translation architectures[EB/OL].[2018-04-05]. https://arxiv.org/pdf/1703.03906.pdf.
[9] GEHRING J, AULI M, GRANGIER D, et al. Convolutional sequence to sequence learning[EB/OL].[2018-04-23]. https://arxiv.org/pdf/1705.03122.pdf.
[10] LI P J, LAM W, BING L D, et al. Cascaded attention based unsupervised information distillation for compressive summarization[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:ACL, 2017:2081-2090.
[11] CHUNG J Y, GULCEHRE C, CHO K H, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL].[2018-04-23]. https://arxiv.org/pdf/1412.3555v1.pdf.
[12] LOPYREV K. Generating news headlines with recurrent neural networks[EB/OL].[2018-03-20]. https://arxiv.org/pdf/1512.01712.pdf.
[13] MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention[EB/OL].[2018-04-08]. https://papers.nips.cc/paper/5542-recurrent-models-of-visual-attention.pdf.
[14] LUONG M-T, PHAM H, MANNING C D. Effective approaches to attention-based neural machine translation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:ACL, 2015:1412-1421.
[15] JEAN S, CHO K H, MEMISEVIC R, et al. On using very large target vocabulary for neural machine translation[C]//Proceedings of the 53rd Annual Meeting of the ACL and the 7th International Joint Conference on Natural Language Processing. Stroudsburg, PA:ACL, 2015:1-10.
[16] AYANA, SHEN S Q, ZHAO Y, et al. Neural headline generation with sentence-wise optimization[EB/OL].[2018-03-23]. https://arxiv.org/pdf/1604.01904.pdf.
[17] LIN C Y, HOVY E. Automatic evaluation of summaries using n-gram co-occurrence statistics[C]//Proceedings of the 2003 Conference of the North American Chapter of the ACL on Human Language Technology. Stroudsburg, PA:ACL, 2003:71-78.
[18] 户保田.基于深度神经网络的文本表示及其应用[D].哈尔滨:哈尔滨工业大学,2016:91-94. (HU B T. Deep neural networks for text representation and application[D]. Harbin:Harbin Institute of Technology, 2016:91-94.)
[19] HU B T, CHEN Q C, ZHU F Z. LCSTS:A large scale Chinese short text summarization dataset[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:ACL, 2015:1967-1972.

基于深度学习的文本自动摘要方案

Automatic text summarization scheme based on deep learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[2]	刘雅璇, 钟勇. 基于头实体注意力的实体关系联合抽取方法[J]. 计算机应用, 2021, 41(9): 2517-2522.
[3]	刘子辰, 李小娟, 韦伟. 基于循环神经网络的专利价格自动评估[J]. 计算机应用, 2021, 41(9): 2532-2538.
[4]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[5]	周险兵, 樊小超, 任鸽, 杨勇. 基于多层次语义特征的英文作文自动评分方法[J]. 计算机应用, 2021, 41(8): 2205-2211.
[6]	王伟, 赵尔平, 崔志远, 孙浩. 基于HowNet义原和Word2vec词向量表示的多特征融合消歧方法[J]. 计算机应用, 2021, 41(8): 2193-2198.
[7]	丁尹, 桑楠, 李晓瑜, 吴飞舟. 基于循环神经网络的电信行业容量数据预测方法[J]. 计算机应用, 2021, 41(8): 2373-2378.
[8]	张元钧, 张曦煌. 基于图卷积与长短期记忆网络的动态网络表示学习模型[J]. 计算机应用, 2021, 41(7): 1857-1864.
[9]	赵小虎, 李晓. 基于多特征提取的图像语义描述算法[J]. 计算机应用, 2021, 41(6): 1640-1646.
[10]	李文惠, 曾上游, 王金金. 基于改进注意力机制的图像描述生成算法[J]. 计算机应用, 2021, 41(5): 1262-1267.
[11]	刘睿珩, 叶霞, 岳增营. 面向自然语言处理任务的预训练模型综述[J]. 计算机应用, 2021, 41(5): 1236-1246.
[12]	孙鹤立, 孙玉柱, 张晓云. 基于生成对抗网络的事件描述生成[J]. 计算机应用, 2021, 41(5): 1256-1261.
[13]	倪水平, 李慧芳. 基于一维卷积神经网络与长短期记忆网络结合的电池荷电状态预测方法[J]. 计算机应用, 2021, 41(5): 1514-1521.
[14]	王朱君, 王石, 李雪晴, 朱俊武. 基于深度学习的事件因果关系抽取综述[J]. 计算机应用, 2021, 41(5): 1247-1255.
[15]	李雪晴, 王石, 王朱君, 朱俊武. 自然语言生成综述[J]. 计算机应用, 2021, 41(5): 1227-1235.