基于指针网络的抽取生成式摘要生成模型

doi:10.11772/j.issn.1001-9081.2021060899

《计算机应用》唯一官方网站 ›› 2021, Vol. 41 ›› Issue (12): 3527-3533.DOI: 10.11772/j.issn.1001-9081.2021060899

所属专题：第十八届中国机器学习会议(CCML 2021)

• 第十八届中国机器学习会议(CCML 2021) • 上一篇下一篇

基于指针网络的抽取生成式摘要生成模型

陈伟, 杨燕()

西南交通大学计算机与人工智能学院，成都 611756

收稿日期:2021-05-12 修回日期:2021-06-24 接受日期:2021-06-24 发布日期:2021-08-20 出版日期:2021-12-10
通讯作者: 杨燕
作者简介:陈伟（1996—），男，四川内江人，硕士研究生，主要研究方向：自然语言处理；
基金资助:
国家自然科学基金面上项目(61976247)

Extractive and abstractive summarization model based on pointer-generator network

Wei CHEN, Yan YANG()

School of Computing and Artificial Intelligence，Southwest Jiaotong University，Chengdu Sichuan 611756，China

Received:2021-05-12 Revised:2021-06-24 Accepted:2021-06-24 Online:2021-08-20 Published:2021-12-10
Contact: Yan YANG
About author:CHEN Wei， born in 1996， M. S. candidate. His research interests include natural language processing.
Supported by:
the Surface Program of National Natural Science Foundation of China(61976247)

摘要/Abstract

摘要：

作为自然语言处理中的热点问题，摘要生成具有重要的研究意义。基于Seq2Seq模型的生成式摘要模型取得了良好的效果，然而抽取式的方法具有挖掘有效特征并抽取文章重要句子的潜力，因此如何利用抽取式方法来改进生成式方法是一个较好的研究方向。鉴于此，提出了融合生成式和抽取式方法的模型。首先，使用TextRank算法并融合主题相似度来抽取文章中有重要意义的句子。然后，设计了融合抽取信息语义的基于Seq2Seq模型的生成式框架来实现摘要生成任务；同时，引入指针网络解决模型训练中的未登录词（OOV）问题。综合以上步骤得到最终摘要，并在CNN/Daily Mail数据集上进行验证。结果表明在ROUGE-1、ROUGE-2和ROUGE-L三个指标上所提模型比传统TextRank算法均有所提升，同时也验证了融合抽取式和生成式方法在摘要生成领域中的有效性。

关键词: 抽取生成式摘要, TextRank算法, Seq2Seq模型, 指针网络, 语义融合

Abstract:

As a hot issue in natural language processing， summarization generation has important research significance. The abstractive method based on Seq2Seq （Sequence-to-Sequence） model has achieved good results， however， the extractive method has the potential of mining effective features and extracting important sentences of articles， so it is a good research direction to improve the abstractive method by using extractive method. In view of this， a fusion model of abstractive method and extractive method was proposed. Firstly， incorporated with topic similarity， TextRank algorithm was used to extract significant sentences from the article. Then， an abstractive framework based on the Seq2Seq model integrating the semantics of extracted information was designed to implement the summarization task； at the same time， pointer-generator network was introduced to solve the problem of Out-Of-Vocabulary （OOV）. Based on the above steps， the final summary was obtained and verified on the CNN/Daily Mail dataset. The results show that on all the three indexes ROUGE-1， ROUGE-2 and ROUGE-L， the proposed model is better than the traditional TextRank algorithm； meanwhile， the effectiveness of fusing extractive method and abstractive method in the field of summarization is also verified.

Key words: extractive and abstractive summarization, TextRank algorithm, Seq2Seq (Sequence-to-Sequence) model, pointer-generator network, semantic fusion

中图分类号:

TP183

陈伟, 杨燕. 基于指针网络的抽取生成式摘要生成模型[J]. 计算机应用, 2021, 41(12): 3527-3533.

Wei CHEN, Yan YANG. Extractive and abstractive summarization model based on pointer-generator network[J]. Journal of Computer Applications, 2021, 41(12): 3527-3533.

图/表 4

图1 抽取生成式摘要生成模型

Fig. 1 Extractive and abstractive summarization model

表1 在CNN/Daily Mail数据集上的实验结果 ( %)

Tab. 1 Experimental results on CNN/Daily Mail dataset

id	模型	ROUGE-1	ROUGE-2	ROUGE-L	id	模型	ROUGE-1	ROUGE-2	ROUGE-L
1	Seq2seq+Attn	31.50	11.95	28.85	6	WordNet+Dual-attn+PGN	36.91	15.97	33.58
2	Seq2seq+Attn（150k）	30.67	11.32	28.11	7	TextRank	37.21	15.25	33.60
3	Seq2seq+Attn+PGN	36.58	15.76	33.33	8	Graph-Based Attentional Neural Model	38.10	13.90	28.70
4	ABS	31.33	11.81	28.83	9	SummaRuNNer	39.60	16.20	35.30
5	Lead-3+Dual-attn+PGN	37.26	16.12	33.87	10	本文模型	38.95	16.90	35.33

表2 不同模型在P、R、F指标上的对比结果 ( %)

Tab. 2 Comparison results of different models on indexes P， R， F

模型	评价指标	ROUGE-1	ROUGE-2	ROUGE-L	模型	评价指标	ROUGE-1	ROUGE-2	ROUGE-L
TextRank	$P$	30.56	12.47	27.53	本文模型	$P$	38.83	17.01	35.26
	$R$	52.3 0	21.62	47.21		$R$	42.19	18.40	38.28
	$F$	37.21	15.25	33.60		$F$	38.95	16.90	35.33

表2 不同模型在P、R、F指标上的对比结果 ( %)

Tab. 2 Comparison results of different models on indexes P， R， F

模型	评价指标	ROUGE-1	ROUGE-2	ROUGE-L	模型	评价指标	ROUGE-1	ROUGE-2	ROUGE-L
TextRank	$P$	30.56	12.47	27.53	本文模型	$P$	38.83	17.01	35.26
	$R$	52.3 0	21.62	47.21		$R$	42.19	18.40	38.28
	$F$	37.21	15.25	33.60		$F$	38.95	16.90	35.33

表3 文本摘要生成实例

Tab. 3 Text summarization instances

模型	实例内容
Original text_1	on the 6th of april 1996， san jose clash and dc united strode out in front of 31 683 expectant fans at the spartan stadium in san jose， california. the historic occasion was the first ever major league soccer match — a brave new dawn for the world ’s favorite sport in a land its charms had yet to conquer. summarizing the action for espn， commentator ty keough eagerly described the momentous "birth of a new era for american soccer." looking back at footage from that balmy evening now it’s hard not to feel a certain nostalgia. baggy shirts， questionable hairstyles and strange rule adaptations to make games more exciting were all part of the formative mls experience. countdown clocks were employed to provide drama at the end of each half …
Reference summary	the 20th mls season begins this weekend. league has changed dramatically since its inception in 1996. some question whether rules regarding salary caps and transfers need to change.
TextRank	according to phil rawlins， co-primary owner and president of the new mls franchise， orlando city soccer club， "the industry and the game itself has moved on dramatically" in the u.s. although the likes of the salary cap encourages fiscal probity， it means mls teams are restricted by rules clubs in other markets are not. according to paulo teixeira， a football agent who has worked to bring in and sell players from the league in recent years， current philosophies with regards player-trading may be have to be tweaked to help the mls grow yet further.
本文模型	san jose clash and dc united strode out in front of 31 683 expectant fans at the spartan stadium in san jose， california. the historic occasion was the first ever major league soccer match — a brave new dawn for the world ’s favorite sport.
Original text_2	french striker bafetimbi gomis， who has a history of fainting， said he is now "feeling well" after collapsing during swansea ’s 3-2 loss at tottenham in the premier league on wednesday. the worrying incident occurred in the first half at white hart lane — after tottenham scored in the seventh minute — but the 29-year-old left the pitch conscious following about five minutes of treatment. the guardian added that he was wearing an oxygen mask. play was temporarily stopped before resuming. as the match progressed， swansea tweeted that gomis was "fine，" with manager garry monk using the same word to describe gomis ’ condition. gomis spent the night in hospital as a precaution， swansea said on its website. "…
Reference summary	bafetimbi gomis collapses within 10 minutes of kickoff at tottenham. but he reportedly left the pitch conscious and wearing an oxygen mask. gomis later said that he was "feeling well" the incident came three years after fabrice muamba collapsed at white hart lane.
TextRank	french striker bafetimbi gomis， who has a history of fainting， said he is now "feeling well" after collapsing during swansea ’s 3-2 loss at tottenham in the premier league on wednesday. gomis spent the night in hospital as a precaution， swansea said on its website." swansea ran tests on gomis， said monk， prior to signing him on a free transfer last july ." gomis has scored two league goals for swansea this season， mostly in a backup role.
本文模型	french striker bafetimbi gomis， who has a history of fainting， said he is now "feeling well" after collapsing during swansea ’s 3-2 loss at tottenham in the premier league on wednesday. the worrying incident occurred in the first half at white hart lane — after tottenham scored in the seventh minute.

表3 文本摘要生成实例

Tab. 3 Text summarization instances

模型	实例内容
Original text_1	on the 6th of april 1996， san jose clash and dc united strode out in front of 31 683 expectant fans at the spartan stadium in san jose， california. the historic occasion was the first ever major league soccer match — a brave new dawn for the world ’s favorite sport in a land its charms had yet to conquer. summarizing the action for espn， commentator ty keough eagerly described the momentous "birth of a new era for american soccer." looking back at footage from that balmy evening now it’s hard not to feel a certain nostalgia. baggy shirts， questionable hairstyles and strange rule adaptations to make games more exciting were all part of the formative mls experience. countdown clocks were employed to provide drama at the end of each half …
Reference summary	the 20th mls season begins this weekend. league has changed dramatically since its inception in 1996. some question whether rules regarding salary caps and transfers need to change.
TextRank	according to phil rawlins， co-primary owner and president of the new mls franchise， orlando city soccer club， "the industry and the game itself has moved on dramatically" in the u.s. although the likes of the salary cap encourages fiscal probity， it means mls teams are restricted by rules clubs in other markets are not. according to paulo teixeira， a football agent who has worked to bring in and sell players from the league in recent years， current philosophies with regards player-trading may be have to be tweaked to help the mls grow yet further.
本文模型	san jose clash and dc united strode out in front of 31 683 expectant fans at the spartan stadium in san jose， california. the historic occasion was the first ever major league soccer match — a brave new dawn for the world ’s favorite sport.
Original text_2	french striker bafetimbi gomis， who has a history of fainting， said he is now "feeling well" after collapsing during swansea ’s 3-2 loss at tottenham in the premier league on wednesday. the worrying incident occurred in the first half at white hart lane — after tottenham scored in the seventh minute — but the 29-year-old left the pitch conscious following about five minutes of treatment. the guardian added that he was wearing an oxygen mask. play was temporarily stopped before resuming. as the match progressed， swansea tweeted that gomis was "fine，" with manager garry monk using the same word to describe gomis ’ condition. gomis spent the night in hospital as a precaution， swansea said on its website. "…
Reference summary	bafetimbi gomis collapses within 10 minutes of kickoff at tottenham. but he reportedly left the pitch conscious and wearing an oxygen mask. gomis later said that he was "feeling well" the incident came three years after fabrice muamba collapsed at white hart lane.
TextRank	french striker bafetimbi gomis， who has a history of fainting， said he is now "feeling well" after collapsing during swansea ’s 3-2 loss at tottenham in the premier league on wednesday. gomis spent the night in hospital as a precaution， swansea said on its website." swansea ran tests on gomis， said monk， prior to signing him on a free transfer last july ." gomis has scored two league goals for swansea this season， mostly in a backup role.
本文模型	french striker bafetimbi gomis， who has a history of fainting， said he is now "feeling well" after collapsing during swansea ’s 3-2 loss at tottenham in the premier league on wednesday. the worrying incident occurred in the first half at white hart lane — after tottenham scored in the seventh minute.

参考文献 22

1	MIHALCEA R， TARAU P. TextRank： bringing order into texts［C］// Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2004： 404-411. 10.3115/1220355.1220517
2	SUTSKEVER I， VINYALS O， LE Q V. Sequence to sequence learning with neural networks［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2014： 3104-3112．
3	BAHDANAU D， CHO K， BENGIO Y. Neural machine translation by jointly learning to align and translate［EB/OL］. （2016-05-19）［2021-01-06］.． 10.3115/v1/w14-4009
4	NALLAPATI R， ZHOU B W， SANTOS C DOS， et al. Abstractive text summarization using sequence-to-sequence RNNs and beyond［C］// Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning. Stroudsburg， PA： ACL， 2016： 280-290． 10.18653/v1/k16-1028
5	RUSH A M， CHOPRA S， WESTON J. A neural attention model for sentence summarization［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2015： 379-389. 10.18653/v1/d15-1044
6	ZHOU Q Y， YANG N， WEI F R， et al. Selective encoding for abstractive sentence summarization［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2017： 1095-1104. 10.18653/v1/p17-1101
7	TAN J W， WAN X J， XIAO J G. Abstractive document summarization with a graph based attentional neural model［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2017：1171-1181. 10.18653/v1/p17-1108
8	GU J T， LU Z D， LI H， et al. Incorporating copying mechanism in sequence-to-sequence learning［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2016：1631-1640. 10.18653/v1/p16-1154
9	GULCEHRE C， AHN S， NALLAPATI R， et al. Pointing the unknown words［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2016：140-149. 10.18653/v1/p16-1014
10	ZENG W Y， LUO W J， FIDLER S， et al. Efficient summarization with read-again and copy mechanism［EB/OL］. （2016-11-10）［2021-01-06］..
11	SEE A， LIU P J， MANNING C D. Get to the point： summarization with pointer-generator networks［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2017： 1073-1083． 10.18653/v1/p17-1099
12	TU Z P， LU Z D， LIU Y， et al. Modeling coverage for neural machine translation［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2016：76-85. 10.18653/v1/p16-1008
13	NEMA P， KHAPRA M M， LAHA A， et al. Diversity driven attention model for query-based abstractive summarization［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2017： 1063-1072. 10.18653/v1/p17-1098
14	FILIPPOVA K， STRUBE M. Dependency tree based sentence compression［C］// Proceedings of the 5th International Natural Language Generation Conference. Stroudsburg， PA： ACL， 2008： 25-32. 10.3115/1708322.1708329
15	MA S M， SUN X， XU J J， et al. Improving semantic relevance for sequence-to-sequence learning of Chinese social media text summarization［C］// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg， PA： ACL， 2017： 635-640. 10.18653/v1/p17-2100
16	CHENG J P， LAPATA M. Neural summarization by extracting sentences and words［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2016： 484-494. 10.18653/v1/p16-1046
17	NALLAPATI R， ZHAI F F， ZHOU B W. SummaRuNNer： a recurrent neural network based sequence model for extractive summarization of documents［C］// Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2017： 3075-3081.
18	VINYALS O， FORTUNATO M， JAITLY N. Pointer networks［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2015： 2692-2700．
19	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. （2017-01-30）［2021-01-06］..
20	HERMANN K M， KOČISKÝ T， GREFENSTETTE E， et al. Teaching machines to read and comprehend［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2015： 1693-1701． 10.18653/v1/d16-1116
21	NG J P， ABRECHT V. Better summarization evaluation with word embeddings for ROUGE［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2015： 1925-1930． 10.18653/v1/d15-1222
22	XIE N T， LI S J， REN H L， et al. Abstractive summarization improved by WordNet-based extractive sentences［C］// Proceedings of the 7th CCF International Conference on Natural Language Processing and Chinese Computing， LNCS11108. Cham： Springer， 2018： 404-415.

[1]	赵征宇, 罗景, 涂新辉. 基于多粒度语义融合的信息检索方法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1775-1780.
[2]	田悦霖, 黄瑞章, 任丽娜. 融合局部语义特征的学者细粒度信息提取方法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2707-2714.
[3]	徐郁, 朱韵攸, 刘筱, 邓雨婷, 廖勇. 基于深度强化学习的电力物资配送多目标路径优化[J]. 《计算机应用》唯一官方网站, 2022, 42(10): 3252-3258.
[4]	林志兴, 王立可. 基于深度特征和Seq2Seq模型的网络态势预测方法[J]. 计算机应用, 2020, 40(8): 2241-2247.

基于指针网络的抽取生成式摘要生成模型

Extractive and abstractive summarization model based on pointer-generator network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 4

参考文献 22

相关文章 4

编辑推荐

Metrics