基于多重映射的自动短文摘方法

doi:10.11772/j.issn.1001-9081.2016.02.0432

计算机应用 ›› 2016, Vol. 36 ›› Issue (2): 432-436.DOI: 10.11772/j.issn.1001-9081.2016.02.0432

• 第三届CCF大数据学术会议(CCF BigData 2015) • 上一篇下一篇

基于多重映射的自动短文摘方法

卢玲, 杨武, 曹琼

重庆理工大学计算机科学与工程学院, 重庆 400054

收稿日期:2015-09-15 修回日期:2015-09-22 出版日期:2016-02-10 发布日期:2016-02-03
通讯作者: 卢玲(1975-),女,重庆人,副教授,硕士,CCF会员,主要研究方向:机器学习、数据挖掘、并行计算。
作者简介:杨武(1965-),男,重庆人,教授,CCF会员,主要研究方向:信息检索、机器学习;曹琼(1979-),女,重庆人,讲师,硕士,主要研究方向:信息检索、数据挖掘。
基金资助:
重庆市教委科学技术研究项目(KJ1500903)。

Automatic short text summarization method based on multiple mapping

LU Ling, YANG Wu, CAO Qiong

College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400050, China

Received:2015-09-15 Revised:2015-09-22 Online:2016-02-10 Published:2016-02-03

摘要/Abstract

摘要： 传统自动文摘一般对字数没有明确限制,运用传统技术进行短文摘提取时,受字数限制,难以获取均衡的性能。针对该问题,提出一种多重映射的自动短文摘方法。通过计算关联度映射值、长度映射值、标题映射值和位置映射值,分别形成多个候选文摘句子集;再运用多重映射策略,将多个候选子集映射到文摘句子集中,同时使用提取文本中心句的方法提高召回率。实验表明,多重映射可在短文摘提取上获得稳定的性能。在NLP&CC2015评测中,该方法的ROUGE-1测试F值达到0.49,ROUGE-2测试F值达到0.35,均优于评测的平均水平,表明了该方法的有效性。

关键词: 自动文摘, 短文摘, 多重映射, 映射规则

Abstract: Traditional automatic text summarization has generally no word count requirements while many social network platforms have word count limitation. Balanced performance is hardly obtained in short text summarization by traditional digest technology because of the limitation of word count. In view of this problem, a new automatic short text summarization method was proposed. Firstly, the values of relationship mapping, length mapping, title mapping and position mapping were calculated to respectively form some sets of candidate sentences. Secondly, the candidate sentences sets were mapped to abstract sentences set by multiple mapping strategies according to series of multiple mapping rules, and the recall ratio was increased by putting central sentences into the set of abstract sentences. The experimental results show that multiple mappings can obtain stable performance in short text summarization, the F measures of ROUGE-1 and ROUGE-2 tests are 0.49 and 0.35 respectively, which are better than the average level of NLP&CC2015 evaluation, proving the effectiveness of the method.

Key words: automatic summarization, short text summarization, multiple mapping, mapping rule

中图分类号:

TP391.1

卢玲, 杨武, 曹琼. 基于多重映射的自动短文摘方法[J]. 计算机应用, 2016, 36(2): 432-436.

LU Ling, YANG Wu, CAO Qiong. Automatic short text summarization method based on multiple mapping[J]. Journal of Computer Applications, 2016, 36(2): 432-436.

参考文献

[1] 蒋效宇.基于关键词抽取的自动文摘算法[J].计算机工程,2012,38(3):183-186. (JIANG X Y. Automatic summarization algorithm based on keyword extraction[J]. Computer Engineering, 2012, 38(3): 183-186.)
[2] 王红玲,周国栋,朱巧明.面向冗余度控制的中文多文档自动文摘[J].中文信息学报,2012,26(2):92-96. (WANG H L, ZHOU G D, ZHU Q M. Chinese multi document summarization based on redundancy control[J]. Journal of Chinese Information Processing, 2012, 26(2): 92-96.)
[3] 曹洋,成颖,裴雷.基于机器学习的自动文摘研究综述[J].图书情报工作,2014,58(18):122-130. (CAO Y, CHENG Y, PEI L. A review on machine learning oriented automatic summarization[J]. Library and Information Service, 2014, 58(18): 122-130.)
[4] 郭庆琳,樊孝忠,柳长安.基于文本聚类的自动文摘系统的研究与实现[J].计算机工程,2006,32(4):30-32. (GUO Q L, FAN X Z, LIU C A. Research and implementation about automatic abstract system based on text clustering[J]. Computer Engineering, 2006, 32(4): 30-32.)
[5] MANI I, BLOEDORM E. Machine learning of generic and user focused summarization[C]//AAAI '98/IAAI '98: Proceedings of the Fifteenth National/Tenth Conference on Artificial Intelligence/Innovative Applications of Artificial Intelligence. Menlo Park, CA: AAAI Press, 1998: 820-826.
[6] LUHN H P. The automatic creation of literature abstracts[J]. IBM Journal of Research and Development, 1958, 2(2): 159-165.
[7] BAXENDALE P B. Machine-made index for technical literature: an experiment[J]. IBM Journal of Research and Development, 1958, 2(4): 354-361.
[8] EDMUNDSON H P. New methods in automatic extracting[J]. Journal of the ACM, 1969, 16(2): 264-285.
[9] RAMEZANIA M, FEIZI-DERAKHSHI M-R. Automated text summarization: an overview[J]. Applied Artificial Intelligence: An International Journal, 2014, 28(2):178-215.
[10] GONG Y, LIU X. Generic text summarization using relevance measure and latent semantic analysis[C]//SIGIR '01: Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2001: 19-25.
[11] 傅间莲,陈群秀.基于规则和统计的中文自动文摘系统[J].中文信息学报,2006,20(5):10-16. (FU J L, CHEN Q X. Research on automatic summarization based on rules and statistics for Chinese texts[J]. Journal of Chinese Information Processing, 2006, 20(5): 10-16.)
[12] GOLDSTEIN J, KANTROWITZ M, MITTAL V, et al. Summarizing text documents: sentence selection and evaluation metrics[C]//SIGIR '99: Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 1999: 121-128.
[13] LIN C-Y. Training a selection function for extraction[C]//CIKM '99: Proceedings of the Eighth International Conference on Information and Knowledge Management. New York: ACM, 1999: 55-62.
[14] OSBORNE M. Using maximum entropy for sentence extraction[C]//AS '02: Proceedings of the ACL-02 Workshop on Automatic Summarization. Stroudsburg, PA: Association for Computational Linguistics, 2002, 4: 1-8.
[15] FRIEDMAN N, GEIGER D, GOLDSZMIDT M. Bayesian network classifiers[J]. Machine Learning, 1997, 29(2): 131-163.
[16] MURRY G, RENALS S, CARLETTA J, et al. Evaluating automatic summaries of meeting recordings[C]//Proceedings of ACL 2005 Workshop on Intrinsic and Extrinsic Evaluation Measures for MT and/or Summarization (MTSE). Stroudsburg, PA: Association for Computational Linguistics, 2005: 33-40.
[17] KAIKHAH K. Automatic text summarization with neural networks[C]//Proceedings of the 2004 2nd International IEEE Conference on Intelligent Systems. Piscataway, NJ: IEEE, 2004, 1: 40-44.
[18] 郭玉箐,万敏,罗振声.面向非受限领域的综合式自动中文文摘方法[J].清华大学学报(自然科学版),2002,42(1):139-142. (GUO Y Q, WAN M, LUO Z S. Automatic abstracting in domain independent Chinese documents[J]. Journal of Tsinghua University (Science & Technology), 2002, 42(1): 139-142.)
[19] 张明慧,王红玲,周国栋.基于LDA主题特征的自动文摘方法[J].计算机应用与软件,2011,28(10):20-22. (ZHANG M H, WANG H L, ZHOU G D. An automatic summarization approach based on LDA topic feature[J]. Computer Applications and Software, 2011, 28(10): 20-22.)
[20] 刘挺,王开铸.自动文摘的四种主要方法[J].情报学报,1999,18(1):10-19. (LIU T, WANG K Z. Four kinds of main methods of automatic abstracting[J]. Journal of the China Society for Scientific and Technical Information, 1999, 18(1): 10-19.)
[21] 蒋昌金,彭宏,陈建超,等.基于主题词权重和句子特征的自动文摘[J].华南理工大学学报(自然科学版),2010,38(7):50-55. (JIANG C J, PENG H, CHEN J C, et al. Automatic text summarization based on thematic word weight and sentence features[J]. Journal of South China University of Technology (Natural Science Edition), 2010, 38(7): 50-55.)
[22] LIN C-Y. ROUGE: a package for automatic evaluation of summaries[C]//WAS 2004: Proceedings of the ACL 2004 Workshop on Text Summarization Branches Out. Stroudsburg, PA: Association for Computational Linguistics, 2004: 74-81.)

基于多重映射的自动短文摘方法

Automatic short text summarization method based on multiple mapping

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

[1]	龙珑邓伟. 绿色网络智能文摘算法研究[J]. 计算机应用, 2012, 32(07): 2030-2032.
[2]	程传鹏杨要科. 自动文摘中的冗余句消除方法[J]. 计算机应用, 2011, 31(12): 3275-3277.
[3]	聂玲刘波. 数据集成中XML Schema到关系模式的转换方法[J]. 计算机应用, 2010, 30(11): 2941-2944.
[4]	傅间莲，陈群秀. 基于连续段落相似度的主题划分算法[J]. 计算机应用, 2005, 25(09): 2022-2024.
[5]	胡晓鹏，李晓航，李岗. 一种基于XML映射规则的数据迁移方法设计和实现[J]. 计算机应用, 2005, 25(08): 1849-1852.
[6]	郭庆琳，樊孝忠，柳长安. 文本聚类在自动文摘中的应用研究[J]. 计算机应用, 2005, 25(05): 1036-1038.