基于强化学习的实体关系联合抽取模型

doi:10.11772/j.issn.1001-9081.2019010182

计算机应用 ›› 2019, Vol. 39 ›› Issue (7): 1918-1924.DOI: 10.11772/j.issn.1001-9081.2019010182

基于强化学习的实体关系联合抽取模型

陈佳沣, 滕冲

武汉大学国家网络安全学院, 武汉 430072

收稿日期:2019-01-24 修回日期:2019-03-21 发布日期:2019-04-15 出版日期:2019-07-10
通讯作者: 滕冲
作者简介:陈佳沣(1995-),男,湖北武汉人,硕士研究生,主要研究方向:数据挖掘、深度学习;滕冲(1974-),女,湖北武汉人,副教授,博士,主要研究方向:数据挖掘、大数据、深度学习、舆情分析。
基金资助:
国家自然科学基金面上项目（61772378）。

Joint entity and relation extraction model based on reinforcement learning

CHEN Jiafeng, TENG Chong

School of Cyber Science and Engineering, Wuhan University, Wuhan Hubei 430072, China

Received:2019-01-24 Revised:2019-03-21 Online:2019-04-15 Published:2019-07-10
Supported by:
This work is partially supported by the Surface Program of National Natural Science Foundation of China (61772378).

摘要/Abstract

摘要：

针对现有的基于远程监督的实体和关系抽取方法存在着标签噪声问题，提出了一种基于强化学习的实体关系联合抽取方法。该模型有两个模块：句子选择器模块和实体关系联合抽取模块。首先，句子选择器模块选择没有标签噪声的高质量句子，将所选句子输入到实体关系联合抽取模型；然后，实体关系联合抽取模块采用序列标注方法对输入的句子进行预测，并向句子选择器模块提供反馈，指导句子选择器模块挑选高质量的句子；最后，句子选择器模块和实体关系联合抽取模块同时训练，将句子选择与序列标注一起优化。实验结果表明，该模型在实体关系联合抽取中的F1值为47.3%，与CoType为代表的联合抽取模型相比，所提模型的F1值提升了1%；与LINE为代表的串行模型相比，所提模型的F1值提升了14%。结果表明强化学习结合实体关系联合抽取模型能够有效地提高序列标注模型的F1值，其中句子选择器能有效地处理数据的噪声。

关键词: 强化学习, 联合抽取, 序列标注, 命名实体识别, 关系分类

Abstract:

Existing entity and relation extraction methods that rely on distant supervision suffer from noisy labeling problem. A model for joint entity and relation extraction from noisy data based on reinforcement learning was proposed to reduce the impact of noise data. There were two modules in the model:an sentence selector module and a sequence labeling module. Firstly, high-quality sentences without labeling noise were selected by instance selector module and the selected sentences were input into sequence labeling module. Secondly, predictions were made by sequence labeling module and the rewards were provided to sentence selector module to help the module select high-quality sentences. Finally, two modules were trained jointly to optimize instance selection and sequence labeling processes. The experimental results show that the F1 value of the proposed model is 47.3% in the joint entity and relation extraction, which is 1% higher than those of joint extraction models represented by CoType and 14% higher than those of serial models represented by LINE(Large-scale Information Network Embedding). The results show that the joint entity and relation extraction model in combination with reinforcement learning can effectively improve F1 value of sequential labeling model, in which the sentence selector can effectively deal with the noise of data.

Key words: reinforcement learning, joint extraction, sequence tagging, named entity recognition, relation classification

中图分类号:

TP389.1

陈佳沣, 滕冲. 基于强化学习的实体关系联合抽取模型[J]. 计算机应用, 2019, 39(7): 1918-1924.

CHEN Jiafeng, TENG Chong. Joint entity and relation extraction model based on reinforcement learning[J]. Journal of Computer Applications, 2019, 39(7): 1918-1924.

参考文献

[1] BANKO M, CAFARELLAM J, SODERLAND S, et al. Open information extraction from the Web[C]//Proceedings of the 20th International Joint Conference on Artificial Intelligence. New York:ACM, 2007:2670-2676.
[2] NADEAU D, SEKINE S. A survey of named entity recognition and classification[J]. Lingvisticae Investigationes, 2005, 30(1):3-26.
[3] RINK B, HARABAGIU A. UTD:classifying semantic relations by combining lexical and semantic resources[C]//Proceedings of the 5th International Workshop on Semantic Evaluation. New York:ACM, 2010:256-259.
[4] LI Q, JI H. Incremental joint extraction of entity mentions and relations[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2014:402-412.
[5] MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016:1105-1116.
[6] ZHENG S C, WANG F. Joint extraction of entities and relations based on a novel tagging scheme[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2017:1227-1236.
[7] MINTZ M, BILLS S, SNOW R, et al. Distant supervision for relation extraction without labeled data[C]//Proceedings of the 2009/47th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2009:1003-1011
[8] FENG J, HUANG M, ZHAO L, et al. Reinforcement learning for relation classification from noisy data[C]//Proceedings of the 2018/32nd Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence. Menlo Park, CA:AAAI, 2018:5779-5786
[9] SUTTON R S, BARTO A G. Reinforcement learning:an introduction[J]. IEEE Transactions on Neural Networks, 1998, 9(5):1054-1054.
[10] LUO G, HUANG X J, LIN C Y, et al. Joint entity recognition and disambiguation[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2015:879-888.
[11] 冯元勇,孙乐,张大鲲,等.基于小规模尾字特征的中文命名实体识別研究[J].电子学报,2008,36(9):1833-1838.(FENG Y Y, SUN L, ZHANG D K, et al. Study on the Chinese named entity recognition using small scale tail hints[J]. Acta Electronica Sinaca, 2008, 36(9):1833-1838.)
[12] 向晓雯,史晓东,曾华琳.一个统计与规则相结合的中文命名实体识别系统[J].计算机应用,2005,25(10):2404-2406.(XIANG X W, SHI X D, ZENG H L. Chinese named entity recognition system using statistics-based and rules-based method[J]. Journal of Computer Applications, 2005, 25(10):2404-2406.)
[13] 佘俊,张学清.音乐命名实体识别方法[J].计算机应用,2010,20(11):2928-2931.(SHE J, ZHANG X Q. Musical named entity recognition method[J]. Journal of Computer Applications, 2010, 30(11):2928-2931.)
[14] 张金龙,王石,钱存发.基于CRF和规则的中文医疗机构名称识[J].计算机应用与软件,2014,31(3):159-162.(ZHANG J L, WANG S, QIAN C F. CRF and rules-based recognition of medical institutions name in Chinese[J]. Computer Applications and Software, 2014, 31(3):159-162.)
[15] CHIU J P C, NICHOLS E. Named entity recognition with bidirectional LSTM-CNNs[C]//Proceedings of the 2016 Transactions of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016:357-370
[16] HUANG Z, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL].[2018-12-02]. https://arxiv.org/pdf/1508.01991.pdf.
[17] LAMPLE G, BALLESTEROS M, SUBRAMANIAN S, et al. Neural architectures for named entity recognition[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016:260-270.
[18] REN X, WU Z, HE W, et al. CoType:joint extraction of typed entities and relations with knowledge bases[C]//Proceedings of the 26th International Conference on World Wide Web. New York:ACM, 2017:1015-1024.
[19] SINGH S, RIEDEL S, MARTIN B, et al. Joint inference of entities, relations, and coreference[C]//Proceedings of the 2013 Workshop on Automated Knowledge Base Construction. New York:ACM, 2013:1-6.
[20] LIN Y, SHEN S, LIU Z, et al. Neural relation extraction with selective attention over instances[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA:Association for Computational Linguistics, 2016:2124-2133.
[21] JI G, LIU K, HE S, et al. Distant supervision for relation extraction with sentence-level attention and entity descriptions[C]//Proceedings of the Thirty-First Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence. Menlo Park, CA:AAAI, 2017:3060-3066.
[22] NARASIMHAN K, YALA A, BARZILAY R. Improving information extraction by acquiring external evidence with reinforcement learning[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2016:2355-2365.
[23] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[J]. Advances in Neural Information Processing Systems, 2013, 26:3111-3119.
[24] TANG J, QU M, WANG M, et al. LINE:large-scale information network embedding[C]//Proceedings of the 24th International Conference on World Wide Web. New York:ACM, 2015:1067-1077.
[25] GORMLEY M R, YU M, DREDZE M. Improved relation extraction with feature-rich compositional embedding models[C]//Proceedings of the 2015 Conference on Empirical Method in Natural Language Processing. Stroudsburg, PA:Association for Computational Linguistics, 2015:1774-1784.
[26] HOFFMANN R, ZHANG C, LING X, et al. Knowledge-based weak supervision for information extraction of overlapping relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies. Stroudsburg, PA:Association for Computational Linguistics, 2011:541-550.

基于强化学习的实体关系联合抽取模型

Joint entity and relation extraction model based on reinforcement learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	刘雅璇, 钟勇. 基于头实体注意力的实体关系联合抽取方法[J]. 计算机应用, 2021, 41(9): 2517-2522.
[2]	武国亮, 徐继宁. 基于命名实体识别任务反馈增强的中文突发事件抽取方法[J]. 计算机应用, 2021, 41(7): 1891-1896.
[3]	王宇, 刘燕丽, 陈劭武. 基于顶点冲突学习的最大公共子图算法[J]. 计算机应用, 2021, 41(6): 1756-1760.
[4]	王建平, 王刚, 毛晓彬, 马恩琪. 基于深度强化学习的二连杆机械臂运动控制方法[J]. 计算机应用, 2021, 41(6): 1799-1804.
[5]	崔博文, 金涛, 王建民. 自由文本电子病历信息抽取综述[J]. 计算机应用, 2021, 41(4): 1055-1063.
[6]	杜嘻嘻, 程华, 房一泉. 基于优势演员-评论家算法的强化自动摘要模型[J]. 计算机应用, 2021, 41(3): 699-705.
[7]	许力, 李建华. 基于句法依存分析的图网络生物医学命名实体识别[J]. 计算机应用, 2021, 41(2): 357-362.
[8]	李灿, 杨雅婷, 马玉鹏, 董瑞. 基于语种相似性挖掘的神经机器翻译语料库扩充方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3145-3150.
[9]	吴赛赛, 梁晓贺, 谢能付, 周爱莲, 郝心宁. 面向领域实体关系联合抽取的标注方法[J]. 计算机应用, 2021, 41(10): 2858-2863.
[10]	丁相国, 桑基韬. 基于关系自适应解码的实体关系联合抽取[J]. 计算机应用, 2021, 41(1): 29-35.
[11]	刘思嘉, 童向荣. 基于强化学习的城市交通路径规划[J]. 计算机应用, 2021, 41(1): 185-190.
[12]	姚兴虎, 谭晓阳. 基于奖励高速路网络的多智能体强化学习中的全局信用分配算法[J]. 计算机应用, 2021, 41(1): 1-7.
[13]	傅魁, 梁少晴, 李冰. 基于改进的深度Q网络结构的商品推荐模型[J]. 计算机应用, 2020, 40(9): 2613-2621.
[14]	张心怡, 冯仕民, 丁恩杰. 面向煤矿的实体识别与关系抽取模型[J]. 计算机应用, 2020, 40(8): 2182-2188.
[15]	胡学敏, 成煜, 陈国文, 张若晗, 童秀迟. 基于深度时空Q网络的定向导航自动驾驶运动规划[J]. 计算机应用, 2020, 40(7): 1919-1925.