Relation extraction method based on negative training and transfer learning

doi:10.11772/j.issn.1001-9081.2022071004

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (8): 2426-2430.DOI: 10.11772/j.issn.1001-9081.2022071004

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Relation extraction method based on negative training and transfer learning

Kezheng CHEN¹^,², Xiaoran GUO³, Yong ZHONG¹^,², Zhenping LI¹^,²

^1.Chengdu Institute of Computer Application，Chinese Academy of Sciences，Chengdu Sichuan 610213，China
^2.School of Computer Science and Technology，University of Chinese Academy of Sciences，Beijing 100049，China
^3.School of Mathematics and Computer Science，Northwest Minzu University，Lanzhou Gansu 730124，China

Received:2022-07-11 Revised:2022-11-03 Accepted:2022-11-21 Online:2023-01-15 Published:2023-08-10
Contact: Yong ZHONG
About author:CHEN Kezheng， born in 1998， M. S. candidate. His research interests include nature language processing， big data.
GUO Xiaoran， born in 1981， Ph. D.， associate professor. Her research interests include information extraction， knowledge graph.
LI Zhenping， born in 1990， Ph. D. candidate. His research interests include natural language processing.
Supported by:
Science and Technology Achievement Transformation Platform Program of Sichuan(2020ZHCG0002);Youth Teacher Innovation Project of Fundamental Research Funds for the Central Universities(31920210090)

基于负训练和迁移学习的关系抽取方法

陈克正¹^,², 郭晓然³, 钟勇¹^,², 李振平¹^,²

^1.中国科学院成都计算机应用研究所, 成都 610213
^2.中国科学院大学计算机科学与技术学院, 北京 100049
^3.西北民族大学数学与计算机科学学院, 兰州 730124

通讯作者: 钟勇
作者简介:陈克正（1998—），男，山东济宁人，硕士研究生，CCF会员，主要研究方向：自然语言处理、大数据
郭晓然（1981—），女，河北藁城人，副教授，博士，主要研究方向：信息抽取、知识图谱
李振平（1990—），男，河南郑州人，博士研究生，主要研究方向：自然语言处理。
基金资助:
四川省科技成果转移转化平台项目(2020ZHCG0002);中央高校基本科研业务费（青年教师创新）项目(31920210090)

Abstract

Abstract:

In relation extraction tasks， distant supervision is a common method for automatic data labeling. However， this method will introduce a large amount of noisy data， which affects the performance of the model. In order to solve the problem of noisy data， a relation extraction method based on negative training and transfer learning was proposed. Firstly， a noisy data recognition model was trained through negative training method. Then， the noisy data were filtered and relabeled according to the predicted probability value of the sample， Finally， a transfer learning method was used to solve the domain shift problem existing in distant supervision tasks， and the precision and recall of the model were further improved. Based on Thangka culture， a relation extraction dataset with national characteristics was constructed. Experimental results show that the F1 score of the proposed method reaches 91.67%， which is 3.95 percentage points higher than that of SENT （Sentence level distant relation Extraction via Negative Training） method， and is much higher than those of the relation extraction methods based on BERT （Bidirectional Encoder Representations from Transformers）， BiLSTM+ATT（Bi-directional Long Short-Term Memory and Attention）， and PCNN （Piecewise Convolutional Neural Network）.

Key words: distant supervision, negative training, knowledge graph, relation extraction, transfer learning, Natural Language Processing

摘要：

远程监督是关系抽取任务中常用的数据自动标注方法，然而该方法会引入大量的噪声数据，从而影响模型的表现效果。为了解决噪声数据的问题，提出一种基于负训练和迁移学习的关系抽取方法。首先通过负训练的方法训练一个噪声数据识别模型；然后根据样本的预测概率值对噪声数据进行过滤和重新标注；最后利用迁移学习的方法解决远程监督存在的域偏移问题，从而进一步提升模型预测的精确率和召回率。以唐卡文化为基础，构建了具有民族特色的关系抽取数据集。实验结果表明，所提方法的F1值达到91.67%，相较于SENT（Sentence level distant relation Extraction via Negative Training）方法，提升了3.95个百分点，并且远高于基于BERT（Bidirectional Encoder Representations from Transformers）、BiLSTM+ATT（Bi-directional Long Short-Term Memory And Attention）、PCNN（Piecewise Convolutional Neural Network）的关系抽取方法。

关键词: 远程监督, 负训练, 知识图谱, 关系抽取, 迁移学习, 自然语言处理

CLC Number:

TP391.1

Kezheng CHEN, Xiaoran GUO, Yong ZHONG, Zhenping LI. Relation extraction method based on negative training and transfer learning[J]. Journal of Computer Applications, 2023, 43(8): 2426-2430.

陈克正, 郭晓然, 钟勇, 李振平. 基于负训练和迁移学习的关系抽取方法[J]. 《计算机应用》唯一官方网站, 2023, 43(8): 2426-2430.

Figures/Tables 6

References 28

1	郭喜跃，何婷婷. 信息抽取研究综述［J］. 计算机科学， 2015， 42（2）： 14-17. 10.11896/j.issn.1002-137X.2015.2.003
	GUO X Y， HE T T. Survey about research on information extraction［J］. Computer Science， 2015， 42（2）：14-17. 10.11896/j.issn.1002-137X.2015.2.003
2	姚萍，李坤伟，张一帆. 知识图谱构建技术综述［J］. 信息系统工程， 2020（5）：121-121， 123. 10.3969/j.issn.1001-2362.2020.05.054
	YAO P， LI K W， ZHANG Y F. Summary of knowledge graph construction technology［J］. China CIO News， 2020（5）： 121-121， 123. 10.3969/j.issn.1001-2362.2020.05.054
3	沈航可，祁志卫，张子辰，等. 知识图谱的候选实体搜索与排序［J］. 计算机系统应用， 2021， 30（11）： 46-53.
	SHEN H K， QI Z W， ZHANG Z C， et al. Candidate entity search and ranking of knowledge map［J］. Computer Systems and Applications， 2021， 30（11）： 46-53.
4	BACH N， BADASKAR S. A review of relation extraction［EB/OL］. ［2022-06-22］.. 10.2139/ssrn.4173454
5	XIONG C Y， POPWER R， CALLAN J. Explicit semantic ranking for academic search via knowledge graph embedding［C］// Proceedings of the 26th International Conference on World Wide Web. Republic and Canton of Geneva： International World Wide Web Conferences Steering Committee， 2017： 1271-1279. 10.1145/3038912.3052558
6	ZHANG Y Z， JIANG Z T， ZHANG T， et al. MIE： a medical information extractor towards medical dialogues［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2020： 6460-6469. 10.18653/v1/2020.acl-main.576
7	MINTZ M， BILLS S， SNOW R， et al. Distant supervision for relation extraction without labeled data［C］// Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Stroudsburg， PA： ACL， 2009： 1003-1011. 10.3115/1690219.1690287
8	RIEDEL S， YAO L M， McCALLUM A. Modeling relations and their mentions without labeled text［C］// Proceedings of the 2010 Joint European Conference on Machine Learning and Knowledge Discovery in Databases， LNCS 6323. Berlin： Springer， 2010： 148-163. 10.5715/jnlp.4.3_1
9	ZENG D J， LIU K， CHEN Y B， et al. Distant supervision for relation extraction via piecewise convolutional neural networks［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2015： 1753-1762. 10.18653/v1/d15-1203
10	QU J F， OUYANG D T， HUA W， et al. Distant supervision for neural relation extraction integrated with word attention and property features［J］. Neural Networks， 2018， 100： 59-69. 10.1016/j.neunet.2018.01.006
11	LIN Y K， SHEN S Q， LIU Z Y， et al. Neural relation extraction with selective attention over instances［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2016： 2124-2133. 10.18653/v1/p16-1200
12	XIAO Y， JIN Y C， CHENG R， et al. Hybrid attention-based Transformer block model for distant supervision relation extraction［J］. Neurocomputing， 2022， 470： 29-39. 10.1016/j.neucom.2021.10.037
13	ZHOU Y R， PAN L M， BAI C Y， et al. Self-selective attention using correlation between instances for distant supervision relation extraction［J］. Neural Networks， 2021， 142： 213-220. 10.1016/j.neunet.2021.04.032
14	CHEN T， SHI H Z， TANG S L， et al. CIL： contrastive instance learning framework for distantly supervised relation extraction［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Stroudsburg， PA： ACL， 2021： 6191-6200. 10.18653/v1/2021.acl-long.483
15	LI D， ZHANG T， HU N， et al. HiCLRE： a hierarchical contrastive learning framework for distantly supervised relation extraction［C］// Findings of the Association for Computational Linguistics： ACL 2022. Stroudsburg， PA： ACL， 2022： 2567-2578. 10.18653/v1/2022.findings-acl.202
16	SHANG Y M， HUANG H Y， MAO X L， et al. Are noisy sentences useless for distant supervised relation extraction？［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 8799-8806. 10.1609/aaai.v34i05.6407
17	MA R T， GUI T， LI L Y， et al. SENT： sentence-level distant relation extraction via negative training［C］// Proceedings of 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2021：6201-6213. 10.18653/v1/2021.acl-long.484
18	KIM Y， YIM J， YUN J， et al. NLNL： negative learning for noisy labels［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 101-110. 10.1109/iccv.2019.00019
19	TORREY L， SHAVLIK J. Transfer learning［M］// OLIVAS E S， GUERRERO， J D M， MARTINEZ-SOBER M， et al. Handbook of Research on Machine Learning Applications and Trends： Algorithms， Methods， and Techniques. Hershey， PA： IGI Global， 2010： 242-264. 10.4018/978-1-60566-766-9.ch011
20	PAN S J， YANG Q. A survey on transfer learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2010， 22（10）：1345-1359. 10.1109/tkde.2009.191
21	CHEN C， JIANG B Y， CHENG Z W， et al. Joint domain matching and classification for cross-domain adaptation via ELM［J］. Neurocomputing， 2019， 349： 314-325. 10.1016/j.neucom.2019.01.056
22	GUO H L， ZHU H J， GUO Z L， et al. Domain adaptation with latent semantic association for named entity recognition［C］// Proceedings of Human Language Technologies： The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2009： 281-289. 10.3115/1620754.1620795
23	FU L S， NGUYEN T H， MIN B N， et al. Domain adaptation for relation extraction with domain adversarial neural network［C］// Proceedings of the 8th International Joint Conference on Natural Language Processing （Volume 2： Short Papers）. ［S.l.］： Asian Federation of Natural Language Processing， 2017： 425-429.
24	SUN C， QIU X P， XU Y G， et al. How to fine-tune BERT for text classification？［C］// Proceedings of the 2019 China National Conference on Chinese Computational Linguistics， LNCS 11856. Cham： Springer， 2019： 194-206.
25	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg， PA： ACL， 2019： 4171-4186. 10.18653/v1/n18-2
26	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 6000-6010.
27	ZHANG S， ZHENG D Q， HU X C， et al. Bidirectional long short-term memory networks for relation classification［C/OL］// Proceedings of the 29th Pacific Asia Conference on Language， Information and Computation ［2020-04-20］..
28	ZHANG Y H， ZHONG V， CHEN D Q， et al. Position-aware attention and supervised data improve slot filling［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： ACL， 2017： 35-45. 10.3115/116580.1138590

序号	句子	远程监督标签	是否标注正确
1	观音，即观世音菩萨	别称	正确
2	观音一般指的就是观世音菩萨	别称	正确
3	多罗观音为观世音菩萨的修行伴侣	别称	错误
4	千手观音是观世音菩萨的三十二相之一	别称	错误

序号	句子	远程监督标签	是否标注正确
1	观音，即观世音菩萨	别称	正确
2	观音一般指的就是观世音菩萨	别称	正确
3	多罗观音为观世音菩萨的修行伴侣	别称	错误
4	千手观音是观世音菩萨的三十二相之一	别称	错误

关系类型	三元组数	关系类型	三元组数
梵音译	869	简称	292
化身	133	藏音译	1 173
藏文	345	合称	1 173
梵意译	587	藏意译	7
别称	1 894	梵文	889

关系类型	三元组数	关系类型	三元组数
梵音译	869	简称	292
化身	133	藏音译	1 173
藏文	345	合称	1 173
梵意译	587	藏意译	7
别称	1 894	梵文	889

关系类型	三元组数	关系类型	三元组数
梵音译	825	简称	2 800
化身	631	藏音译	49
藏文	295	合称	3 488
梵意译	445	藏意译	0
别称	10 629	梵文	3 071

Relation extraction method based on negative training and transfer learning

基于负训练和迁移学习的关系抽取方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 28

Related Articles 15

Recommended Articles

Metrics

模型	精确率	召回率	F1
PCNN	0.695 5	0.638 7	0.661 4
BiLSTM	0.703 4	0.659 9	0.678 9
BiLSTM+ATT	0.732 8	0.603 2	0.683 3
BERT	0.805 6	0.786 4	0.791 5
SENT	0.909 1	0.855 8	0.877 2
本文模型	0.9396	0.9101	0.9167

[1]	Qi SHUAI, Hairui WANG, Guifu ZHU. Chinese story ending generation model based on bidirectional contrastive training [J]. Journal of Computer Applications, 2024, 44(9): 2683-2688.
[2]	Guixiang XUE, Hui WANG, Weifeng ZHOU, Yu LIU, Yan LI. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network [J]. Journal of Computer Applications, 2024, 44(9): 2952-2957.
[3]	Jie WU, Ansi ZHANG, Maodong WU, Yizong ZHANG, Congbao WANG. Overview of research and application of knowledge graph in equipment fault diagnosis [J]. Journal of Computer Applications, 2024, 44(9): 2651-2659.
[4]	Quanmei ZHANG, Runping HUANG, Fei TENG, Haibo ZHANG, Nan ZHOU. Automatic international classification of disease coding method incorporating heterogeneous information [J]. Journal of Computer Applications, 2024, 44(8): 2476-2482.
[5]	Yubo ZHAO, Liping ZHANG, Sheng YAN, Min HOU, Mao GAO. Relation extraction between discipline knowledge entities based on improved piecewise convolutional neural network and knowledge distillation [J]. Journal of Computer Applications, 2024, 44(8): 2421-2429.
[6]	Yuan TANG, Yanping CHEN, Ying HU, Ruizhang HUANG, Yongbin QIN. Relation extraction model based on multi-scale hybrid attention convolutional neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2011-2017.
[7]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[8]	Qianhui LU, Yu ZHANG, Mengling WANG, Tingwei WU, Yuzhong SHAN. Classification model of nuclear power equipment quality text based on improved recurrent pooling network [J]. Journal of Computer Applications, 2024, 44(7): 2034-2040.
[9]	Chao WEI, Yanping CHEN, Kai WANG, Yongbin QIN, Ruizhang HUANG. Relation extraction method based on mask prompt and gated memory network calibration [J]. Journal of Computer Applications, 2024, 44(6): 1713-1719.
[10]	Jianjing LI, Guanfeng LI, Feizhou QIN, Weijun LI. Multi-relation approximate reasoning model based on uncertain knowledge graph embedding [J]. Journal of Computer Applications, 2024, 44(6): 1751-1759.
[11]	Yao LIU, Yumeng LI, Miaomiao SONG. Cognitive graph based on business process [J]. Journal of Computer Applications, 2024, 44(6): 1699-1705.
[12]	Youren YU, Yangsen ZHANG, Yuru JIANG, Gaijuan HUANG. Chinese named entity recognition model incorporating multi-granularity linguistic knowledge and hierarchical information [J]. Journal of Computer Applications, 2024, 44(6): 1706-1712.
[13]	Feiyu ZHAI, Handa MA. Hybrid classical-quantum classification model based on DenseNet [J]. Journal of Computer Applications, 2024, 44(6): 1905-1910.
[14]	Wangjun SHI, Jing WANG, Xiaojun NING, Youfang LIN. Sleep stage classification model by meta transfer learning in few-shot scenarios [J]. Journal of Computer Applications, 2024, 44(5): 1445-1451.
[15]	Hongtian LI, Xinhao SHI, Weiguo PAN, Cheng XU, Bingxin XU, Jiazheng YUAN. Few-shot object detection via fusing multi-scale and attention mechanism [J]. Journal of Computer Applications, 2024, 44(5): 1437-1444.