基于上下文语义增强的实体关系联合抽取

doi:10.11772/j.issn.1001-9081.2022040625

《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (5): 1438-1444.DOI: 10.11772/j.issn.1001-9081.2022040625

所属专题：人工智能

基于上下文语义增强的实体关系联合抽取

雷景生¹, 剌凯俊¹, 杨胜英¹(), 吴怡²

^1.浙江科技学院信息与电子工程学院，杭州 310023
^2.浙江省肿瘤医院，杭州 310022

收稿日期:2022-05-07 修回日期:2022-07-28 接受日期:2022-08-02 发布日期:2022-09-29 出版日期:2023-05-10
通讯作者: 杨胜英
作者简介:雷景生（1966—），男，陕西韩城人，教授，博士，主要研究方向：数据科学与大数据、机器学习、人工智能
剌凯俊（1996—），男，山西介休人，硕士研究生，主要研究方向：自然语言处理、关系抽取
杨胜英（1989—），男，山东东营人，讲师，博士，主要研究方向：机器学习、人工智能 syyang@zust.edu.cn
吴怡（1988—），女，浙江杭州人，硕士，主要研究方向：人工智能、护理对话。
基金资助:
国家自然科学基金资助项目(61972357);浙江省重点研发计划项目(2019C03135);浙江省医药卫生科技计划项目(2022KY104)

Joint entity and relation extraction based on contextual semantic enhancement

Jingsheng LEI¹, Kaijun LA¹, Shengying YANG¹(), Yi WU²

^1.School of Information and Electronic Engineering，Zhejiang University of Science and Technology，Hangzhou Zhejiang 310023，China
^2.Zhejiang Cancer Hospital，Hangzhou Zhejiang 310022，China

Received:2022-05-07 Revised:2022-07-28 Accepted:2022-08-02 Online:2022-09-29 Published:2023-05-10
Contact: Shengying YANG
About author:LEI Jingsheng， born in 1966， Ph. D.， professor. His research interests include data science and big data， machine learning， artificial intelligence.
LA Kaijun， born in 1996， M. S. candidate. His research interests include natural language processing， relation extraction.
YANG Shengying， born in 1989， Ph. D.， lecturer. His research interests include machine learning， artificial intelligence.
WU Yi， born in 1988， M. S. Her research interests include artificial intelligence， nursing dialogue.
Supported by:
National Natural Science Foundation of China(61972357);Key Research and Development Program of Zhejiang Province of China(2019C03135);Medical and Health Science and Technology Program of Zhejiang Province(2022KY104)

摘要/Abstract

摘要：

基于span的联合抽取模型在实体和关系抽取（RE）任务中共享实体span的语义表示，能有效降低流水线模型带来的级联误差，但现有模型无法充分地将上下文信息融入实体和关系的表示中。针对上述问题，提出一个基于上下文语义增强的实体关系联合抽取（JERCE）模型。首先通过对比学习的方法获取句子级文本和实体间文本的语义特征表示；然后，将该表示加入实体和关系的表示中，对实体关系进行联合预测；最后，动态调整两个任务的损失以使联合模型的整体性能最优化。在公共数据集CoNLL04、ADE和ACE05上进行实验，结果显示JERCE模型与触发器感知记忆流框架（TriMF）相比，实体识别F1值分别提升了1.04、0.13和2.12个百分点，RE的F1值则分别提升了1.19、1.14和0.44个百分点。实验结果表明，JERCE模型可以充分获取上下文中的语义信息。

关键词: 命名实体识别, 关系抽取, 对比学习, 文本span, 加权损失

Abstract:

Span-based joint extraction model shares the semantic representation of entity spans in entity and Relation Extraction （RE） tasks， which effectively reduces the cascade error caused by pipeline models. However， the existing models cannot adequately integrate contextual information into the representation of entities and relations. To solve this problem， a Joint Entity and Relation extraction model based on Contextual semantic Enhancement （JERCE） was proposed. Firstly， the semantic feature representations of sentence-level text and inter-entity text were obtained by contrastive learning method. Then， the representations were added into the representations of entity and relation to predict entities and relations jointly. Finally， the loss values of the two tasks were adjusted dynamically to optimize the overall performance of the joint model. In experiments on public datasets CoNLL04， ADE and ACE05， compared with Trigger-sense Memory Flow framework （TriMF）， the proposed JERCE model has the F1 scores of entity recognition improved by 1.04， 0.13 and 2.12 percentage points respectively， and the F1 scores of RE increased by 1.19， 1.14 and 0.44 percentage points respectively. Experimental results show that the JERCE model can fully obtain semantic information in context.

Key words: Named Entity Recognition (NER), Relation Extraction (RE), contrastive learning, text span, weighted loss

中图分类号:

TP183

雷景生, 剌凯俊, 杨胜英, 吴怡. 基于上下文语义增强的实体关系联合抽取[J]. 计算机应用, 2023, 43(5): 1438-1444.

Jingsheng LEI, Kaijun LA, Shengying YANG, Yi WU. Joint entity and relation extraction based on contextual semantic enhancement[J]. Journal of Computer Applications, 2023, 43(5): 1438-1444.

图/表 9

图1 span示例

Fig. 1 Examples of span

图2 JERCE的模型架构

Fig. 2 Model architecture of JERCE

图3 BERT模型

Fig. 3 BERT model

图4 对比学习模块的框架

Fig. 4 Framework of contrastive learning module

表1 不同的模型在CoNLL04、ADE和ACE05上的实验结果 ( %)

Tab.1 Experimental results of different models on CoNLL04， ADE and ACE05

数据集	模型	实体识别			关系抽取
数据集	模型	$P$	$R$	$F 1$	$P$	$R$	$F 1$
CoNLL04	Relation-Metric	84.46	84.67	84.57	67.97	58.18	62.68
	MTQA	89.00	86.60	87.80	69.20	68.20	68.90
	SpERT	88.25	89.64	88.94	73.04	70.00	71.47
	ERIGAT	89.88	83.97	86.82	75.87	68.02	71.73
	eRPR MHS	86.85	85.62	86.23	64.20	64.69	64.44
	MRC4ERE++	89.04	87.99	88.51	72.22	70.75	71.48
	TriMF	89.35	89.22	89.28	72.64	70.72	71.67
	JERCE	90.23	90.42	90.32	74.98	70.86	72.86
ADE	Relation-Metric	86.16	88.08	87.11	77.36	77.25	77.29
	SpERT	88.99	89.59	89.28	77.77	79.96	78.84
	ERIGAT	90.53	87.12	88.79	84.71	75.79	80.09
	eRPR MHS	86.65	86.03	86.34	74.35	86.12	79.80
	TriMF	89.08	90.27	89.67	74.19	84.38	78.96
	JERCE	89.99	89.62	89.80	79.22	80.99	80.10
ACE05	MRC4ERE++	87.03	86.96	86.99	62.02	62.31	62.16
	SpERT	85.48	84.77	85.12	61.32	60.03	60.67
	MTQA	84.70	84.9	84.80	64.80	56.20	60.2
	eRPR MHS	86.26	84.66	85.45	60.60	60.84	60.72
	TriMF	87.66	87.47	87.56	61.98	62.87	62.42
	JERCE	89.25	90.11	89.68	66.05	59.97	62.86

表1 不同的模型在CoNLL04、ADE和ACE05上的实验结果 ( %)

Tab.1 Experimental results of different models on CoNLL04， ADE and ACE05

数据集	模型	实体识别			关系抽取
数据集	模型	$P$	$R$	$F 1$	$P$	$R$	$F 1$
CoNLL04	Relation-Metric	84.46	84.67	84.57	67.97	58.18	62.68
	MTQA	89.00	86.60	87.80	69.20	68.20	68.90
	SpERT	88.25	89.64	88.94	73.04	70.00	71.47
	ERIGAT	89.88	83.97	86.82	75.87	68.02	71.73
	eRPR MHS	86.85	85.62	86.23	64.20	64.69	64.44
	MRC4ERE++	89.04	87.99	88.51	72.22	70.75	71.48
	TriMF	89.35	89.22	89.28	72.64	70.72	71.67
	JERCE	90.23	90.42	90.32	74.98	70.86	72.86
ADE	Relation-Metric	86.16	88.08	87.11	77.36	77.25	77.29
	SpERT	88.99	89.59	89.28	77.77	79.96	78.84
	ERIGAT	90.53	87.12	88.79	84.71	75.79	80.09
	eRPR MHS	86.65	86.03	86.34	74.35	86.12	79.80
	TriMF	89.08	90.27	89.67	74.19	84.38	78.96
	JERCE	89.99	89.62	89.80	79.22	80.99	80.10
ACE05	MRC4ERE++	87.03	86.96	86.99	62.02	62.31	62.16
	SpERT	85.48	84.77	85.12	61.32	60.03	60.67
	MTQA	84.70	84.9	84.80	64.80	56.20	60.2
	eRPR MHS	86.26	84.66	85.45	60.60	60.84	60.72
	TriMF	87.66	87.47	87.56	61.98	62.87	62.42
	JERCE	89.25	90.11	89.68	66.05	59.97	62.86

表2 语义增强消融实验的F1值 ( %)

Tab.2 F1values insemantic enhancement ablation experiments

消融方法	实体识别	关系抽取
JERCE	89.68	62.86
-ContextEnhanced	89.01	60.77
-SentenceEnhanced	87.75	61.58
both	86.80	59.89

表3 加权损失对模型F1值的影响 ( %)

Tab.3 Influence of weighted loss on model F1 value

加权损失	实体识别	关系抽取
JERCE	89.68	62.86
- $L j o i n t$	88.94	61.12

表3 加权损失对模型F1值的影响 ( %)

Tab.3 Influence of weighted loss on model F1 value

加权损失	实体识别	关系抽取
JERCE	89.68	62.86
- $L j o i n t$	88.94	61.12

图5 批量大小对实体和关系抽取性能的影响

Fig. 5 Influence of batch size on entity and relation extraction performance

表4 常见错误示例

Tab. 4 Common error examples

错误类型	错误示例
边界模糊	［Mr.［Harry Potter］_PER］_PER
逻辑错误	Miller is also scheduled to meet with Crimean Deputy［David］and［［Black Sea Fleet］］Commander［Jack］.
逻辑缺失	［Becton］sells needle containers to doctors and hospitals but may develop a container for home use， said［Linda］， an assistant product manager.

参考文献 23

1	鄂海红，张文静，肖思琪，等. 深度学习实体关系抽取研究综述［J］. 软件学报， 2019， 30（6）： 1793-1818. 10.13328/j.cnki.jos.005817
	E H H， ZHANG W J， XIAO S Q， et al. Survey of entity relationship extraction based on deep learning［J］. Journal of Software， 2019， 30（6）： 1793-1818. 10.13328/j.cnki.jos.005817
2	CHI R J， WU B， HU L M， et al. Enhancing joint entity and relation extraction with language modeling and hierarchical attention［C］// Proceedings of the 2019 International Joint Conference， Asia-Pacific Web and Web-Age Information Management Joint International Conference on Web and Big Data， LNCS 11641. Cham： Springer， 2019： 314-328.
3	EBERTS M， ULGES A. Span-based joint entity and relation extraction with Transformer pre-training［C］// Proceedings of the 24th European Conference on Artificial Intelligence. Amsterdam： IOS Press， 2020： 2006-2013. 10.18653/v1/2021.eacl-main.319
4	PETERS M E， NEUMANN M， IYYER M， et al. Deep contextualized word representations［C］// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long Papers）. Stroudsburg， PA： ACL， 2018： 2227-2237. 10.18653/v1/n18-1202
5	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg， PA： ACL， 2019： 4171-4186. 10.18653/v1/n18-2
6	张少伟，王鑫，陈子睿，等. 有监督实体关系联合抽取方法研究综述［J］. 计算机科学与探索， 2022， 16（4）： 713-733. 10.3778/j.issn.1673-9418.2107114
	ZHANG S W， WANG X， CHEN Z R， et al. Survey of supervised joint entity relation extraction methods［J］. Journal of Frontiers of Computer Science and Technology， 2022， 16（4）： 713-733. 10.3778/j.issn.1673-9418.2107114
7	GUPTA P， SCHÜLTZE H， ANDRASSY B. Table filling multi-task recurrent neural network for joint entity and relation extraction［C］// Proceedings of the 26th International Conference on Computational Linguistics： Technical Papers. ［S.l.］： The COLING 2016 Organizing Committee， 2016： 2537-2547.
8	ZHAO T Y， YAN Z， CAO Y B， et al. Entity relative position representation based multi-head selection for joint entity and relation extraction［C］// Proceedings of the 19th Chinese National Conference on Computational Linguistics. Beijing： Chinese Information Processing Society of China， 2020： 962-973. 10.1007/978-3-030-63031-7_14
9	SUI D B， CHEN Y B， LIU K， et al. Joint entity and relation extraction with set prediction networks［EB/OL］. （2020-11-05）［2022-03-20］.. 10.1109/tnnls.2023.3264735
10	SHEN Y L， MA X Y， TANG Y C， et al. A trigger-sense memory flow framework for joint entity and relation extraction［C］// Proceedings of the Web Conference 2021. New York： ACM， 2021： 1704-1715. 10.1145/3442381.3449895
11	MIKOLOV T， SUTSKEVER I， CHEN K， et al. Distributed representations of words and phrases and their compositionality［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems — Volume 2. Red Hook， NY： Curran Associates Inc.， 2013： 3111-3119.
12	SAUNSHI N， PLEVRAKIS O， ARORA S， et al. A theoretical analysis of contrastive unsupervised representation learning［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 5628-5637.
13	FANG H C， WANG S C， ZHOU M， et al. CERT： contrastive self-supervised learning for language understanding［EB/OL］. （2020-06-18）［2022-03-20］.. 10.36227/techrxiv.12308378.v1
14	ITER D， GUU K， LANSING L， et al. Pretraining with contrastive sentence objectives improves discourse performance of language models［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2020： 4859-4870. 10.18653/v1/2020.acl-main.439
15	CIPOLLA R， GAL Y， KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7482-7491. 10.1109/cvpr.2018.00781
16	DODDINGTON G， MITCHELL A， PRZYBOCKI M， et al. The Automatic Content Extraction （ACE） program — tasks， data， and evaluation［C］// Proceedings of the 4th International Conference on Language Resources and Evaluation. Paris： European Language Resources Association， 2004： 837-840.
17	ROTH D， YIH W T. A linear programming formulation for global inference in natural language tasks［C］// Proceedings of the 8th Conference on Computational Natural Language Learning at HLT-NAACL 2004. Stroudsburg， PA： ACL， 2004： 1-8.
18	GURULINGAPPA H， RAJPUT A M， ROBERTS A， et al. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports［J］. Journal of Biomedical Informatics， 2012， 45（5）： 885-892. 10.1016/j.jbi.2012.04.008
19	LI Q， JI H. Incremental joint extraction of entity mentions and relations［C］// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg， PA： ACL， 2014： 402-412. 10.3115/v1/p14-1038
20	TRAN T， KAVULURU R. Neural metric learning for fast end-to-end relation extraction［EB/OL］. （2019-08-27）［2022-03-20］.. 10.1093/database/bay092
21	LI X Y， YIN F， SUN Z J， et al. Entity-relation extraction as multi-turn question answering［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg， PA： ACL， 2019： 1340-1350. 10.18653/v1/p19-1129
22	LAI Q H， ZHOU Z H， LIU S. Joint entity-relation extraction via improved graph attention networks［J］. Symmetry， 2020， 12（10）： No.1746. 10.3390/sym12101746
23	ZHAO T Y， YAN Z， CAO Y B， et al. Asking effective and diverse questions： a machine reading comprehension based framework for joint entity-relation extraction［C］// Proceedings of the 29th International Joint Conferences on Artificial Intelligence. California： ijcai.org， 2021： 3948-3954. 10.24963/ijcai.2020/546

[1]	杨兴耀, 陈羽, 于炯, 张祖莲, 陈嘉颖, 王东晓. 结合自我特征和对比学习的推荐模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2704-2710.
[2]	孙焕良, 王思懿, 刘俊岭, 许景科. 社交媒体数据中水灾事件求助信息提取模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2437-2445.
[3]	赵宇博, 张丽萍, 闫盛, 侯敏, 高茂. 基于改进分段卷积神经网络和知识蒸馏的学科知识实体间关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2421-2429.
[4]	唐媛, 陈艳平, 扈应, 黄瑞章, 秦永彬. 基于多尺度混合注意力卷积神经网络的关系抽取模型[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2011-2017.
[5]	毛典辉, 李学博, 刘峻岭, 张登辉, 颜文婧. 基于并行异构图和序列注意力机制的中文实体关系抽取模型[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2018-2025.
[6]	徐松, 张文博, 王一帆. 基于时空信息的轻量视频显著性目标检测网络[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2192-2199.
[7]	蒋小霞, 黄瑞章, 白瑞娜, 任丽娜, 陈艳平. 基于事件表示和对比学习的深度事件聚类方法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1734-1742.
[8]	魏超, 陈艳平, 王凯, 秦永彬, 黄瑞章. 基于掩码提示与门控记忆网络校准的关系抽取方法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1713-1719.
[9]	于右任, 张仰森, 蒋玉茹, 黄改娟. 融合多粒度语言知识与层级信息的中文命名实体识别模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1706-1712.
[10]	汪炅, 唐韬韬, 贾彩燕. 无负采样的正样本增强图对比学习推荐方法PAGCL[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1485-1492.
[11]	郭洁, 林佳瑜, 梁祖红, 罗孝波, 孙海涛. 基于知识感知和跨层次对比学习的推荐方法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1121-1127.
[12]	袁泉, 陈昌平, 陈泽, 詹林峰. 基于BERT的两次注意力机制远程监督关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1080-1085.
[13]	董永峰, 白佳明, 王利琴, 王旭. 融合先验知识和字形特征的中文命名实体识别[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 702-708.
[14]	郭安迪, 贾真, 李天瑞. 基于伪实体数据增强的高精准率医学领域实体关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 393-402.
[15]	罗歆然, 李天瑞, 贾真. 基于自注意力机制与词汇增强的中文医学命名实体识别[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 385-392.

基于上下文语义增强的实体关系联合抽取

Joint entity and relation extraction based on contextual semantic enhancement

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 23

相关文章 15

编辑推荐

Metrics