Document-level relation extraction based on entity representation enhancement

doi:10.11772/j.issn.1001-9081.2024050682

Abstract

Abstract:

Aiming at problems of ignoring entity mention differences and lack of complexity calculation paradigm for entity-pair relation extraction in the existing entity representation learning for Document-level Relation Extraction （DocRE） tasks， a DocRE model based on Entity Representation Enhancement （DREERE） was proposed. Firstly， an attention mechanism was used to evaluate the differences of entity mentions in determining different entity-pair relations， so as to obtain more flexible entity representations. Secondly， the entity-pair sentence importance distribution computed by the encoder was used to evaluate the complexity of entity-pair relation extraction， and the two-hop information among entity-pairs was used selectively to enhance entity-pair representations. Experiments were carried out on the popular datasets DocRED， Re-DocRED and DWIE. The results show that DREERE model improves the F1 value by 0.06， 0.14， and 0.23 percentage points， respectively， and the ign-F1 （F1 score calculated by ignoring the triples that appear in the training set） value by 0.07， 0.09 and 0.12 percentage points， respectively， compared to the optimal baseline models such as ATLOP （Adaptive Thresholding and Localized cOntext Pooling） and E2GRE （Entity and Evidence Guided Relation Extraction）， indicating that DREERE model is able to acquire semantic information of entities in documents effectively.

Key words: Document-level Relation Extraction (DocRE), attention mechanism, Evidence Retrieval (ER), representation learning, two-hop information

摘要：

针对现有的文档级关系抽取（DocRE）任务的实体表示学习存在的忽视实体提及差异性和缺少实体对关系抽取复杂度的计算范式的问题，提出一种基于实体表示增强的DocRE模型（DREERE）。首先，利用注意力机制评估实体提及在判定不同实体对关系时的差异性，得到更灵活的实体表示；其次，利用编码器计算得到的实体对句子重要性分布评估实体对关系抽取的复杂度，再选择性地利用实体对之间的两跳信息增强实体对的表示；最后，在3个流行的数据集DocRED、Re-DocRED和DWIE上进行实验。结果显示，与最优基线模型（如ATLOP（Adaptive Thresholding and Localized cOntext Pooling）、E2GRE（Entity and Evidence Guided Relation Extraction））相比，DREERE的F1值分别提高了0.06、0.14和0.23个百分点，忽略训练集出现的三元组而计算得到的F1分数（ign-F1）值分别提高了0.07、0.09和0.12个百分点，可见该模型能够有效获取文档里的实体语义信息。

关键词: 文档级关系抽取, 注意力机制, 证据搜索, 表示学习, 两跳信息

CLC Number:

TP391.1

Haijie WANG, Guangxin ZHANG, Hai SHI, Shu CHEN. Document-level relation extraction based on entity representation enhancement[J]. Journal of Computer Applications, 2025, 45(6): 1809-1816.

王海杰, 张广鑫, 史海, 陈树. 基于实体表示增强的文档级关系抽取[J]. 《计算机应用》唯一官方网站, 2025, 45(6): 1809-1816.

Figures/Tables 10

References 36

1	谢德鹏，常青. 关系抽取综述［J］. 计算机应用研究， 2020， 37（7）：1921-1924， 1930.
	XIE D P， CHANG Q. Review of relation extraction［J］. Application Research of Computers， 2020， 37（7）： 1921-1924， 1930.
2	ZHOU W， HUANG K， MA T， et al. Document-level relation extraction with adaptive thresholding and localized context pooling［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 14612-14620.
3	XU B， WANG Q， LYU Y， et al. Entity structure within and throughout： modeling mention dependencies for document-level relation extraction［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021：14149-14157.
4	ZHANG N， CHEN X， XIE X， et al. Document-level relation extraction as semantic segmentation［C］// Proceedings of the 30th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2021：3999-4006.
5	TAN Q， HE R， BING L， et al. Document-level relation extraction with adaptive focal loss and knowledge distillation［C］// Findings of the Association for Computational Linguistics： ACL 2022. Stroudsburg： ACL， 2022：1672-1681.
6	ZENG S， WU Y， CHANG B. SIRE： separate intra- and inter-sentential reasoning for document-level relation extraction［C］// Findings of the Association for Computational Linguistics： ACL-IJCNLP 2021. Stroudsburg： ACL， 2021： 524-534.
7	MA Y， WANG A， OKAZAKI N. DREEAM： guiding attention with evidence for improving document-level relation extraction［C］// Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg： ACL， 2023： 1971-1983.
8	ZENG D， LIU K， CHEN Y， et al. Distant supervision for relation extraction via piecewise convolutional neural networks［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2015：1753-1762.
9	FENG J， HUANG M， ZHAO L， et al. Reinforcement learning for relation classification from noisy data ［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 5779-5786.
10	YE D， LIN Y， DU J， et al. Coreferential reasoning learning for language representation ［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2020： 7170-7186.
11	WANG Z， WEN R， CHEN X， et al. Finding influential instances for distantly supervised relation extraction ［C］// Proceedings of the 29th International Conference on Computational Linguistics. ［S.l.］： International Committee on Computational Linguistics， 2022： 2639-2650.
12	袁泉，陈昌平，陈泽，等.基于BERT的两次注意力机制远程监督关系抽取［J］. 计算机应用，2024， 44（4）：1080-1085.
	YUAN Q， CHEN C P， CHEN Z， et al. Twice attention mechanism distantly supervised relation extraction based on BER ［J］. Journal of Computer Applications， 2024， 44（4）： 1080-1085.
13	YU H， ZHANG N， DENG S， et al. Bridging text and knowledge with multi-prototype embedding for few-shot relational triple extraction［C］// Proceedings of the 28th International Conference on Computational Linguistics. ［S.l.］： International Committee on Computational Linguistics， 2020： 6399-6410.
14	WU T， LI X， LI Y F， et al. Curriculum-meta learning for order-robust continual relation extraction［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021：10363-10369.
15	HUANG Y， LI Z， DENG W， et al. D-BERT： incorporating dependency-based attention into BERT for relation extraction ［J］. CAAI Transactions on Intelligence Technology， 2021， 6（4）： 417-425.
16	LI Z， HU F， WANG C， et al. Selective kernel networks for weakly supervised relation extraction［J］. CAAI Transactions on Intelligence Technology， 2021， 6（2）： 224-234.
17	ZHANG G， CHEN S. Siamese representation learning for unsupervised relation extraction［C］// Proceedings of the 26th European Conference on Artificial Intelligence. Amsterdam： IOS Press， 2023： 3002-3009.
18	CHRISTOPOULOU F， MIWA M， ANANIADOU S. Connecting the dots： document-level neural relation extraction with edge-oriented graphs［C］// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg： ACL， 2019：4925-4936.
19	LI B， YE W， SHENG Z， et al. Graph enhanced dual attention network for document-level relation extraction［C］// Proceedings of the 28th International Conference on Computational Linguistics. ［S.l.］： International Committee on Computational Linguistics， 2020： 1551-1560.
20	NAN G， GUO Z， SEKULIC I， et al. Reasoning with latent structure refinement for document-level relation extraction［C］// Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2020：1546-1557.
21	WANG D， HU W， CAO E， et al. Global-to-local neural networks for document-level relation extraction［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2020：3711-3721.
22	SOROKIN D， GUREVYCH I. Context-aware representations for knowledge base relation extraction［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2017：1784-1789.
23	ZENG S， XU R， CHANG B， et al. Double graph based reasoning for document-level relation extraction［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2020：1630-1640.
24	XU W， CHEN K， ZHAO T.Document-level relation extraction with reconstruction ［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 14167-14175.
25	ZHANG Z， YU B， SHU X， et al. Document-level relation extraction with dual-tier heterogeneous graph［C］// Proceedings of the 28th International Conference on Computational Linguistics. ［S.l.］： International Committee on Computational Linguistics， 2020： 1630-1641.
26	ZHOU H， XU Y， YAO W， et al. Global context-enhanced graph convolutional networks for document-level relation extraction［C］// Proceedings of the 28th International Conference on Computational Linguistics. ［S.l.］： International Committee on Computational Linguistics， 2020：5259-5270.
27	TANG H， CAO Y， ZHANG Z， et al. HIN： hierarchical inference network for document-level relation extraction［C］// Proceedings of the 2020 Pacific-Asia Conference on Knowledge Discovery and Data Mining， LNCS 12084. Cham： Springer， 2020： 197-209.
28	HUANG K， QI P， WANG G， et al. Entity and evidence guided document-level relation extraction［C］// Proceedings of the 6th Workshop on Representation Learning for NLP. Stroudsburg： ACL， 2021： 307-315.
29	XIE Y， SHEN J， LI S， et al. Eider： evidence-enhanced document-level relation extraction ［C］// Findings of the Association for Computational Linguistics： ACL 2022. Stroudsburg： ACL， 2022： 257-268.
30	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
31	YAO Y， YE D M， LI P， et al. DocRED： a large-scale document-level relation extraction dataset［C］// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg： ACL， 2019：764-777.
32	TAN Q， XU L， BING L， et al. Revisiting DocRED-addressing the false negative problem in relation extraction［C］// Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2022： 8472-8487.
33	ZAPOROJETS K， DELEU J， DEVELDER C， et al. DWIE： an entity-centric dataset for multi-task document-level information extraction［J］. Information Processing and Management， 2021， 58（4）： No.102563.
34	WOLF T， DEBUT L， SANH V， et al. Transformers： state-of-the-art natural language processing［C］// Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing： System Demonstrations. Stroudsburg： ACL， 2020： 38-45.
35	DEVLIN J， CHANG M W， LEE K T， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg： ACL， 2019：4171-4186.
36	YU J， YANG D， TIAN S. Relation-specific attentions over entity mentions for enhanced document-level relation extraction［C］// Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2022：1523-1529.

数据集	文档平均三元组数	三元组总数	无证据三元组总数
DocRED	12.5	38 180	1 421（3.7%）
Re-DocRED	28.1	85 932	38 670（45.0%）

数据集	文档平均三元组数	三元组总数	无证据三元组总数
DocRED	12.5	38 180	1 421（3.7%）
Re-DocRED	28.1	85 932	38 670（45.0%）

模型	训练阶段	批次数	批处理数	编码器学习率	分类器学习率
DREERE-BERT	stage1	60	8	0.000 030	0.000 10
DREERE-BERT	stage2	52	8	0.000 010	0.000 10
DREERE-RoBERTa	stage1	60	8	0.000 020	0.000 10
DREERE-RoBERTa	stage2	52	8	0.000 003	0.000 05

模型	训练阶段	批次数	批处理数	编码器学习率	分类器学习率
DREERE-BERT	stage1	60	8	0.000 030	0.000 10
DREERE-BERT	stage2	52	8	0.000 010	0.000 10
DREERE-RoBERTa	stage1	60	8	0.000 020	0.000 10
DREERE-RoBERTa	stage2	52	8	0.000 003	0.000 05

模型	PrLM	开发集			测试集
模型	PrLM	ign-F1	F1	evi-F1	ign-F1	F1	evi-F1
LSR	BERT-base	52.41	59.00	─	56.97	59.50	─
GAIN		59.14	61.22	─	59.00	61.24	─
HeterGSAN		58.13	60.18	─	57.12	59.45	─
SSAN		56.68	58.95	─	56.06	58.41	─
Coref-BERT		55.32	57.51	─	54.54	56.96	─
ATLOP		59.22	61.09	─	59.31	61.30	─
E2GRE		55.22	58.72	47.12	─	─	─
DREEAM		59.60	61.42	52.08	59.12	61.32	51.71
DREERE‑BERT		59.96	61.84	52.40	59.50	61.48	52.09
RoBERTa	RoBERTa-large	57.19	59.40	─	57.74	60.06	─
SSAN		60.25	62.08	─	59.47	61.42	─
ATLOP		61.32	63.18	─	61.39	63.40	─
DREEAM		61.71	63.49	54.15	61.62	63.55	54.01
DREERE‑RoBERTa		61.85	63.73	54.18	61.69	63.61	54.09