Relation extraction method combining semantic enhancement and perception attention

doi:10.11772/j.issn.1001-9081.2024060776

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (6): 1801-1808.DOI: 10.11772/j.issn.1001-9081.2024060776

• Artificial intelligence • Previous Articles

Relation extraction method combining semantic enhancement and perception attention

Dawei YANG¹, Xihai XU², Wei SONG¹^,³()

^1.School of Artificial Intelligence and Computer Science，Jiangnan University，Wuxi Jiangsu 214122，China
^2.Wuxi Autolink Information Technology Company Limited，Wuxi Jiangsu 214125，China
^3.Jiangsu Provincial Engineering Laboratory of Pattern Recognition and Computational Intelligence （Jiangnan University），Wuxi Jiangsu 214122，China

Received:2024-06-12 Revised:2024-08-08 Accepted:2024-08-16 Online:2024-09-10 Published:2025-06-10
Contact: Wei SONG
About author:YANG Dawei， born in 1995， M. S. candidate. His research interests include nature language processing， relation extraction.
XU Xihai， born in 1986， M. S. His research interests include intelligent driving， development of intelligent driving products.
SONG Wei， born in 1981， Ph. D.， professor. His research interests include data mining， machine learning， pattern recognition.
Supported by:
National Natural Science Foundation of China(62076110)

结合语义增强和感知注意力的关系抽取方法

杨大伟¹, 徐西海², 宋威¹^,³()

^1.江南大学人工智能与计算机学院，江苏无锡 214122
^2.无锡车联天下信息技术有限公司，江苏无锡 214125
^3.江苏省模式识别与计算智能工程实验室（江南大学），江苏无锡 214122

通讯作者: 宋威
作者简介:杨大伟（1995—），男，山东泰安人，硕士研究生，CCF会员，主要研究方向：自然语言处理、关系抽取
徐西海（1986—），男，山东泰安人，硕士，主要研究方向：汽车智能驾驶、智能驾驶产品开发
宋威（1981—），男，湖北施恩人，教授，博士，主要研究方向：数据挖掘、机器学习、模式识别。songwei@jiangnan.edu.cn
基金资助:
国家自然科学基金资助项目(62076110)

Abstract

Abstract:

Focusing on the issues that text feature extraction lacks consideration of the contextual discriminative features of sentences and fails to fully utilize the association information among instances and relation labels， a method combining Semantic enhancement and Perception attention for Relation Extraction （SPRE） was proposed. Firstly， during the sentence feature encoding phase， a Semantic Enhancement Mechanism （SEM） was constructed to extract salient semantic features of sentences， and a salient information enhanced sentence representation was obtained through entity-aware word embeddings and Salient Feature Perception （SFP）. Then， a Perception Attention Mechanism （PAM） was designed to integrate sentence features. In the mechanism， the matching degree among sentences and relation labels was evaluated by perceiving the semantic information among sentences and relation labels， the consistency information among entity types of sentences and the corresponding relations， and the similarity information among sentences， so as to fully utilize the dependencies between instances and relation labels in a bag， thereby further enhancing noise reduction capability of the method. Finally， after conducting relation prediction by a classifier， the network parameters were adjusted according to cross-entropy between the predicted results and the actual results. Experimental results on NYT-10 （New York Times 10） and GDS （Google Distant Supervision） datasets show that on NYT-10 dataset， compared with the BERT （Bidirectional Encoder Representations from Transformers）-based relation extraction method PARE （Passage-Attended Relation Extraction）， the proposed method achieves an Area Under Curve （AUC） increase of 2.1 percentage points and an average precision Precision@N （P@N） — P@M increase of 2.4 percentage points for the top 100， 200， and 300 data entries ranked in descending order of confidence； on GDS dataset， the AUC and P@M of the proposed method are 90.5% and 97.8% respectively. The proposed method outperforms mainstream distant supervision relation extraction methods on both datasets significantly， verifying the effectiveness of this method. It can be seen that in mainstream distant supervision relation extraction tasks， the proposed method can enhance the model’s ability to learn data features effectively.

Key words: distant supervision, relation extraction, semantic enhancement, perception attention, noise reduction

摘要：

针对文本特征提取时缺乏考虑句子的上下文判别性特征以及未能充分利用实例和关系标签之间的关联信息的问题，提出一种结合语义增强和感知注意力的关系抽取方法（SPRE）。首先，在句子特征编码阶段，构建语义增强机制（SEM）提取句子的显著性语义特征，通过实体感知词嵌入和显著特征感知（SFP）得到显著信息增强的句子表示；其次，设计感知注意力机制（PAM）整合句子特征，通过感知句子与关系标签之间的语义信息、句子的实体类型与对应关系的实体类型之间的一致性信息，以及句子之间的相似性信息评估句子与关系标签的匹配程度，充分利用包中实例与关系标签的依赖关系，进一步提高方法的降噪能力；最后，利用分类器进行关系预测并根据预测结果与实际结果的交叉熵调整网络参数。在NYT-10（New York Times 10）和GDS（Google Distant Supervision）数据集上的实验结果表明，在NYT-10数据集上，与基于BERT（Bidirectional Encoder Representations from Transformers）的关系抽取方法PARE（Passage-Attended Relation Extraction）相比，所提方法在曲线下面积（AUC）上提升了2.1个百分点，在按置信度降序排列后前100、200 和300条数据精确率Precision@N（P@N）的平均值P@M上提升了2.4个百分点；在GDS数据集上，所提方法的AUC和P@M分别达到了90.5%和97.8%。所提方法在上述2个数据集上均明显优于主流的远程监督关系抽取方法，验证了该方法的有效性。可见，在主流的远程监督关系抽取任务中，所提方法能有效地提升模型对数据特征的学习能力。

关键词: 远程监督, 关系抽取, 语义增强, 感知注意力, 降噪

CLC Number:

TP391.1

Dawei YANG, Xihai XU, Wei SONG. Relation extraction method combining semantic enhancement and perception attention[J]. Journal of Computer Applications, 2025, 45(6): 1801-1808.

杨大伟, 徐西海, 宋威. 结合语义增强和感知注意力的关系抽取方法[J]. 《计算机应用》唯一官方网站, 2025, 45(6): 1801-1808.

Figures/Tables 11

Tab. 1 Examples of RE

S1：Bill Gates is the principal founder of

Microsoft.

founder

True

S2： Bill Gates founded Microsoft in 1975.

founded

True

S3：Bill Gates speaking at a Microsoft held ……

—

False

Fig.1 Framework of SPRE

Fig. 2 Structure of SEM

Tab. 2 Statistics of NYT-10 and GDS datasets

数据集	训练集		测试集
数据集	实体对数	示例数	实体对数	示例数
NYT-10	293 003	570 088	96 678	172 448
GDS	6 498	11 297	3 247	5 663

Tab. 3 Hyperparameter setting of two datasets

超参数	值
超参数	NYT-10数据集	GDS数据集
词嵌入维度 $d w$	50	50
卷积核数k	230	230
位置嵌入维度	5	5
卷积核大小	3	3
超参数λ	20	17
GCN输入层数	100	150
GCN的隐藏层数	750	900
GCN的输出层数	1 250	150
分类器输入层数	690	300
丢失率dropout	0.5	0.5
超参数 $γ$	0.1	0.1
学习率	0.5	0.5
batch size	160	160

Tab. 3 Hyperparameter setting of two datasets

超参数	值
超参数	NYT-10数据集	GDS数据集
词嵌入维度 $d w$	50	50
卷积核数k	230	230
位置嵌入维度	5	5
卷积核大小	3	3
超参数λ	20	17
GCN输入层数	100	150
GCN的隐藏层数	750	900
GCN的输出层数	1 250	150
分类器输入层数	690	300
丢失率dropout	0.5	0.5
超参数 $γ$	0.1	0.1
学习率	0.5	0.5
batch size	160	160

Fig. 3 PR curves of different methods on two datasets

Tab. 4 P@N （N=100， 200， 300）， P@M and AUC of SPRE and comparison methods on NYT-10 and GDS datasets

方法	NYT-10					GDS
方法	P@100	P@200	P@300	P@M	AUC	P@100	P@200	P@300	P@M	AUC
PCNN+ATT	72.9	71.5	69.6	71.3	38.4	96.4	93.3	91.5	93.7	79.9
PCNN+ATT+ENT	83.0	80.0	74.7	79.2	44.8	93.9	93.7	93.5	93.7	84.9
MUTICAST	83.7	79.2	74.2	79.0	40.2	—	—	—	—	—
FAN	85.8	83.4	79.9	83.0	44.8	—	—	—	—	—
DSRE‑VAE	84.0	77.0	75.3	78.8	43.5	96.9	96.7	96.3	96.6	87.6
CIL	81.5	75.5	72.1	76.4	42.1	97.0	96.5	96.5	96.6	90.2
HiCLRE	82.0	78.5	74.0	78.2	45.3	—	—	—	—	—
CGRE	88.9	86.4	81.8	85.7	47.4	98.0	96.7	96.5	97.0	90.3
PARE	90.0	84.0	82.3	85.4	47.5	98.5	97.5	97.0	97.7	90.4
SPRE	92.0	88.0	83.3	87.8	49.6	98.5	98.0	97.0	97.8	90.5

Fig. 4 PR curves of SPRE and three ablation methods on two datasets

Tab. 5 of P@N （N=100， 200， 300）， P@M and AUC of SPRE and ablation methods on NYT-10 and GDS datasets

方法	NYT-10					GDS
方法	P@100	P@200	P@300	P@M	AUC	P@100	P@200	P@300	P@M	AUC
SPRE w/o PAM	90.0	84.0	78.7	84.2	45.7	98.0	95.0	95.3	96.1	88.3
SPRE w/o SFP	89.0	85.0	80.7	84.9	48.2	98.0	97.0	93.7	96.2	89.6
SPRE w/o all	83.0	80.0	74.7	79.2	44.8	93.9	93.7	93.5	94.2	84.9
SPRE	92.0	88.0	83.3	87.8	49.6	98.5	98.0	97.0	97.8	90.5

Tab. 6 P@N and AUC of SPRE under different γ values on NYT-10 dataset

$γ$	P@100	P@200	P@300	P@M	AUC
0.00	90.0	85.5	84.3	86.6	47.7
0.05	91.0	86.5	85.3	87.6	48.1
0.10	92.0	88.0	83.3	87.8	49.6
0.15	90.0	90.5	85.0	88.5	48.7
0.20	87.0	81.0	81.3	83.1	47.8

Tab. 6 P@N and AUC of SPRE under different γ values on NYT-10 dataset

$γ$	P@100	P@200	P@300	P@M	AUC
0.00	90.0	85.5	84.3	86.6	47.7
0.05	91.0	86.5	85.3	87.6	48.1
0.10	92.0	88.0	83.3	87.8	49.6
0.15	90.0	90.5	85.0	88.5	48.7
0.20	87.0	81.0	81.3	83.1	47.8

Fig. 5 PR curves of SPRE under different γ values on NYT-10 dataset

References 31

1	KEJRIWAL M， SEQUEDA J， LOPZ V. Knowledge graphs： construction management and querying ［J］. Semantic Web， 2019， 10（6）： 961-962.
2	SUN Y， TANG D， DUAN N， et al. Joint learning of question answering and question generation［J］. IEEE Transactions on Knowledge and Data Engineering， 2020， 32（5）： 971-982.
3	ALLAHYARI M， POURIYEH S， ASSEFI M， et al. Text summarization techniques： a brief survey ［J］. International Journal of Advanced Computer Science and Applications， 2017， 8（10）： 397-405.
4	郭喜跃，何婷婷. 信息抽取研究综述［J］. 计算机科学， 2015， 42（2）： 14-17.
	GUO X Y， HE T T. Survey about research on information extraction［J］. Computer Science， 2015， 42（2）： 14-17.
5	MINTZ M， BILLS S， SNOW R， et al. Distant supervision for relation extraction without labeled data［C］// Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP. Stroudsburg： ACL， 2009： 1003-1011.
6	BOLLACKER K， EVANS C， PARITOSH P， et al. Freebase： a collaboratively created graph database for structuring human knowledge［C］// Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York： ACM， 2008： 1247-1250.
7	RIEDEL S， YAO L， McCALLUM A K. Modeling relations and their mentions without labeled text［C］// Proceedings of the 2010 European Conference on Machine Learning and Knowledge Discovery in Databases， LNCS 6323. Berlin： Springer， 2010： 148-163.
8	ZENG D， LIU K， CHEN Y， et al. Distant supervision for relation extraction via piecewise convolutional neural networks［C］// Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2015： 1753-1762.
9	LIN Y， SHEN S， LIU Z， et al. Neural relation extraction with selective attention over instances［C］// Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2016： 2124-2133.
10	VASWANI A， SHAZEER N， PARMAR N. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
11	XING R， LUO J. Distant supervised relation extraction with separate head-tail CNN ［C］// Proceedings of the 5th Workshop on Noisy User-generated Text. Stroudsburg： ACL， 2019： 249-258.
12	VASHISHTH S， JOSHI R， PRAYAGA S S， et al. Reside： improving distantly-supervised neural relation extraction using side information ［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2018： 1257-1266.
13	JAT S， KHANDELWAL S， TALUKAR P. Improving distantly supervised relation extraction using word and entity based attention［EB/OL］. ［2024-05-13］..
14	鄂海红，张文静，肖思琪，等. 深度学习实体关系抽取研究综述［J］. 软件学报， 2019， 30（6）： 1793-1818.
	E H H， ZHANG W J， XIAO S Q， et al. Survey of entity relationship extraction based on deep learning ［J］. Journal of Software， 2019， 30（6）： 1793-1818.
15	江南大学. 基于去噪卷积神经网络的远程监督实体关系抽取方法： 201911306495.X［P］. 2020-04-28.
	Jiangnan University. Remote supervision entity relation extraction method based on denoising convolutional neural networks： 201911306495.X［P］. 2020-04-28.
16	江南大学. 基于改进深度残差网络和注意力机制的实体关系抽取方法： 201910880164.0［P］. 2019-12-27.
	Jiangnan University. Entity relation extraction method based on improved deep residual networks and attention mechanism： 201910880164.0 ［P］. 2019-12-27.
17	LIU T， WANG K， CHANG B， et al. A soft-label method for noise tolerant distantly supervised relation extraction［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2017： 1790-1795.
18	QIN P， XU W， WANG W Y. DSGAN： generative adversarial training for distant supervision relation extraction［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2018： 496-505.
19	FENG J， HUANG M， ZHAO L， et al. Reinforcement learning for relation classification from noisy data ［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 5779-5786.
20	LIANG T M， LIU Y， ZHANG H， et al. Distantly-supervised long-tailed relation extraction using constraint graphs［J］. IEEE Transactions on Knowledge and Data Engineering， 2023， 35（7）： 6852-6865.
21	SONG W， GU W， ZHU F， et al. Interaction-and-response network for distantly supervised relation extraction［J］. IEEE Transactions on Neural Networks and Learning Systems， 2024， 35（7）： 9523-9537.
22	MIKOLOV T， CHEN K， CORRADO G， et al. Efficient estimation of word representations in vector space［EB/OL］. ［2024-05-10］..
23	LI Y， LONG G， SHEN T， et al. Self-attention enhanced selective gate with entity-aware embedding for distantly supervised relation extraction［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 8269-8276.
24	宋威，朱富鑫. 基于全局和局部特征感知网络的关系提取方法［J］. 中文信息学报， 2020， 34（11）： 96-103.
	SONG W， ZHU F X. Global and local feature-aware network for relation extraction［J］. Journal of Chinese Information Processing， 2020， 34（11）： 96-103.
25	SRIVASTAVA N， HINTON G， KRIZHEVSKY A， et al. Dropout： a simple way to prevent neural networks from overfitting［J］. Journal of Machine Learning Research， 2014， 15： 1929-1958.
26	CHEN T， SHI H， LIU L， et al. Empower distantly supervised relation extraction with collaborative adversarial training［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 12675-12682.
27	HAO K， YU B， HU W. Knowing false negatives： an adversarial training method for distantly supervised relation extraction［C］// Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2021： 9661-9672.
28	CHRISTOPOULOU F， MIWA M， ANANIADOU S. Distantly supervised relation extraction with sentence reconstruction and knowledgebase priors ［C］// Proceedings of 2021 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2021： 11-26.
29	CHEN T， SHI H， TANG S， et al. CIL： contrastive instance learning framework for distantly supervised relation extraction ［C］// Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2021： 6191-6200.
30	LI D， ZHANG T， HU N， et al. HiCLRE： a hierarchical contrastive learning framework for distantly supervised relation extraction［C］// Findings of the Association for Computational Linguistics： ACL 2022. Stroudsburg： ACL， 2022：2567-2578.
31	RATHORE V， BADOLA K， SINGLA P， et al. PARE： a simple and strong baseline for monolingual and multilingual distantly supervised relation extraction ［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics （Volume 2： Short Papers）. Stroudsburg： ACL， 2022：340-354.

[1]	Jie HU, Cui WU, Jun SUN, Yan ZHANG. Document-level relation extraction model based on anaphora and logical reasoning [J]. Journal of Computer Applications, 2025, 45(5): 1496-1503.
[2]	Bin LI, Min LIN, Siriguleng, Yingjie GAO, Yurong WANG, Shujun ZHANG. Joint entity-relation extraction method for ancient Chinese books based on prompt learning and global pointer network [J]. Journal of Computer Applications, 2025, 45(1): 75-81.
[3]	Yubo ZHAO, Liping ZHANG, Sheng YAN, Min HOU, Mao GAO. Relation extraction between discipline knowledge entities based on improved piecewise convolutional neural network and knowledge distillation [J]. Journal of Computer Applications, 2024, 44(8): 2421-2429.
[4]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[5]	Yuan TANG, Yanping CHEN, Ying HU, Ruizhang HUANG, Yongbin QIN. Relation extraction model based on multi-scale hybrid attention convolutional neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2011-2017.
[6]	Chao WEI, Yanping CHEN, Kai WANG, Yongbin QIN, Ruizhang HUANG. Relation extraction method based on mask prompt and gated memory network calibration [J]. Journal of Computer Applications, 2024, 44(6): 1713-1719.
[7]	Quan YUAN, Changping CHEN, Ze CHEN, Linfeng ZHAN. Twice attention mechanism distantly supervised relation extraction based on BERT [J]. Journal of Computer Applications, 2024, 44(4): 1080-1085.
[8]	Andi GUO, Zhen JIA, Tianrui LI. High-precision entity and relation extraction in medical domain based on pseudo-entity data augmentation [J]. Journal of Computer Applications, 2024, 44(2): 393-402.
[9]	Kezheng CHEN, Xiaoran GUO, Yong ZHONG, Zhenping LI. Relation extraction method based on negative training and transfer learning [J]. Journal of Computer Applications, 2023, 43(8): 2426-2430.
[10]	Shengwei MA, Ruizhang HUANG, Lina REN, Chuan LIN. Structured deep text clustering model based on multi-layer semantic fusion [J]. Journal of Computer Applications, 2023, 43(8): 2364-2369.
[11]	Menglin HUANG, Lei DUAN, Yuanhao ZHANG, Peiyan WANG, Renhao LI. Prompt learning based unsupervised relation extraction model [J]. Journal of Computer Applications, 2023, 43(7): 2010-2016.
[12]	Jingsheng LEI, Kaijun LA, Shengying YANG, Yi WU. Joint entity and relation extraction based on contextual semantic enhancement [J]. Journal of Computer Applications, 2023, 43(5): 1438-1444.
[13]	Shunhang CHENG, Zhihua LI, Tao WEI. Threat intelligence entity relation extraction method integrating bootstrapping and semantic role labeling [J]. Journal of Computer Applications, 2023, 43(5): 1445-1453.
[14]	Quan YUAN, Yunpeng XU, Chengliang TANG. Document-level relation extraction method based on path labels [J]. Journal of Computer Applications, 2023, 43(4): 1029-1035.
[15]	Liang XU, Chun ZHANG, Ning ZHANG, Xuetao TIAN. Zero-shot relation extraction model via multi-template fusion in Prompt [J]. Journal of Computer Applications, 2023, 43(12): 3668-3675.

Relation extraction method combining semantic enhancement and perception attention

结合语义增强和感知注意力的关系抽取方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 31

Related Articles 15

Recommended Articles

Metrics