Open-world knowledge reasoning model based on path and enhanced triplet text

doi:10.11772/j.issn.1001-9081.2024030265

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (4): 1177-1183.DOI: 10.11772/j.issn.1001-9081.2024030265

• Artificial intelligence • Previous Articles Next Articles

Open-world knowledge reasoning model based on path and enhanced triplet text

Liqin WANG¹^,²^,³, Zhilei GENG¹, Yingshuang LI⁴(), Yongfeng DONG¹^,²^,³, Meng BIAN⁵

^1.School of Artificial Intelligence，Hebei University of Technology，Tianjin 300401，China
^2.Hebei Province Key Laboratory of Big Data Calculation （Hebei University of Technology），Tianjin 300401，China
^3.Hebei Engineering Research Center of Data-Driven Industrial Intelligence （Hebei University of Technology），Tianjin 300401，China
^4.Information Security and Technology Service Center，Hebei University of Technology，Tianjin 300401，China
^5.Tianjin Nankai District Teacher Development Center，Tianjin 300401，China

Received:2024-03-13 Revised:2024-04-19 Accepted:2024-04-22 Online:2024-07-01 Published:2025-04-10
Contact: Yingshuang LI
About author:WANG Liqin， born in 1980， Ph. D.， senior experimentalist. Her research interests include intelligent information processing， knowledge graph.
GENG Zhilei， born in 1998， M. S. candidate. His research interests include natural language processing， knowledge graph.
DONG Yongfeng， born in 1977， Ph. D.， professor. His research interests include artificial intelligence， knowledge graph.
BIAN Meng， born in 1980， M. S.， senior teacher. Her research interests include modern education.
Supported by:
Science and Technology Research Program of Higher Education Institutions in Hebei Province(ZD2022082);Higher Education Teaching Reform Research and Practice Project of Hebei Province(2022GJJG049)

基于路径和增强三元组文本的开放世界知识推理模型

王利琴¹^,²^,³, 耿智雷¹, 李英双⁴(), 董永峰¹^,²^,³, 边萌⁵

^1.河北工业大学人工智能与数据科学学院，天津 300401
^2.河北省大数据计算重点实验室（河北工业大学），天津 300401
^3.河北省数据驱动工业智能工程研究中心（河北工业大学），天津 300401
^4.河北工业大学信息安全与技术服务中心，天津 300401
^5.天津市南开区教师发展中心，天津 300110

通讯作者: 李英双
作者简介:王利琴（1980—），女，河北张北人，高级实验师，博士，CCF会员，主要研究方向：智能信息处理、知识图谱；
耿智雷（1998—），男，河南商丘人，硕士研究生，主要研究方向：自然语言处理、知识图谱；
董永峰（1977—），男，河北定州人，教授，博士，CCF高级会员，主要研究方向：人工智能、知识图谱；
边萌（1980—），女，天津人，高级教师，硕士，主要研究方向：现代教育。
基金资助:
河北省高等学校科学技术研究项目(ZD2022082);河北省高等教育教学改革研究与实践项目(2022GJJG049)

Abstract

Abstract:

Traditional knowledge reasoning methods based on representation learning can only be used for closed-world knowledge reasoning. Conducting open-world knowledge reasoning effectively is a hot issue currently. Therefore， a knowledge reasoning model based on path and enhanced triplet text， named PEOR （Path and Enhanced triplet text for Open world knowledge Reasoning）， was proposed. First， multiple paths generated by structures between entity pairs and enhanced triplets generated by individual entity neighborhood structures were utilized. Among then， the path text was obtained by concatenating the text of triplets in the path， and the enhanced triplet text was obtained by concatenating the text of head entity neighborhood， relation， and tail entity neighborhood. Then， BERT （Bidirectional Encoder Representations from Transformers） was employed to encode the path text and enhanced triplet text separately. Finally， semantic matching attention calculation was performed using path vectors and triplet vectors， followed by aggregation of semantic information from multiple paths using semantic matching attention. Comparison experimental results on three open-world knowledge graph datasets： WN18RR， FB15k-237， and NELL-995 show that compared with suboptimal model BERTRL （BERT-based Relational Learning）， the proposed model has Hits@10 （Hit ratio） metric improved by 2.6， 2.3 and 8.5 percentage points， respectively， validating the effectiveness of the proposed model.

Key words: Knowledge Graph (KG), text information, pre-trained language model, open-world knowledge reasoning, attention mechanism

摘要：

传统的基于表示学习的知识推理方法只能用于封闭世界的知识推理，有效进行开放世界的知识推理是目前的热点问题。因此，提出一种基于路径和增强三元组文本的开放世界知识推理模型PEOR（Path and Enhanced triplet text for Open world knowledge Reasoning）。首先，使用由实体对间结构生成的多条路径和单个实体周围结构生成的增强三元组，其中路径文本通过拼接路径中的三元组文本得到，而增强三元组文本通过拼接头实体邻域文本、关系文本和尾实体邻域文本得到；其次，使用BERT（Bidirectional Encoder Representations from Transformers）分别编码路径文本和增强三元组文本；最后，使用路径向量和三元组向量计算语义匹配注意力，再使用语义匹配注意力聚合多条路径的语义信息。在3个开放世界知识图谱数据集WN18RR、FB15k-237和NELL-995上的对比实验结果表明，与次优模型BERTRL（BERT-based Relational Learning）相比，所提模型的命中率（Hits@10）指标分别提升了2.6、2.3和8.5个百分点，验证了所提模型的有效性。

关键词: 知识图谱, 文本信息, 预训练语言模型, 开放世界知识推理, 注意力机制

CLC Number:

TP391.1

Liqin WANG, Zhilei GENG, Yingshuang LI, Yongfeng DONG, Meng BIAN. Open-world knowledge reasoning model based on path and enhanced triplet text[J]. Journal of Computer Applications, 2025, 45(4): 1177-1183.

王利琴, 耿智雷, 李英双, 董永峰, 边萌. 基于路径和增强三元组文本的开放世界知识推理模型[J]. 《计算机应用》唯一官方网站, 2025, 45(4): 1177-1183.

Figures/Tables 6

References 28

1	徐增林，盛泳潘，贺丽荣，等. 知识图谱技术综述［J］. 电子科技大学学报， 2016， 45（4）： 589-606.
	XU Z L， SHENG Y P， HE L R， et al. Review on knowledge graph techniques［J］. Journal of University of Electronic Science and Technology of China， 2016， 45（4）： 589-606.
2	黄恒琪，于娟，廖晓，等. 知识图谱研究综述［J］. 计算机系统应用， 2019， 28（6）： 1-12.
	HUANG H Q， YU J， LIAO X， et al. Review on knowledge graphs［J］. Computer Systems and Applications， 2019， 28（6）： 1-12.
3	WANG Z， ZHANG J， FENG J， et al. Knowledge graph embedding by translating on hyperplanes［C］// Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2014： 1112-1119.
4	TROUILLON T， WEBLBL J， RIEDEL S， et al. Complex embeddings for simple link prediction［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： JMLR.org， 2016： 2071-2080.
5	DETTMERS T， MINERVINI P， STENETORP P， et al. Convolutional 2D knowledge graph embeddings［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 1811-1818.
6	LAO N， COHEN W W. Relational retrieval using a combination of path-constrained random walks［J］. Machine Learning， 2010， 81（1）： 53-67.
7	LIN X， LIANG Y， GIUNCHIGLIA F， et al. Relation path embedding in knowledge graphs［J］. Neural Computing and Applications， 2019， 31（9）： 5629-5639.
8	YAO L， MAO C， LUO Y. KG-BERT： BERT for knowledge graph completion［EB/OL］. ［2023-12-02］. .
9	DEVLIN J， CHANG M W， LEE K. BERT： pre-training of deep bidirectional Transformers for language understanding［C］// Proceedings of the 2019 Annual Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg： ACL， 2019： 4171-4186.
10	ZHA H， CHEN Z， YAN X. Inductive relation prediction by BERT［C］// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2022： 5923-5931.
11	TERU K K， DENIS E G， HAMILTON W L. Inductive relation prediction by subgraph reasoning［C］// Proceedings of the 37th International Conference on Machine Learning. New York： JMLR.org， 2020： 9448-9457.
12	CHEN J， HE H， WU F， et al. Topology-aware correlations between relations for inductive link prediction in knowledge graphs［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 6271-6278.
13	MAI S， ZHENG S， YANG Y， et al. Communicative message passing for inductive relation reasoning［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 4294-4302.
14	WANG H， REN H， LESKOVEC J. Relational message passing for knowledge graph completion［C］// Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York： ACM， 2021： 1697-1707.
15	LIN Q， LIU J， XU F， et al. Incorporating context graph with logical reasoning for inductive relation prediction［C］// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2022： 893-903.
16	BRONSTEIN M M， BRUNA J， LeCUN Y， et al. Geometric deep learning： going beyond Euclidean data［J］. IEEE Signal Processing Magazine， 2017， 34（4）： 18-42.
17	HAMILTON W L， YING R， LESKOVEC J. Inductive representation learning on large graphs［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 1025-1035.
18	KWAK H， JUNG H B K. Subgraph representation learning with hard negative samples for inductive link prediction［C］// Proceedings of the 2022 IEEE International Conference on Acoustics， Speech and Signal Processing. Piscataway： IEEE， 2022： 4768-4772.
19	ZHENG S， MAI S， SUN Y， et al. Subgraph-aware few-shot inductive link prediction via meta-learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2023， 35（6）： 6512-6517.
20	SOCHER R， CHEN D， MANNING C D， et al. Reasoning with neural tensor networks for knowledge base completion［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2013： 926-934.
21	DAZA D， COCHEZ M， GROTH P. Inductive entity representations from text via link prediction［C］// Proceedings of the 2021 Web Conference. New York： ACM， 2021： 798-808.
22	WANG L， ZHAO W， WEI Z， et al. SimKGC： simple contrastive knowledge graph completion with pre-trained language models［C］// Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2022： 4281-4294.
23	GESESE G A， SACK H， ALAM M. RAILD： towards leveraging relation features for inductive link prediction in knowledge graphs［C］// Proceedings of the 11th International Joint Conference on Knowledge Graphs. New York： ACM， 2022： 82-90.
24	TOUTANOVA K， CHEN D. Observed versus latent features for knowledge base and text inference［C］// Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality. Stroudsburg： ACL， 2015： 57-66.
25	XIONG W， HOANG T， WANG W Y. DeepPath： a reinforcement learning method for knowledge graph reasoning［C］// Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg： ACL， 2017： 564-573.
26	MEILICKE C， FINK M， WANG Y， et al. Fine-grained evaluation of rule-and embedding-based systems for knowledge graph completion［C］// Proceedings of the 2018 International Semantic Web Conference， LNCS 11136. Cham： Springer， 2018： 3-20.
27	YANG F， YANG Z， COHEN W W. Differentiable learning of logical rules for knowledge base reasoning［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 2316-2325.
28	SADEGHIAN A， ARMANDPOUR M， DING P， et al. DRUM： end-to-end differentiable rule mining on knowledge graphs［C］// Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2019： 15347-15357.

版本	划分	WN18RR				FB15k-237				NELL-995
版本	划分	#R	#E	#TR1	#TR2	#R	#E	#TR1	#TR2	#R	#E	#TR1	#TR2
V1	训练集	9	2 746	5 410	630	183	2 000	4 245	489	14	10 915	4 687	414
V1	测试集	9	922	1 618	188	146	1 500	1 993	205	14	225	833	100
V2	训练集	10	6 954	15 262	1 838	203	3 000	9 739	1 166	88	2 564	8 219	922
V2	测试集	10	2 923	4 011	441	176	2 000	4 145	478	79	4 937	4 586	476
V3	训练集	11	12 078	25 901	3 097	218	4 000	17 986	2 194	142	4 647	16 393	1 851
V3	测试集	11	5 084	6 320	605	187	3 000	7 406	865	122	4 921	8 048	809
V4	训练集	9	3 861	7 940	934	222	5 000	27 203	3 352	14	10 915	4 687	414
V4	测试集	9	7 208	12 334	1 429	204	3 500	11 714	1 424	14	225	833	100

版本	划分	WN18RR				FB15k-237				NELL-995
版本	划分	#R	#E	#TR1	#TR2	#R	#E	#TR1	#TR2	#R	#E	#TR1	#TR2
V1	训练集	9	2 746	5 410	630	183	2 000	4 245	489	14	10 915	4 687	414
V1	测试集	9	922	1 618	188	146	1 500	1 993	205	14	225	833	100
V2	训练集	10	6 954	15 262	1 838	203	3 000	9 739	1 166	88	2 564	8 219	922
V2	测试集	10	2 923	4 011	441	176	2 000	4 145	478	79	4 937	4 586	476
V3	训练集	11	12 078	25 901	3 097	218	4 000	17 986	2 194	142	4 647	16 393	1 851
V3	测试集	11	5 084	6 320	605	187	3 000	7 406	865	122	4 921	8 048	809
V4	训练集	9	3 861	7 940	934	222	5 000	27 203	3 352	14	10 915	4 687	414
V4	测试集	9	7 208	12 334	1 429	204	3 500	11 714	1 424	14	225	833	100

模型	WN18RR					FB15k-237					NELL-995
模型	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值
RuleN	90.2	89.0	76.4	85.7	85.8	75.2	88.7	91.2	91.7	86.7	84.9	88.4	87.2	80.5	85.2
Neural LP	86.0	83.7	62.9	82.0	78.6	69.6	76.5	73.9	75.7	73.9	64.6	83.6	87.5	85.6	80.3
DRUM	86.0	84.0	63.2	82.0	78.3	69.7	76.4	74.0	76.2	74.0	59.8	83.9	87.7	85.9	79.3
GraIL	94.3	94.1	85.8	92.7	91.7	84.6	90.5	91.6	94.4	90.2	86.0	92.6	93.3	87.5	89.8
ConGLR	99.5	99.6	93.7	99.8	98.1	85.6	92.3	93.9	95.0	91.7	86.4	95.2	96.1	88.4	91.5
KG-BERT	85.7	93.0	83.2	88.1	87.5	76.3	90.8	81.5	87.7	84.0	53.6	77.4	83.7	69.1	71.0
BERTRL	96.9	98.2	92.3	98.9	96.5	89.9	93.5	94.3	92.8	92.3	81.8	94.4	95.4	87.5	89.7
PEOR	99.6	99.7	95.8	99.9	98.7	93.7	96.9	96.8	98.6	96.5	90.0	96.6	97.8	92.4	94.7

模型	WN18RR					FB15k-237					NELL-995
模型	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值
RuleN	90.2	89.0	76.4	85.7	85.8	75.2	88.7	91.2	91.7	86.7	84.9	88.4	87.2	80.5	85.2
Neural LP	86.0	83.7	62.9	82.0	78.6	69.6	76.5	73.9	75.7	73.9	64.6	83.6	87.5	85.6	80.3
DRUM	86.0	84.0	63.2	82.0	78.3	69.7	76.4	74.0	76.2	74.0	59.8	83.9	87.7	85.9	79.3
GraIL	94.3	94.1	85.8	92.7	91.7	84.6	90.5	91.6	94.4	90.2	86.0	92.6	93.3	87.5	89.8
ConGLR	99.5	99.6	93.7	99.8	98.1	85.6	92.3	93.9	95.0	91.7	86.4	95.2	96.1	88.4	91.5
KG-BERT	85.7	93.0	83.2	88.1	87.5	76.3	90.8	81.5	87.7	84.0	53.6	77.4	83.7	69.1	71.0
BERTRL	96.9	98.2	92.3	98.9	96.5	89.9	93.5	94.3	92.8	92.3	81.8	94.4	95.4	87.5	89.7
PEOR	99.6	99.7	95.8	99.9	98.7	93.7	96.9	96.8	98.6	96.5	90.0	96.6	97.8	92.4	94.7

模型	WN18RR					FB15k-237					NELL-995
模型	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值
RuleN	80.8	78.2	53.3	71.5	70.9	49.7	77.8	87.6	85.6	75.1	53.5	81.7	77.2	61.3	68.4
Neural LP	74.3	68.9	46.1	67.1	64.1	52.9	98.9	52.9	55.8	65.1	40.7	78.7	82.7	80.5	70.6
DRUM	74.3	68.9	46.1	67.1	64.1	52.9	58.7	52.9	55.8	55.8	19.4	78.5	82.7	80.5	65.2
GraIL	82.4	78.6	58.4	73.4	73.2	64.1	81.8	82.8	89.2	79.7	58.5	93.2	91.4	73.1	79.0
ConGLR	85.6	92.9	70.7	92.9	85.0	68.2	85.9	88.6	89.3	83.0	81.0	94.9	94.3	81.6	88.4
KG-BERT	81.4	83.9	79.5	85.3	82.5	78.1	84.0	80.6	85.0	82.9	40.7	62.4	61.0	63.6	61.9
BERTRL	91.4	94.1	89.4	93.6	92.1	84.9	93.3	91.2	93.9	91.0	80.9	89.3	94.2	83.6	86.5
PEOR	94.9	96.7	92.4	95.0	94.7	88.0	95.5	94.2	95.7	93.3	90.5	97.6	99.3	92.7	95.0

Open-world knowledge reasoning model based on path and enhanced triplet text

基于路径和增强三元组文本的开放世界知识推理模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 28

Related Articles 15

Recommended Articles

Metrics

[1]	Jie HU, Qiyang ZHENG, Jun SUN, Yan ZHANG. Multi-label classification model based on multi-label relational graph and local dynamic reconstruction learning [J]. Journal of Computer Applications, 2025, 45(4): 1104-1112.
[2]	Shiyue GUO, Jianwu DANG, Yangping WANG, Jiu YONG. 3D hand pose estimation combining attention mechanism and multi-scale feature fusion [J]. Journal of Computer Applications, 2025, 45(4): 1293-1299.
[3]	Liwei ZHANG, Quan LIANG, Yutao HU, Qiaole ZHU. Channel shuffle attention mechanism based on group convolution [J]. Journal of Computer Applications, 2025, 45(4): 1069-1076.
[4]	Kunyuan JIANG, Xiaoxia LI, Li WANG, Yaodan CAO, Xiaoqiang ZHANG, Nan DING, Yingyue ZHOU. Boundary-cross supervised semantic segmentation network with decoupled residual self-attention [J]. Journal of Computer Applications, 2025, 45(4): 1120-1129.
[5]	Chun XU, Shuangyan JI, Huan MA, Enwei SUN, Mengmeng WANG, Mingyu SU. Consultation recommendation method based on knowledge graph and dialogue structure [J]. Journal of Computer Applications, 2025, 45(4): 1157-1168.
[6]	Haijun GENG, Yun DONG, Zhiguo HU, Haotian CHI, Jing YANG, Xia YIN. Encrypted traffic classification method based on Attention-1DCNN-CE [J]. Journal of Computer Applications, 2025, 45(3): 872-882.
[7]	Tianqi ZHANG, Shuang TAN, Xiwen SHEN, Juan TANG. Image watermarking method combining attention mechanism and multi-scale feature [J]. Journal of Computer Applications, 2025, 45(2): 616-623.
[8]	Haiteng MENG, Xiaole ZHAO, Tianrui LI. Lightweight image super-resolution reconstruction based on asymmetric information distillation network [J]. Journal of Computer Applications, 2025, 45(2): 601-609.
[9]	Dixin WANG, Jiahao WANG, Min LI, Hao CHEN, Guangyao HU, Yu GONG. Abnormal attack detection for underwater acoustic communication network [J]. Journal of Computer Applications, 2025, 45(2): 526-533.
[10]	Yan LI, Guanhua YE, Yawen LI, Meiyu LIANG. Enterprise ESG indicator prediction model based on richness coordination technology [J]. Journal of Computer Applications, 2025, 45(2): 670-676.
[11]	Qijian CAI, Wei TAN. Semantic graph enhanced multi-modal recommendation algorithm [J]. Journal of Computer Applications, 2025, 45(2): 421-427.
[12]	Lifang WANG, Jingshuang WU, Pengliang YIN, Lihua HU. Action recognition algorithm based on attention mechanism and energy function [J]. Journal of Computer Applications, 2025, 45(1): 234-239.
[13]	Rui LI, Guanfeng LI, Dezhou HU, Wenxin GAO. Knowledge graph multi-hop reasoning model fusing path and subgraph features [J]. Journal of Computer Applications, 2025, 45(1): 32-39.
[14]	Jie XU, Yong ZHONG, Yang WANG, Changfu ZHANG, Guanci YANG. Facial attribute estimation and expression recognition based on contextual channel attention mechanism [J]. Journal of Computer Applications, 2025, 45(1): 253-260.
[15]	Junying CHEN, Shijie GUO, Lingling CHEN. Lightweight human pose estimation based on decoupled attention and ghost convolution [J]. Journal of Computer Applications, 2025, 45(1): 223-233.

模型	WN18RR					FB15k-237					NELL-995
模型	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值
-Attention	92.0	95.5	89.3	90.7	92.3	83.1	90.2	87.5	85.6	86.3	87.3	92.4	94.5	86.5	90.1
-Path	90.2	93.6	88.9	85.2	89.7	80.7	86.9	83.5	82.6	83.4	85.5	89.4	83.4	84.4	85.6
-Triplet	91.3	93.8	88.9	87.3	90.8	82.6	88.9	85.2	84.3	85.2	86.5	90.2	90.8	85.7	88.3
PEOR	94.5	97.6	92.9	95.6	95.1	88.0	96.9	96.2	95.7	94.7	90.1	95.2	98.8	88.5	93.1

模型	WN18RR					FB15k-237					NELL-995
模型	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值	V1	V2	V3	V4	平均值
-Attention	92.0	95.5	89.3	90.7	92.3	83.1	90.2	87.5	85.6	86.3	87.3	92.4	94.5	86.5	90.1
-Path	90.2	93.6	88.9	85.2	89.7	80.7	86.9	83.5	82.6	83.4	85.5	89.4	83.4	84.4	85.6
-Triplet	91.3	93.8	88.9	87.3	90.8	82.6	88.9	85.2	84.3	85.2	86.5	90.2	90.8	85.7	88.3
PEOR	94.5	97.6	92.9	95.6	95.1	88.0	96.9	96.2	95.7	94.7	90.1	95.2	98.8	88.5	93.1