Multi-relation approximate reasoning model based on uncertain knowledge graph embedding

doi:10.11772/j.issn.1001-9081.2023060762

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (6): 1751-1759.DOI: 10.11772/j.issn.1001-9081.2023060762

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Multi-relation approximate reasoning model based on uncertain knowledge graph embedding

Jianjing LI¹, Guanfeng LI¹(), Feizhou QIN¹, Weijun LI²

^1.School of Information Engineering，Ningxia University，Yinchuan Ningxia 750021，China
^2.School of Computer Science and Engineering，North Minzu University，Yinchuan Ningxia 750021，China

Received:2023-06-14 Revised:2023-08-17 Accepted:2023-08-21 Online:2023-08-30 Published:2024-06-10
Contact: Guanfeng LI
About author:LI Jianjing， born in 1997， M. S. candidate. His research interests include knowledge graph， representation learning， deep learning.
QIN Feizhou， born in 1972， M. S.， professor. Her research interests include graph database.
LI Weijun， born in 1979， Ph. D.， lecturer. His research interests include knowledge engineering， intelligent data processing.
Supported by:
National Natural Science Foundation of China(62066038);Ningxia Natural Science Foundation(2022AAC03026);Graduate Innovation Project of Ningxia University in 2023

基于不确定知识图谱嵌入的多关系近似推理模型

李健京¹, 李贯峰¹(), 秦飞舟¹, 李卫军²

^1.宁夏大学信息工程学院，银川 750021
^2.北方民族大学计算机科学与工程学院，银川 750021

通讯作者: 李贯峰
作者简介:李健京（1997—），男，福建莆田人，硕士研究生，CCF会员，主要研究方向：知识图谱、表示学习、深度学习
秦飞舟（1972—），女，江苏泰兴人，教授，硕士，CCF会员，主要研究方向：图数据库
李卫军（1979—），男，陕西渭南人，讲师，博士，主要研究方向：知识工程、智能数据处理。
基金资助:
国家自然科学基金资助项目(62066038);宁夏自然科学基金资助项目(2022AAC03026);宁夏大学2023研究生创新项目

Abstract

Abstract:

Because the uncertain embedding model of large-scale Knowledge Graph （KG） can not perform approximate reasoning on multiple logical relationships， a multi-relation approximate reasoning model based on Uncertain KG Embedding （UKGE） named UDConEx （Uncertainty DistMult （Distance Multiplicative） and complex Convolution Embedding） was proposed. Firstly， the UDConEx combined the characteristics of DistMult and ComplEx （Complex Embedding）， enabling it to infer symmetric and asymmetric relationships. Subsequently， Convolutional Neural Network（CNN） was employed by the UDConEx to capture the interactive information in the uncertain KG， thereby enabling it to reason inverse and transitive relationships. Lastly， the neural network was employed to carry out confidence learning of uncertain KG information， enabling the UDConEx to perform approximate reasoning within the UKGE space. The experimental results on three public data sets of CN15k， NL27k， and PPI5k show that， compared with MUKGE （Multiplex UKGE） model， the Mean Absolute Error （MAE） of confidence prediction is reduced by 6.3%， 30.1% and 44.9% for CN15k， NL27k and PPI5k respectively； in the task of relation fact ranking， the linear-based Normalized Discounted Cumulative Gain （NDCG） is improved by 5.8% and 2.6% for CN15k and NL27k respectively； in the multi-relation approximate reasoning task， it is verified that the UDConEx has the approximate reasoning ability of multiple logical relationships. The inability of traditional embedding models to predict confidence is compensated for by the UDConEx， which achieves approximate reasoning for multiple logical relationships and offers enhanced accuracy and interpretability in uncertainty KG reasoning.

Key words: Knowledge Graph (KG), multi-relation reasoning, approximate reasoning, uncertainty, Convolutional Neural Network (CNN)

摘要：

针对大规模知识图谱（KG）的不确定性嵌入模型中无法对多种逻辑关系进行近似推理的问题，提出一种基于不确定KG嵌入（UKGE）的多关系近似推理模型UDConEx（Uncertainty DistMult （Distance Multiplicative） and complex Convolution Embedding）。首先，UDConEx结合DistMult和ComplEx（Complex Embedding）模型的特点，使得UDConEx具有推理对称与非对称关系的能力；其次，UDConEx采用卷积神经网络（CNN）捕获不确定性KG中的交互信息，使它具有推理逆关系和传递关系的能力；最后，UDConEx利用神经网络对KG的不确定信息进行置信度学习，在UKGE空间中可以进行近似推理。在CN15k、NL27k和PPI5k这3个公开数据集上的实验结果表明，相较于MUKGE（Multiplex UKGE）模型，UDConEx在CN15k、NL27k和PPI5k的置信度预测任务中平均绝对误差（MAE）分别降低了6.3%，30.1%和44.9%；在关系事实排名任务中，基于线性的归一化折损累计增益（NDCG）在CN15k和NL27k数据集中分别提升了5.8%和2.6%；在多关系近似推理任务中验证了UDConEx具有多种逻辑关系的近似推理能力。UDConEx弥补了传统嵌入模型无法进行置信度预测的不足，实现了对多种逻辑关系的近似推理，具有更精确、具有可解释性的不确定性知识图谱推理能力。

关键词: 知识图谱, 多关系推理, 近似推理, 不确定性, 卷积神经网络

CLC Number:

TP181

Jianjing LI, Guanfeng LI, Feizhou QIN, Weijun LI. Multi-relation approximate reasoning model based on uncertain knowledge graph embedding[J]. Journal of Computer Applications, 2024, 44(6): 1751-1759.

李健京, 李贯峰, 秦飞舟, 李卫军. 基于不确定知识图谱嵌入的多关系近似推理模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1751-1759.

Figures/Tables 6

References 37

1	CHEN X， JIA S， XIANG Y. A review： knowledge reasoning over knowledge graph ［J］. Expert Systems with Applications， 2020， 141： 112948
2	MITCHELL T， COHEN W， HRUSCHKA E， et al. Never-ending learning ［J］. Communications of the ACM， 2018， 61（5）： 103-115.
3	WU W， LI H， WANG H， et al. Probase： a probabilistic taxonomy for text understanding ［C］// Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data. New York： ACM， 2012： 481-492.
4	SPEER R， CHIN J， HAVASI C. ConceptNet 5.5： an open multilingual graph of general knowledge ［C］// Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto： AAAI， 2017： 4444-4451.
5	WANG Q， MAO Z， WANG B， et al. Knowledge graph embedding： a survey of approaches and applications ［J］. IEEE Transactions on Knowledge and Data Engineering， 2017， 29（12）： 2724-2743
6	刘知远，孙茂松，林衍凯，等.知识表示学习研究进展［J］. 计算机研究与发展， 2016， 53（2）： 247-261.
	LIU Z Y， SUN M S， LIN Y K， et al. Knowledge representation learning： a review ［J］. Journal of Computer Research and Development， 2016， 53（2）： 247-261.
7	BORDES A， USUNIER N， GARCIA-DURÁN A， et al. Translating embeddings for modeling multi-relational data［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2013： 2787–2795.
8	YANG B， W-T YIH， HE X， et al. Embedding entities and relations for learning and inference in knowledge bases［EB/OL］. （2015-08-29）［2023-05-30］. .
9	NICKEL M， TRESP V， H-P KRIEGEL. A three-way model for collective learning on multi-relational data ［C］// Proceedings of the 28th International Conference on Machine Learning. Madison： Omnipress， 2011： 809-816.
10	CHEN X， CHEN M， SHI W， et al. Embedding uncertain knowledge graphs ［C］// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2019： 3363-3370.
11	CHEN Z-M， YEH M-Y， T-W KUO. PASSLEAF： a pool-based semi-supervised learning framework for uncertain knowledge graph embedding［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 4019-4026.
12	TROUILLON T， WELBL J， RIEDEL S， et al. Complex embeddings for simple link prediction［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： JMLR.org， 2016： 2071-2080.
13	JI S， PAN S， CAMBRIA E， et al. A survey on knowledge graphs： representation， acquisition， and applications［J］. IEEE Transactions on Neural Networks and Learning Systems， 2023， 33（2）： 494-514.
14	WANG Z， ZHANG J， FENG J， et al. Knowledge graph embedding by translating on hyperplanes ［C］// Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2014： 1112-1119.
15	JI G， HE S， XU L， et al. Knowledge graph embedding via dynamic mapping matrix［C］// Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing （Volume 1： Long Papers）. Stroudsburg： ACL， 2015： 687-696.
16	LIN Y， LIU Z， SUN M， et al. Learning entity and relation embeddings for knowledge graph completion ［C］// Proceedings of the 29th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2015： 2181-2187.
17	SUN Z， DENG Z-H， NIE J-Y， et al. RotatE： knowledge graph embedding by relational rotation in complex space［EB/OL］. ［2023-05-30］. .
18	ZHANG Z， CAI J， ZHANG Y， et al. Learning hierarchy-aware knowledge graph embeddings for link prediction ［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 3065-3072.
19	LU H， HU H， LIN X. DensE： an enhanced non-commutative representation for knowledge graph embedding with adaptive semantic hierarchy ［J］. Neurocomputing， 2022， 476： 115-125.
20	ZHANG Q， WANG R， YANG J， et al. Structural context-based knowledge graph embedding for link prediction［J］. Neurocomputing， 2022， 470： 109-120.
21	ZHANG Y， YAO Q， DAI W， et al. AutoSF： searching scoring functions for knowledge graph embedding ［C］// Proceedings of the 2020 IEEE 36th International Conference on Data Engineering. Piscataway： IEEE， 2020： 433-444.
22	FAN M， ZHOU Q， ZHENG T F. Learning embedding representations for knowledge inference on imperfect and incomplete repositories ［C］// Proceedings of the 2016 IEEE/WIC/ACM International Conference on Web Intelligence. Piscataway： IEEE， 2016： 42-48.
23	KERTKEIDKACHORN N， LIU X， ICHISE R. GTransE： generalizing translation-based model on uncertain knowledge graph embedding ［C］// Proceedings of the 2019 Annual Conference of the Japanese Society for Artificial Intelligence. Cham： Springer， 2019： 170-178.
24	HU J， CHENG R， HUANG Z， et al. On embedding uncertain graphs［C］// Proceedings of the 2017 ACM Conference on Information and Knowledge Management. New York： ACM， 2017： 157-166.
25	PAI S， COSTABELLO L. Learning embeddings from knowledge graphs with numeric edge attributes ［C/OL］// Proceedings of the 29th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2021： 2869-2875.
26	YANG S， ZHANG W， TANG R， et al. Approximate inferring with confidence predicting based on uncertain knowledge graph embedding ［J］. Information Sciences， 2022， 609： 679-690.
27	CHEN X， BORATKO M， CHEN M， et al. Probabilistic box embeddings for uncertain knowledge graph reasoning［C］// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2021： 882-893.
28	ABBOUD R， CEYLAN I I， LUKASIEWICZ T， et al. BoxE： a box embedding model for knowledge base completion ［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2020： 9649-9661.
29	徐遥，何世柱，刘康，等.基于Beta分布和半监督学习的非确定性知识图谱嵌入模型［J］.中文信息学报， 2022， 36（10）：54-62.
	XU Y， HE S Z， LIU K， et al. Uncertain knowledge graph embedding by Beta distribution and semi-supervised learning ［J］. Journal of Chinese Information Processing， 2022， 36（10）：54-62.
30	REN H Y， LESKOVEC J. Beta embeddings for multi-hop logical reasoning in knowledge graphs［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2020： 19716-19726.
31	LI J， LI G. Analysis of semantic-based knowledge graph embedding models in uncertainty knowledge graphs［C］// Proceedings of the 2022 IEEE 24th International Conference on High Performance Computing & Communications； 8th International Conference on Data Science & Systems； 20th International Conference on Smart City； 8th International Conference on Dependability in Sensor， Cloud & Big Data Systems & Application. Piscataway： IEEE， 2022： 2084-2092.
32	ZHANG X， MA Z. Fuzzy RDF knowledge graph embeddings through vector space model［J］. IEEE Transactions on Fuzzy Systems， 2023， 31（3）： 835-844.
33	LIU Q， ZHANG Q， ZHAO F， et al. Uncertain knowledge graph embedding： an effective method combining multi-relation and multi-path［J］. Frontiers of Computer Science， 2024， 18（3）： 183311.
34	DETTMERS T， MINERVINI P， STENETORP P， et al. Convolutional 2D knowledge graph embeddings ［C］// Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto： AAAI Press， 2018： 1811-1818.
35	NGUYEN D Q， NGUYEN T D， NGUYEN D Q， et al. A novel embedding model for knowledge base completion based on convolutional neural network［C］// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 2 （Short Papers）. Stroudsburg： ACL， 2018： 327-333.
36	SZKLARCZYK D， MORRIS J H， COOK H， et al. The STRING database in 2017： quality-controlled protein-protein association networks， made broadly accessible ［J］. Nucleic Acids Research， 2016： 45（1）： 362-368.
37	LIU T-Y. Learning to rank for information retrieval［J］. Foundations and Trends^® in Information Retrieval， 2009：3（3）： 225-331.

模型	CN15k		NL27k		PPI5k
模型	MSE	MAE	MSE	MAE	MSE	MAE
UKGE_rect	8.61	19.90	2.36	6.90	0.95	3.79
UKGE_logi	9.86	20.05	2.67	7.03	0.96	4.09
UComplEx	8.11*	20.17*	1.65*	4.80*	0.53*	3.69*
UKG _S E	7.71	21.34	4.11*	9.79*	0.98	5.98
BEUrRE	7.49	19.88	2.01	6.89	0.94*	3.67*
UBetaE	7.20	18.85*	2.11*	6.93*	0.76*	3.74*
MUKGE_rect	7.05	19.73	1.63	6.67	0.50	3.61
MUKGE_logi	9.99	20.42	2.11	6.12	0.66	4.21
UDConEx_logi	7.70	18.93	1.47	4.67	0.30	2.06
UDConEx_rect	7.04	18.48	1.44	4.66	0.30	1.99

模型	CN15k		NL27k		PPI5k
模型	MSE	MAE	MSE	MAE	MSE	MAE
UKGE_rect	8.61	19.90	2.36	6.90	0.95	3.79
UKGE_logi	9.86	20.05	2.67	7.03	0.96	4.09
UComplEx	8.11*	20.17*	1.65*	4.80*	0.53*	3.69*
UKG _S E	7.71	21.34	4.11*	9.79*	0.98	5.98
BEUrRE	7.49	19.88	2.01	6.89	0.94*	3.67*
UBetaE	7.20	18.85*	2.11*	6.93*	0.76*	3.74*
MUKGE_rect	7.05	19.73	1.63	6.67	0.50	3.61
MUKGE_logi	9.99	20.42	2.11	6.12	0.66	4.21
UDConEx_logi	7.70	18.93	1.47	4.67	0.30	2.06
UDConEx_rect	7.04	18.48	1.44	4.66	0.30	1.99

模型	CN15k		NL27k		PPI5k
模型	linear	exp	linear	exp	linear	exp
TransE	0.601	0.591	0.730	0.722	0.710	0.700
DistMult	0.689	0.677	0.911	0.897	0.894	0.880
ComplEx	0.723	0.712	0.921	0.913	0.896	0.881
UKGE_rect	0.773	0.775	0.939	0.942	0.946	0.946
UKGE_logi	0.789	0.788	0.955	0.956	0.970	0.969
UComplEx	0.841	0.850	0.944	0.946	0.986	0.986
UKG _S E	0.780	0.795	0.841*	0.843	0.980	0.979
BEUrRE	0.801	0.803	0.966	0.970	0.948*	0.948*
UBetaE	0.845*	0.851*	0.944*	0.945*	0.987*	0.986*
MUKGE_rect	0.832	0.835	0.877	0.882	0.959	0.958
MUKGE_logi	0.849	0.850	0.945	0.947	0.991	0.990
UDConEx_logi	0.898	0.899	0.970	0.971	0.988	0.987
UDConEx_rect	0.886	0.887	0.959	0.963	0.989	0.988

模型	CN15k		NL27k		PPI5k
模型	linear	exp	linear	exp	linear	exp
TransE	0.601	0.591	0.730	0.722	0.710	0.700
DistMult	0.689	0.677	0.911	0.897	0.894	0.880
ComplEx	0.723	0.712	0.921	0.913	0.896	0.881
UKGE_rect	0.773	0.775	0.939	0.942	0.946	0.946
UKGE_logi	0.789	0.788	0.955	0.956	0.970	0.969
UComplEx	0.841	0.850	0.944	0.946	0.986	0.986
UKG _S E	0.780	0.795	0.841*	0.843	0.980	0.979
BEUrRE	0.801	0.803	0.966	0.970	0.948*	0.948*
UBetaE	0.845*	0.851*	0.944*	0.945*	0.987*	0.986*
MUKGE_rect	0.832	0.835	0.877	0.882	0.959	0.958
MUKGE_logi	0.849	0.850	0.945	0.947	0.991	0.990
UDConEx_logi	0.898	0.899	0.970	0.971	0.988	0.987
UDConEx_rect	0.886	0.887	0.959	0.963	0.989	0.988

数据集	原始不确定KG				逻辑关系	链接预测的不确定KG
数据集	头实体	关系	尾实体	置信度	逻辑关系	头实体	关系	尾实体	置信度
CN15k	paradise	synonym	heaven	0.984 3	对称关系	heaven	synonym	paradise	0.841 5
								eden	0.937 7
								god	0.817 5
	kill	causes	die	0.892 7	非对称关系	die	causes	lose	0.987 6
								fear	0.966 6
								funeral	0.896 2
	letter	part of	alphabet	0.984 3	逆关系	alphabet	form of	letter	0.871 6
								script	0.844 7
								rudiment	0.828 4
NL27k	Toyota	Competes with	General Motors	0.929 2	传递关系	Toyota	Competes with	Chrysler	0.967 4
	General Motors		Chrysler	0.999 9				BMW	0.953 2
	General Motors		BMW	0.991 2				Ford	0.950 4

Multi-relation approximate reasoning model based on uncertain knowledge graph embedding

基于不确定知识图谱嵌入的多关系近似推理模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 6

References 37

Related Articles 15

Recommended Articles

Metrics

[1]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[2]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[3]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[4]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[5]	Wei LI, Xiaorong ZHANG, Peng CHEN, Qing LI, Changqing ZHANG. Crowd counting algorithm with multi-scale fusion based on normal inverse Gamma distribution [J]. Journal of Computer Applications, 2024, 44(7): 2243-2249.
[6]	Yan ZHOU, Yang LI. Rectified cross pseudo supervision method with attention mechanism for stroke lesion segmentation [J]. Journal of Computer Applications, 2024, 44(6): 1942-1948.
[7]	Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance [J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.
[8]	Min SUN, Qian CHENG, Xining DING. CBAM-CGRU-SVM based malware detection method for Android [J]. Journal of Computer Applications, 2024, 44(5): 1539-1545.
[9]	Wenshuo GAO, Xiaoyun CHEN. Point cloud classification network based on node structure [J]. Journal of Computer Applications, 2024, 44(5): 1471-1478.
[10]	Tianhua CHEN, Jiaxuan ZHU, Jie YIN. Bird recognition algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1114-1120.
[11]	Lijun XU, Hui LI, Zuyang LIU, Kansong CHEN, Weixuan MA. 3D-GA-Unet： MRI image segmentation algorithm for glioma based on 3D-Ghost CNN [J]. Journal of Computer Applications, 2024, 44(4): 1294-1302.
[12]	Jie GUO, Jiayu LIN, Zuhong LIANG, Xiaobo LUO, Haitao SUN. Recommendation method based on knowledge‑awareness and cross-level contrastive learning [J]. Journal of Computer Applications, 2024, 44(4): 1121-1127.
[13]	Jie WANG, Hua MENG. Image classification algorithm based on overall topological structure of point cloud [J]. Journal of Computer Applications, 2024, 44(4): 1107-1113.
[14]	Ruifeng HOU, Pengcheng ZHANG, Liyuan ZHANG, Zhiguo GUI, Yi LIU, Haowen ZHANG, Shubin WANG. Iterative denoising network based on total variation regular term expansion [J]. Journal of Computer Applications, 2024, 44(3): 916-921.
[15]	Jingxian ZHOU, Xina LI. UAV detection and recognition based on improved convolutional neural network and radio frequency fingerprint [J]. Journal of Computer Applications, 2024, 44(3): 876-882.