Knowledge graph embedding model based on improved Inception structure

doi:10.11772/j.issn.1001-9081.2021071265

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (4): 1065-1071.DOI: 10.11772/j.issn.1001-9081.2021071265

• The 36 CCF National Conference of Computer Applications (CCF NCCA 2020) • Previous Articles

Knowledge graph embedding model based on improved Inception structure

Xiaopeng YU¹^,², Ruhan HE¹^,², Jin HUANG²^,³(), Junjie ZHANG¹^,², Xinrong HU²^,³

^1.Hubei Provincial Engineering Research Center for Intelligent Textile and Fashion;（Wuhan Textile University），Wuhan Hubei 430200，China
^2.School of Computer Science and Artificial Intelligence，Wuhan Textile University，Wuhan Hubei 430200，China
^3.Engineering Research Center of Hubei Province for Clothing Information （Wuhan Textile University），Wuhan Hubei 430200，China

Received:2021-07-16 Revised:2021-08-20 Accepted:2021-08-25 Online:2021-08-20 Published:2022-04-10
Contact: Jin HUANG
About author:YU Xiaopeng， born in 1994， M. S. candidate. His research interests include knowledge graph， natural language processing.
HE Ruhan， born 1974， Ph. D.， professor. His research interests include machine learning， multimedia retrieval.
ZHANG Junjie， born in 1980， Ph. D.， associate professor. His research interests include recommender system， knowledge graph.
HU Xinrong， born in 1973，Ph. D.， professor. Her research interests include virtual fitting， image processing.
Supported by:
Key project of Science and Technology Research Plan of Hubei Provincial Department of Education(20141603)

基于改进Inception结构的知识图谱嵌入模型

余晓鹏¹^,², 何儒汉¹^,², 黄晋²^,³(), 张俊杰¹^,², 胡新荣²^,³

^1.纺织服装智能化湖北省工程研究中心（武汉纺织大学），武汉 430200
^2.武汉纺织大学计算机与人工智能学院，武汉 430200
^3.湖北省服装信息化工程技术研究中心（武汉纺织大学），武汉 430200

通讯作者: 黄晋
作者简介:余晓鹏（1994—），男，山西大同人，硕士研究生，CCF会员，主要研究方向：知识图谱、自然语言处理
何儒汉（1974—），男，湖北武汉人，教授，博士，CCF会员，主要研究方向：机器学习、多媒体检索
张俊杰（1980—），男，湖北武汉人，副教授，博士，CCF会员，主要研究方向：推荐系统、知识图谱
胡新荣（1973—），女，湖北武汉人，教授，博士，CCF会员，主要研究方向：虚拟试衣、图像处理。
基金资助:
湖北省教育厅科学技术研究计划重点项目(20141603)

Abstract

Abstract:

KGE(Knowledge Graph Embedding） maps entities and relationships into a low-dimensional continuous vector space， uses machine learning methods to implement relational data applications， such as knowledge analysis， reasoning， and completion. Taking ConvE （Convolution Embedding） as a representative， CNN （Convolutional Neural Network） is applied to knowledge graph embedding to capture the interactive information of entities and relationships， but the ability of the standard convolutional to capture feature interaction information is insufficient， and its feature expression ability is low. Aiming at the problem of insufficient feature interaction ability， an improved Inception structure was proposed， based on which a knowledge graph embedding model named InceE was constructed. Firstly， hybrid dilated convolution replaced standard convolution to improve the ability to capture feature interaction information. Secondly， the residual network structure was used to reduce the loss of feature information. The experiments were carried out on the datasets Kinship， FB15k， WN18 to verify the effectiveness of link prediction by InceE. Compared with ArcE and QuatRE models on the Kinship and FB15k datasets， the Hit@1 of InceE increased by 1.6 and 1.5 percentage points； compared with ConvE on the three datasets， the Hit@1 of InceE increased by 6.3， 20.8， and 1.0 percentage points. The experimental results show that InceE has a stronger ability to capture feature interactive information.

Key words: Knowledge Graph Embedding(KGE), feature interaction, Inception, hybrid dilated convolution, residual learning, link prediction

摘要：

知识图谱嵌入（KGE）将实体和关系映射到低维连续向量空间中，以利用机器学习方法实现关系数据的应用，如知识分析、推理、补全等。以ConvE为代表将卷积神经网络（CNN）应用于知识图谱嵌入中，以捕捉实体和关系的交互信息，但其标准卷积捕捉特征交互信息能力不足，特征表达能力低下。针对特征交互能力不足问题，提出了一种改进的Inception结构，在此基础上构建一个知识图谱嵌入模型InceE。首先，该结构使用混合空洞卷积替代标准卷积，以提高特征交互信息捕捉能力；其次，使用残差网络结构，以减少特征信息丢失。实验使用基准数据集Kinship、FB15k、WN18验证InceE链接预测有效性。在Kinship、FB15k数据集上，相较于ArcE和QuatRE模型，InceE的Hit@1分别提升了1.6和1.5个百分点；在三个数据集上，与ConvE对比，InceE的Hit@1分别提升了6.3、20.8和1.0个百分点。实验结果表明InceE具有更强的特征交互信息捕捉能力。

关键词: 知识图谱嵌入, 特征交互, Inception, 混合空洞卷积, 残差学习, 链接预测

CLC Number:

TP183

Xiaopeng YU, Ruhan HE, Jin HUANG, Junjie ZHANG, Xinrong HU. Knowledge graph embedding model based on improved Inception structure[J]. Journal of Computer Applications, 2022, 42(4): 1065-1071.

余晓鹏, 何儒汉, 黄晋, 张俊杰, 胡新荣. 基于改进Inception结构的知识图谱嵌入模型[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1065-1071.

Figures/Tables 10

Tab. 1 Knowledge graph embedding model and scoring function

模型	评分函数	参数	模型	评分函数	参数
TransE	$ξ (h, r, t) = - \| \| h + r - t \| \| 1 / 2$	$h, r, t ∈ R d$	DisMult	$ξ (h, r, t) = h Τ d i a g (r) t$	$h, r, t ∈ R k$
TransR	$ξ (h, r, t) = - \| \| M r h + r - M r t \| \| 2$	$h, t ∈ R d, r ∈ R k,$ $M r ∈ R k × d$	CompIEx	$ξ (h, r, t) = R c (h Τ d i a g (r) t ¯)$	$h, r, t ∈ C k$
ManifoldE	$ξ (h, r, t) = - (\| \| h + r - t \| \| 22 - θ r 2) 2$	$h, r, t ∈ R k$	ConvE	$ξ (h, r, t) = f (v e c (f (\| r ¯, h ¯ \| * ω) W) t$	$h, r, t ∈ R k$
RotatE	$ξ (h, r, t) = - \| \| h ∘ r - t \| \| 2$	$h, r, t ∈ C k, \| r i \| = 1$	InceE	$ξ (h, r, t) = f ((e 2 W + b) t Τ)$	$h, r, t ∈ R k$
Rescal	$ξ (h, r, t) = h Τ M r t$	$h, t ∈ R k, M r ∈ R k × k$

Tab. 1 Knowledge graph embedding model and scoring function

模型	评分函数	参数	模型	评分函数	参数
TransE	$ξ (h, r, t) = - \| \| h + r - t \| \| 1 / 2$	$h, r, t ∈ R d$	DisMult	$ξ (h, r, t) = h Τ d i a g (r) t$	$h, r, t ∈ R k$
TransR	$ξ (h, r, t) = - \| \| M r h + r - M r t \| \| 2$	$h, t ∈ R d, r ∈ R k,$ $M r ∈ R k × d$	CompIEx	$ξ (h, r, t) = R c (h Τ d i a g (r) t ¯)$	$h, r, t ∈ C k$
ManifoldE	$ξ (h, r, t) = - (\| \| h + r - t \| \| 22 - θ r 2) 2$	$h, r, t ∈ R k$	ConvE	$ξ (h, r, t) = f (v e c (f (\| r ¯, h ¯ \| * ω) W) t$	$h, r, t ∈ R k$
RotatE	$ξ (h, r, t) = - \| \| h ∘ r - t \| \| 2$	$h, r, t ∈ C k, \| r i \| = 1$	InceE	$ξ (h, r, t) = f ((e 2 W + b) t Τ)$	$h, r, t ∈ R k$
Rescal	$ξ (h, r, t) = h Τ M r t$	$h, t ∈ R k, M r ∈ R k × k$

Fig. 1 InceE model flow

Fig. 2 Structure of InceE network model

Tab. 2 Dataset statistics

数据集	#Relation	#Entity	#Triplet
数据集	#Relation	#Entity	Train	Valid	Test
FB15k	1 345	14 951	483 142	50 000	59 071
WN18	18	40 943	141 442	5 000	5 000
Kinship	25	104	8 544	1 068	1 074

Tab. 3 Experimental results of different models on Kinship dataset

模型	MRR	Hit@1/%	Hit@3/%	Hit@10/%
CompIEx	0.823	73.3	89.9	97.1
ConvE	0.833	73.8	91.7	98.1
ConvKB	0.614	43.6	75.5	95.3
R-GCN	0.109	3.0	8.8	23.9
SimplE	0.752	62.6	85.4	97.2
RotatE	0.843	76.0	91.9	97.8
HAKE	0.852	76.9	92.8	98.0
InteractE	0.777	66.4	87.0	95.9
CompGCN	0.778	66.7	86.8	96.7
CoKE	0.793	69.3	87.8	95.4
ArcE	0.864	78.5	93.9	98.7
InceE	0.873	80.1	93.7	98.3

Tab. 4 Experimental comparison results of different models on FB15k dataset

模型	FB15k
模型	MRR	Hit@1/%	Hit@3/%	Hit@10/%
TransE	0.463	29.7	57.8	74.9
HOIE	0.524	40.2	61.3	73.9
SimplEx	0.727	66.0	77.3	83.8
ConvE	0.657	55.8	72.3	83.1
R-GCN	0.696	60.1	76.0	84.2
RotatE	0.797	74.6	83.0	88.4
RSNs	0.780	72.2	―	87.3
QuatRE	0.808	75.1	85.1	89.6
InceE	0.815	76.6	85.0	89.6

Tab. 5 Experimental comparison results of different models on WN18 dataset

模型	WN18
模型	MRR	Hit@1/%	Hit@3/%	Hit@10/%
TransE	0.495	11.3	88.8	94.3
HOIE	0.938	93.0	94.5	94.9
SimplEx	0.941	93.6	94.5	94.7
ConvE	0.942	93.5	94.7	95.5
R-GCN	0.696	60.1	76.0	84.2
RotatE	0.949	94.4	95.2	95.9
RSNs	0.940	92.2	―	95.3
QuatRE	0.939	92.3	95.3	96.3
InceE	0.949	94.5	95.1	95.5

Tab. 6 Experimental results of different models based on CNN on Kinship dataset

模型	MRR	Hit@1/%	Hit@3/%	Hit@10/%
ConvE	0.833	73.8	91.7	98.1
InteractE	0.777	66.4	87.0	95.9
ArcE	0.864	78.5	93.9	98.7
InceE	0.873	80.1	93.7	98.3

Tab. 7 Experimental results of InceE model whether to add residual learning on Kinship dataset

模型	MRR	Hit@1/%	Hit@3/%	Hit@3/%
InceE-Residual	0.860	77.9	93.6	98.3
InceE	0.873	80.1	93.7	98.3

Fig. 3 Comparison results of standard convolution and hybrid dilated convolution

References 31

1	BOLLACKER K， EVANS C， PARITOSH P， et al. Freebase： a collaboratively created graph database for structuring human knowledge［C］//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York：ACM， 2008： 1247-1250. 10.1145/1376616.1376746
2	MILLER G A. WordNet： a lexical database for English［J］. Communications of the ACM，1995， 38（11）： 39-41. 10.1145/219717.219748
3	SUCHANEK F M， KASNECI G， WEIKUM G. YAGO： a core of semantic knowledge［C］// Proceedings of the 16th International Conference on World Wide Web. New York： ACM， 2007： 697-706. 10.1145/1242572.1242667
4	ZHANG F Z， YUAN N J， LIAN D， et al. Collaborative knowledge base embedding for recommender systems［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 353-362. 10.1145/2939672.2939673
5	刘知远，孙茂松，林衍凯，等.知识表示学习研究进展［J］.计算机研究与发展，2016，53（2）：247-261. 10.7544/issn1000-1239.2016.20160020
	LIU Z Y， SUN M S， LIN Y K， et al. Knowledge representation learning： a review［J］.Journal of Computer Research and Development，2016，53（2）：247-261. 10.7544/issn1000-1239.2016.20160020
6	BORDES A， USUNIER N， GARCIA-DURAN A， et al. Translating embeddings for modeling multi-relational data［C］// Proceedings of the 26th International Conference on Neural Information Processing Systems. New York： ACM， 2013： 2787-2795. 10.1007/978-3-662-44848-9_28
7	NICKEL M， TRESP V， KRIEGEL H P. A three-way model for collective learning on multi-relational data［C］// Proceedings of the 28th International Conference on Machine Learning. New York： ACM， 2011： 809-816. 10.1145/2187836.2187874
8	DETTMERS T， MINERVINI P， STENETORP P， et al. Convolutional 2D knowledge graph embeddings［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2018： 1811-1818.
9	SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 1-9. 10.1109/cvpr.2015.7298594
10	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
11	WANG Z， ZHANG J W， FENG J L， et al. Knowledge graph embedding by translating on hyperplanes［C］// Proceedings of the 28th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2014： 1112-1119. 10.1609/aaai.v33i01.33017152
12	LIN Y， LIU Z， SUN M，et al. Learning entity and relation embeddings for knowledge graph completion［C］// Proceedings of the 29th AAAI Conference on Artificial Intelligence. Palo Alto：AAAI Press，2015：2181-2187.
13	XIAO H， HUANG M L， ZHU X Y. From one point to a manifold： Knowledge graph embedding for precise link prediction［C］// Proceedings of the 25th International Joint Conference on Artificial Intelligence. Palo Alto：AAAI Press， 2016： 1315-1321.
14	SUN Z Q， DENG Z H， NIE J Y， et al. RotatE： Knowledge graph embedding by relational rotation in complex space［C］// Proceedings of the 2019 International Conference on Learning Representations. New Orleans： ICLR， 2019： 1-18.
15	KAZEMI S M， POOLE D. Simple embedding for link prediction in knowledge graphs［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook： Curran Associates， Inc.， 2018： 4289-4300.
16	YANG B S， YIH W T， HE X D， et al. Embedding entities and relations for learning and inference in knowledge bases［EB/OL］. ［2015-08-29］. .
17	TROUILLON T， WELBL J， RIEDEL S， et al. Complex embeddings for simple link prediction［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： PMLR.org， 2016：2071-2080.
18	LIU H X， WU Y X， YANG Y M. Analogical inference for multi-relational embeddings［C］// Proceedings of the 34th International Conference on Machine Learning. New York： PMLR.org， 2017： 2168-2178.
19	DONG X， GABRILOVICH E， HEITZ G， et al. Knowledge vault： a web-scale approach to probabilistic knowledge fusion［C］// Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2014： 601-610. 10.1145/2623330.2623623
20	SOCHER R， CHEN D Q， MANNING C D， et al. Reasoning with neural tensor networks for knowledge base completion［C］// Proceedings of the 26th Neural Information Processing Systems. Red Hook： Curran Associates， Inc.， 2013： 926-934.
21	NGUYEN D Q， NGUYEN T， NGUYEN D Q， et al. A novel embedding model for knowledge base completion based on convolutional neural network［C］// Proceedings of the 2018 North American Chapter of the Association for Computational Linguistics. Stroudsburg， PA： Association for Computational Linguistics， 2018： 327-333. 10.18653/v1/n18-2053
22	VASHISHTH S， SANYAL S， NITIN V， et al. InteractE： Improving convolution-based knowledge graph embeddings by increasing feature interactions［C］// Proceedings of the 2020 AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 3009-3016. 10.1609/aaai.v34i03.5694
23	SCHLICHTKRULL M， KIPF T N， BLOEM P， et al. Modeling relational data with graph convolutional networks［C］// Proceedings of the 2018 European Semantic Web Conference. Cham： Springer， 2018： 593-607. 10.1007/978-3-319-93417-4_38
24	LIN X V， SOCHER R， XIONG C M. Multi-hop knowledge graph reasoning with reward shaping［C］// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg， PA： Association for Computational Linguistics， 2018：3243-3253. 10.18653/v1/d18-1362
25	ZHANG Z Q， CAI J Y， ZHANG Y D， et al. Learning hierarchy-aware knowledge graph embeddings for link prediction［C］// Proceedings of the 2020 AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 3065-3072. 10.1609/aaai.v34i03.5701
26	VASHISHTH S， SANYAL S， NITIN V， et al. Composition-based multi-relational graph convolutional networks［C］// Proceedings of the Eighth International Conference on Learning Representations， New Orleans： ICLR， 2020： 1-14. 10.1609/aaai.v34i03.5694
27	WANG Q， HUANG P P， WANG H F， et al. CoKE： Contextualized knowledge graph embedding ［EB/OL］. ［2020-04-04］. .
28	REN F L， LI J C， ZHANG H H， et al. Knowledge graph embedding with atrous convolution and residual learning［C］// Proceedings of the 28th International Conference on Computational Linguistics. Barcelona： International Committee on Computational Linguistics， 2020： 1532-1543. 10.18653/v1/2020.coling-main.134
29	NICKEL M， ROSASCO L， POGGIO T. Holographic embeddings of knowledge graphs［C］// Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2016：1955-1961.
30	GUO L B， SUN Z Q， HU W. Learning to exploit long-term relational dependencies in knowledge graphs［C］// Proceedings of the 36th International Conference on Machine Learning. Birmingham： PMLR， 2019： 2505-2514. 10.1162/dint_a_00016
31	NGUYEN D Q， VU T， NGUYEN T D， et al. QuatRE： relation-aware quaternions for knowledge graph embeddings［EB/OL］.［2020-09-26］. . 10.1109/tpami.2022.3147798/mm1

[1]	Yimin CAO, Lei CAI, Jingyang GAO. Gene data generation method based on generative adversarial network [J]. Journal of Computer Applications, 2022, 42(3): 783-790.
[2]	Renzhi PAN, Fulan QIAN, Shu ZHAO, Yanping ZHANG. Recommendation model for user attribute preference modeling based on convolutional neural network interaction [J]. Journal of Computer Applications, 2022, 42(2): 404-411.
[3]	Hengxin LI, Kan CHANG, Yufei TAN, Mingyang LING, Tuanfa QIN. Color image demosaicking network based on inter-channel correlation and enhanced information distillation [J]. Journal of Computer Applications, 2022, 42(1): 245-251.
[4]	CAI Biao, LI Ruicen, WU Yuanyuan. Impact and enhancement of similarity features on link prediction [J]. Journal of Computer Applications, 2021, 41(9): 2569-2577.
[5]	BIAN Lingzhi, WANG Zhijie. Credit scoring model based on enhanced multi-dimensional and multi-grained cascade forest [J]. Journal of Computer Applications, 2021, 41(9): 2539-2544.
[6]	ZHANG Yuanjun, ZHANG Xihuang. Dynamic network representation learning model based on graph convolutional network and long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1857-1864.
[7]	NIU Kangli, CHEN Yuzhang, SHEN Junfeng, ZENG Zhangfan, PAN Yongcai, WANG Yichong. Dual-channel night vision image restoration method based on deep learning [J]. Journal of Computer Applications, 2021, 41(6): 1775-1784.
[8]	HUANG Li, LU Long. Segmentation of ischemic stroke lesion based on long-distance dependency encoding and deep residual U-Net [J]. Journal of Computer Applications, 2021, 41(6): 1820-1827.
[9]	LIANG Min, WANG Haorong, ZHANG Yao, LI Jie. Image super-resolution reconstruction method based on accelerated residual network [J]. Journal of Computer Applications, 2021, 41(5): 1438-1444.
[10]	Huibo LI, Yunxiao ZHAO, Liang BAI. Dynamic graph representation learning method based on deep neural network and gated recurrent unit [J]. Journal of Computer Applications, 2021, 41(12): 3432-3437.
[11]	Haiyong WANG, Kaixin ZHANG, Weizheng GUAN. Single image super-resolution reconstruction method based on dense Inception [J]. Journal of Computer Applications, 2021, 41(12): 3666-3671.
[12]	Jianfang CAO, Minmin YAN, Yiming JIA, Xiaodong TIAN. Application of Inception-v3 model integrated with transfer learning in dynasty identification of ancient murals [J]. Journal of Computer Applications, 2021, 41(11): 3219-3227.
[13]	CHEN Saijian, ZHU Yuanping. Joint super-resolution and deblurring method based on generative adversarial network for text images [J]. Journal of Computer Applications, 2020, 40(3): 859-864.
[14]	Pengfei QI, Lihua ZHOU, Guowang DU, Hao HUANG, Tong HUANG. Clustering-based hyperlink prediction [J]. Journal of Computer Applications, 2020, 40(2): 434-440.
[15]	LIU Yuyang, LI Longjie, SHAN Na, CHEN Xiaoyun. Link prediction method fusing clustering coefficients [J]. Journal of Computer Applications, 2020, 40(1): 28-35.

Knowledge graph embedding model based on improved Inception structure

基于改进Inception结构的知识图谱嵌入模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 31

Related Articles 15

Recommended Articles

Metrics