Knowledge graph attention network fusing collaborative filtering information

doi:10.11772/j.issn.1001-9081.2021071269

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (4): 1087-1092.DOI: 10.11772/j.issn.1001-9081.2021071269

Special Issue: CCF第36届中国计算机应用大会 (CCF NCCA 2021)

• The 36 CCF National Conference of Computer Applications (CCF NCCA 2020) • Previous Articles Next Articles

Knowledge graph attention network fusing collaborative filtering information

Junhua GU¹^,², Rui WANG¹, Ningning LI¹, Suqi ZHANG³()

^1.School of Artificial Intelligence and Data Science，Hebei University of Technology，Tianjin 300401，China
^2.Hebei Province Key Laboratory of Big Data Computing （Hebei University of Technology），Tianjin 300401，China
^3.School of Information Engineering，Tianjin University of Commerce，Tianjin 300134，China

Received:2021-07-16 Revised:2021-08-17 Accepted:2021-08-23 Online:2021-08-17 Published:2022-04-10
Contact: Suqi ZHANG
About author:GU Junhua， born in 1966， Ph. D.， professor. His research interests include intelligent information processing， data mining.
WANG Rui， born in 1995， M. S. candidate. His research interests include recommendation system.
LI Ningning， born in 1994， M. S. candidate. Her research interests include recommendation system.
Supported by:
National Natural Science Foundation of China(61802282);Technology Innovation Guidance Special Project of Tianjin Science and Technology Plan(20YDTPJC00670)

融合协同过滤信息的知识图注意力网络

顾军华¹^,², 王锐¹, 李宁宁¹, 张素琪³()

^1.河北工业大学人工智能与数据科学学院，天津 300401
^2.河北省大数据计算重点实验室（河北工业大学），天津 300401
^3.天津商业大学信息工程学院，天津 300134

通讯作者: 张素琪
作者简介:顾军华（1966—），男，天津人，教授，博士，CCF会员，主要研究方向：智能信息处理、数据挖掘
王锐（1995—），男，江苏泰州人，硕士研究生，主要研究方向：推荐系统
李宁宁（1994—），女，河南周口人，硕士研究生，主要研究方向：推荐系统
基金资助:
国家自然科学基金资助项目(61802282);天津市科技计划项目技术创新引导专项(20YDTPJC00670)

Abstract

Abstract:

Since Knowledge Graph（KG） can alleviate the problems of data sparsity and cold start in collaborative filtering algorithm， it has been widely studied and applied in the recommendation field. Many existing recommendation models based on KG confuse the collaborative filtering information in user-item bipartite graph and the association information between entities in KG， resulting in the learned user vector and item vector cannot accurately express the characteristics of users and items， and even introducing wrong information to interfere with recommendation. Regarding the issues above， a model called KG Attention Network fusing Collaborative Filtering information （KGANCF） was proposed. Firstly， the collaborative filtering information of users and items was dug out by the collaborative filtering layer of the network from the user-item bipartite graph， avoiding the interference of the entity information of KG. Then， the graph attention mechanism was applied in the KG attention embedding layer， the attribute information closely related to users and items was extracted from KG. Finally， the collaborative filtering information and the attribute information in KG were merged at the prediction layer to obtain the final vector representations of users and items， and then the scores of users to items were predicted. The experiments were carried out on MovieLens-20M and Last.FM datasets. Compared with the results of Collaborative Knowledge-aware Attentive Network （CKAN）， on Movielens-20M， F1-score of KGANCF improves by 1.1 percentage points while Area Under Curve （AUC） improves by 0.6 percentage points； on Last.FM， F1-score improves by 3.3 percentage points and AUC improves by 8.5 percentage points. Experimental results show that KGANCF can effectively improve the accuracy of recommendation results， and is significantly better than CKE （Collaborative Knowledge base Embedding），KGCN （Knowledge Graph Convolutional Network），KGAT （Knowledge Graph Attention Network） and CKAN models on datasets with sparse KG.

Key words: recommender system, Knowledge Graph (KG), collaborative filtering, attention mechanism, graph neural network

摘要：

知识图谱（KG）能够缓解协同过滤算法存在的数据稀疏和冷启动问题，在推荐领域被广泛地研究和应用。现有的很多基于KG的推荐模型混淆了用户物品二部图中的协同过滤信息和KG中实体间的关联信息，导致学习到的用户向量和物品向量无法准确表达其特征，甚至引入与用户、物品无关的信息从而干扰推荐。针对上述问题提出一种融合协同信息的知识图注意力网络（KGANCF）。首先，为了避免KG实体信息的干扰，网络的协同过滤层从用户物品二部图中挖掘出用户和物品的协同过滤信息；然后，在知识图注意力嵌入层中应用图注意力机制，从KG中继续提取与用户和物品密切相关的属性信息；最后，在预测层将用户物品的协同过滤信息和KG中的属性信息融合，得到用户和物品最终向量表示，进而预测用户对物品的评分。在MovieLens-20M和Last.FM数据集上进行了实验，与协同知识感知注意力网络（CKAN）相比，KGANCF在MovieLens-20M数据集上的F1分数提升了1.1个百分点，曲线下面积（AUC）提升了0.6个百分点；而在KG相对稀疏的Last.FM数据集上，模型的F1分数提升了3.3个百分点，AUC提升了8.5个百分点。实验结果表明，KGANCF能够有效提高推荐结果的准确度，在KG稀疏的数据集上显著优于协同知识嵌入（CKE）、知识图谱卷积网络（KGCN）、知识图注意网络（KGAT）和CKAN模型。

关键词: 推荐系统, 知识图谱, 协同过滤, 注意力机制, 图神经网络

CLC Number:

TP181

Junhua GU, Rui WANG, Ningning LI, Suqi ZHANG. Knowledge graph attention network fusing collaborative filtering information[J]. Journal of Computer Applications, 2022, 42(4): 1087-1092.

顾军华, 王锐, 李宁宁, 张素琪. 融合协同过滤信息的知识图注意力网络[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1087-1092.

Figures/Tables 8

References 21

1	黄立威，江碧涛，吕守业，等.基于深度学习的推荐系统研究综述［J］.计算机学报，2018， 41（7）：1619-1647. 10.11897/SP.J.1016.2018.01619
	HUANG L W， JIANG B T， LV S Y， et al. Survey on deep learning based recommender systems［J］. Chinese Journal of Computers， 2018， 41（7）： 1619-1647. 10.11897/SP.J.1016.2018.01619
2	WANG X， WANG D， XU C， et al. Explainable reasoning over knowledge graphs for recommendation［C］// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2019： 5329-5336. 10.1609/aaai.v33i01.33015329
3	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780. 10.1162/neco.1997.9.8.1735
4	WANG H， ZHAO M， XIE X， et al. Knowledge graph convolutional networks for recommender systems［C］// Proceedings of the 2019 World Wide Web Conference. New York： ACM， 2019： 3307-3313. 10.1145/3308558.3313417
5	WANG X， HE X， CAO Y， et al. KGAT： Knowledge graph attention network for recommendation［C］// Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York： ACM， 2019： 950-958. 10.1145/3292500.3330989
6	WANG Z， LIN G， TAN H， et al. CKAN： collaborative knowledge-aware attentive network for recommender systems［C］// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2020： 219-228. 10.1145/3397271.3401141
7	HE X， DENG K， WANG X， et al. LightGCN： simplifying and powering graph convolution network for recommendation［C］// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2020： 639-648. 10.1145/3397271.3401063
8	HE X， LIAO L， ZHANG H， et al. Neural collaborative filtering［C］// Proceedings of the 26th International Conference on World Wide Web. New York： ACM， 2017： 173-182. 10.1145/3038912.3052569
9	CHENG H T， KOC L， HARMSEN J， et al. Wide & deep learning for recommender systems［C］// Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. New York： ACM， 2016： 7-10. 10.1145/2988450.2988454
10	WANG X， HE X， WANG M， et al. Neural graph collaborative filtering［C］// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2019： 165-174. 10.1145/3331184.3331267
11	CAO Y， WANG X， HE X， et al. Unifying knowledge graph learning and recommendation： towards a better understanding of user preferences［C］// Proceedings of the 2019 World Wide Web Conference. New York： ACM， 2019： 151-161. 10.1145/3308558.3313705
12	WU L， YANG Y， ZHANG K， et al. Joint item recommendation and attribute inference： an adaptive graph convolutional network approach［C］// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2020： 679-688. 10.1145/3397271.3401144
13	WANG H， ZHANG F， WANG J， et al. RippleNet： Propagating user preferences on the knowledge graph for recommender systems［C］// Proceedings of the 27th ACM International Conference on Information and Knowledge Management. New York： ACM， 2018： 417-426. 10.1145/3269206.3271739
14	SUN R， CAO X， ZHAO Y， et al. Multi-modal knowledge graphs for recommender systems［C］// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York： ACM， 2020： 1405-1414. 10.1145/3340531.3411947
15	XIA L， HUANG C， XU Y， et al. Knowledge-enhanced hierarchical graph transformer network for multi-behavior recommendation［C］// Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2021： 4486-4493.
16	WANG X， HUANG T， WANG D， et al. Learning intents behind interactions with knowledge graph for recommendation［C］// Proceedings of the Web Conference 2021. New York： ACM， 2021： 878-887. 10.1145/3442381.3450133
17	HAHNLOSER R H R， SARPESHKAR R， MAHOWALD M A， et al. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit［J］. Nature， 2000， 405（6789）： 947-951. 10.1038/35016072
18	HAN J， MORAGA C. The influence of the sigmoid function parameters on the speed of backpropagation learning［C］// Proceedings of the 1995 International Workshop on Artificial Neural Networks from Natural to Artificial Neural Computation. Berlin： Springer， 1995： 195-201. 10.1007/3-540-59497-3_175
19	GLOROT X， BENGIO Y. Understanding the difficulty of training deep feedforward neural networks［J］. Journal of Machine Learning Research， 2010， 9：249-256.
20	ZHANG F， YUAN N J， LIAN D， et al. Collaborative knowledge base embedding for recommender systems［C］// Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2016： 353-362. 10.1145/2939672.2939673
21	LIN Y， LIU Z， SUN M， et al. Learning entity and relation embeddings for knowledge graph completion［C］// Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence. New York： ACM， 2015： 2181-2187.

数据集	d	S_u	S_v	λ	Lr	batch
Last.FM	64	4	32	1×10^-5	1×10^-5	1 024
MovieLens-20M	64	8	32	1×10^-5	1×10^-5	1 024

数据集	d	S_u	S_v	λ	Lr	batch
Last.FM	64	4	32	1×10^-5	1×10^-5	1 024
MovieLens-20M	64	8	32	1×10^-5	1×10^-5	1 024

模型	Last.FM		MovieLens-20M
模型	AUC	F1	AUC	F1
CKE	0.747	0.674	0.927	0.874
KGCN	0.802	0.708	0.977	0.930
KGAT	0.829	0.742	0.975	0.929
CKAN	0.842	0.769	0.976	0.929
KGANCF	0.927	0.802	0.982	0.940

模型	Last.FM		MovieLens-20M
模型	AUC	F1	AUC	F1
CKE	0.747	0.674	0.927	0.874
KGCN	0.802	0.708	0.977	0.930
KGAT	0.829	0.742	0.975	0.929
CKAN	0.842	0.769	0.976	0.929
KGANCF	0.927	0.802	0.982	0.940

协同过滤层数	AUC
协同过滤层数	Last.FM	MovieLens-20M
1	0.891	0.977
2	0.918	0.981
3	0.927	0.982
4	0.922	0.982

Knowledge graph attention network fusing collaborative filtering information

融合协同过滤信息的知识图注意力网络

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 21

Related Articles 15

Recommended Articles

Metrics

知识图注意力嵌入层数	AUC
知识图注意力嵌入层数	Last.FM	MovieLens-20M
1	0.927	0.982
2	0.923	0.982
3	0.918	0.980

[1]	Yu DU, Yan ZHU. Constructing pre-trained dynamic graph neural network to predict disappearance of academic cooperation behavior [J]. Journal of Computer Applications, 2024, 44(9): 2726-2731.
[2]	Tingjie TANG, Jiajin HUANG, Jin QIN. Session-based recommendation with graph auxiliary learning [J]. Journal of Computer Applications, 2024, 44(9): 2711-2718.
[3]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[4]	Xingyao YANG, Yu CHEN, Jiong YU, Zulian ZHANG, Jiaying CHEN, Dongxiao WANG. Recommendation model combining self-features and contrastive learning [J]. Journal of Computer Applications, 2024, 44(9): 2704-2710.
[5]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[6]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[7]	Hang YANG, Wanggen LI, Gensheng ZHANG, Zhige WANG, Xin KAI. Multi-layer information interactive fusion algorithm based on graph neural network for session-based recommendation [J]. Journal of Computer Applications, 2024, 44(9): 2719-2725.
[8]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[9]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[10]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[11]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[12]	Fan YANG, Yao ZOU, Mingzhi ZHU, Zhenwei MA, Dawei CHENG, Changjun JIANG. Credit card fraud detection model based on graph attention Transformation neural network [J]. Journal of Computer Applications, 2024, 44(8): 2634-2642.
[13]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[14]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[15]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.