Sequential recommendation model based on multi-level graph contrastive learning

doi:10.11772/j.issn.1001-9081.2024010126

Abstract

Abstract:

Aiming at the problem that current sequential recommendation models based on contrastive learning only consider representation learning at the level of items and items or sequences and sequences， and are incapable of capturing fined and distinguishable user representations and item representations， a Multi-Level Graph Contrastive Learning based Sequential Recommendation （MLGCL-SR） model was proposed. Firstly， the item transition graph was constructed on the basis of sequential order of user-clicked items and performed to embedded representation. Secondly， user representation learning was conducted on the embedded item transition graph using an optimized Bidirectional Gated Graph Neural Network （BI-GGNN）. Finally， parameter updates for primary recommendation prediction task were performed through a cross-entropy loss function， and the multi-level contrastive learning tasks were used to assist in parameter updates of recommendation prediction tasks at embedding layer， node layer， and sequential layer； specifically， on the embedding layer and node layer for better item representations in recommendation prediction tasks， while on the sequential layer for better user representations in recommendation prediction tasks. Experimental results on three benchmark datasets， Sports， Beauty， and Toys， demonstrated that compared to the optimal baseline model MCLRec （Meta-optimized Contrastive Learning for sequential Recommendation）， MLGCL-SR model has significant improvements on Hit Ratio （HR） and Normalized Discounted Cumulative Gain （NDCG） metrics， and increases on NDCG@10， the metric better reflecting recommendation effectiveness by 14.2%， 19.1%， and 23.1% respectively， validating the effectiveness of the model.

Key words: recommendation system, sequential recommendation, contrastive learning, Bidirectional Gated Graph Neural Network (BI-GGNN)

摘要：

针对现有的基于对比学习的序列推荐模型只考虑了项目和项目或者序列和序列这一个层级的表示学习，而没有办法学习到精细的、易于区分的用户与项目表示的问题，提出一种基于多层次图对比学习的序列推荐（MLGCL-SR）模型。首先，根据用户点击项目的顺序构建项目转移图，并对它进行嵌入表示；其次，利用优化后的双向门控图神经网络（BI-GGNN）在嵌入的项目转移图上进行用户的表示学习；最后，对主要推荐预测任务通过交叉熵损失函数进行参数更新，并使用多层次对比学习任务在嵌入层、节点层、序列层三个层次辅助推荐预测任务进行参数更新，其中在嵌入层和节点层辅助推荐预测任务进行更好的项目表示，而在序列层辅助推荐预测任务进行更好的用户表示。在3个基准数据集Sports、Beauty、Toys上的实验结果表明，相较于对比的最优模型MCLRec （Meta-optimized Contrastive Learning for sequential Recommendation），MLGCL-SR模型在命中率（HR）和归一化折损累计增益（NDCG）指标上有了显著的提升，在能较好反映推荐效果的NDCG@10指标上分别提升了14.2%、19.1%和23.1%，验证了模型的有效性。

关键词: 推荐系统, 序列推荐, 对比学习, 双向门控图神经网络

CLC Number:

TP391.3

Xiaosheng YU, Zhixin WANG. Sequential recommendation model based on multi-level graph contrastive learning[J]. Journal of Computer Applications, 2025, 45(1): 106-114.

余肖生, 王智鑫. 基于多层次图对比学习的序列推荐模型[J]. 《计算机应用》唯一官方网站, 2025, 45(1): 106-114.

Figures/Tables 13

References 40

1	WANG S， HU L， WANG Y， et al. Sequential recommender systems： challenges， progress and prospects ［C］// Proceedings of the 28th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2019： 6332-6338.
2	HERLOCKER J L， KONSTAN J A， BORCHERS A， et al. An algorithmic framework for performing collaborative filtering ［C］// Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2017： 230-237.
3	HE R， FANG C， WANG Z， et al. Vista： a visually， socially， and temporally-aware model for artistic recommendation ［C］// Proceedings of the 10th ACM Conference on Recommender Systems. New York： ACM， 2016： 309-316.
4	杨航，李汪根，张根生，等. 基于图神经网络的多层信息交互融合算法用于会话推荐［J］. 计算机应用， 2024， 44（9）：2719-2725.
	YANG H， LI W G， ZHANG G S， et al. Multi-layer information interactive fusion algorithm based on graph neural network for session-based recommendation［J］. Journal of Computer Applications， 2024， 44（9）：2719-2725.
5	WU S， TANG Y， ZHU Y， et al. Session-based recommendation with graph neural networks ［C］// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2019： 346-353.
6	CHEN Q， ZHAO H， LI W， et al. Behavior sequence Transformer for e-commerce recommendation in Alibaba ［C］// Proceedings of the 1st International Workshop on Deep Learning Practice for High-Dimensional Sparse Data. New York： ACM， 2019： No.12.
7	LIN W， ZHAO X， WANG Y， et al. AutoDenoise： automatic data instance denoising for recommendations ［C］// Proceedings of the ACM Web Conference 2023. New York： ACM， 2023： 1003-1011.
8	XIE X， SUN F， LIU Z， et al. Contrastive learning for sequential recommendation ［C］// Proceedings of the IEEE 38th International Conference on Data Engineering. Piscataway： IEEE， 2022： 1259-1273.
9	WU J， WANG X， FENG F， et al. Self-supervised graph learning for recommendation ［C］// Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2021： 726-735.
10	SHAO W， CHEN X， ZHAO J， et al. Sequential recommendation with adaptive preference disentanglement［EB/OL］. ［2023-12-06］..
11	RAC M， KOMPAN M， BIELIKOVA M. Session-based k-NNs with semantic suggestions for next-item prediction［EB/OL］. ［2023-12-16］..
12	YUAN F， HE X， JIANG H， et al. Future data helps training： modeling future contexts for session-based recommendation ［C］// Proceedings of the Web Conference 2020. New York： ACM， 2020： 303-313.
13	VASSØY B， RUOCCO M， DE SOUZA DA SILVA E， et al. Time is of the essence： a joint hierarchical RNN and point process model for time and item predictions ［C］// Proceedings of the 12th ACM International Conference on Web Search and Data Mining. New York： ACM， 2019： 591-599.
14	CHEN X， XU H， ZHANG Y， et al. Sequential recommendation with user memory networks ［C］// Proceedings of the 1th ACM International Conference on Web Search and Data Mining. New York： ACM， 2018： 108-116.
15	WU J， OU Z， SONG M. Session-based recommendation with context-aware attention network ［C］// Proceedings of the 7th International Conference on Information Technology： IoT and Smart City. New York： ACM， 2019： 141-146.
16	DEVLIN J， CHANG M W， LEE K， et al. BERT： pre-training of deep bidirectional transformers for language understanding ［C］// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies， Volume 1 （Long and Short Papers）. Stroudsburg： ACL， 2019： 4171-4186.
17	BUDZIANOWSKI P， VULIĆ I. Hello， it's GPT-2 - how can I help you？ towards the use of pretrained language models for task-oriented dialogue systems ［C］// Proceedings of the 3rd Workshop on Neural Generation and Translation. Stroudsburg： ACL， 2019： 15-22.
18	QIU X， SUN T， XU Y， et al. Pre-trained models for natural language processing： a survey［J］. SCIENCE CHINA Technological Sciences， 2020， 63（10）： 1872-1897.
19	JAISWAL A， BABU A R， ZADEH M Z， et al. A survey on contrastive self-supervised learning［J］. Technologies， 2021， 9（1）： No.2.
20	LIU Z， CHEN Y， LI J， et al. Contrastive self-supervised sequential recommendation with robust augmentation［EB/OL］. ［2024-08-14］..
21	QIU R， HUANG Z， YIN H， et al. Contrastive learning for representation degeneration problem in sequential recommendation ［C］// Proceedings of the 15th ACM International Conference on Web Search and Data Mining. New York： ACM， 2022： 813-823.
22	QIN X， YUAN H， ZHAO P， et al. Meta-optimized contrastive learning for sequential recommendation ［C］// Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2023： 89-98.
23	WANG F， WANG Y， LI D， et al. CL4CTR： a contrastive learning framework for CTR prediction ［C］// Proceedings of the 16th ACM International Conference on Web Search and Data Mining. New York： ACM， 2023： 805-813.
24	WANG Z， LIU H， WEI W， et al. Multi-level contrastive learning framework for sequential recommendation ［C］// Proceedings of the 31st ACM International Conference on Information and Knowledge Management. New York： ACM， 2022： 2098-2107.
25	WU F， ZHANG T， DE SOUZA A H， Jr， et al. Simplifying graph convolutional networks ［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 6861-6871.
26	CHEN L， WU L， HONG R， et al. Revisiting graph based collaborative filtering： a linear residual graph convolutional network approach ［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2020： 27-34.
27	BRODY S， ALON U， YAHAV E. How attentive are graph attention networks？［EB/OL］. ［2024-01-31］..
28	WANG Z， WEI W， CONG G， et al. Global context enhanced graph neural networks for session-based recommendation ［C］// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2020： 169-178.
29	BECK D， HAFFARI G， COHN T. Graph-to-sequence learning using gated graph neural networks ［C］// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics （Volume 1： Long Papers）. Stroudsburg： ACL， 2018： 273-283.
30	MIN E， RONG Y， XU T， et al. Neighbour interaction based click-through rate prediction via graph-masked transformer ［C］// Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2022： 353-362.
31	YOU Y， CHEN T， SUI Y， et al. Graph contrastive learning with augmentations ［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2020： 5812-5823.
32	QIU J， CHEN Q， DONG Y， et al. GCC： graph contrastive coding for graph neural network pre-training［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York： ACM， 2020： 1150-1160.
33	LI B， HOU Y， CHE W. Data augmentation approaches in natural language processing： a survey［J］. AI Open， 2022， 3： 71-90.
34	LINDEN G， SMITH B， YORK J. Amazon.com recommendations： item-to-item collaborative filtering［J］. IEEE Internet Computing， 2003， 7（1）： 76-80.
35	HAVASI M， JENATTON R， FORT S， et al. Training independent subnetworks for robust prediction［EB/OL］. ［2024-08-04］..
36	HE K， FAN H， WU Y， et al. Momentum contrast for unsupervised visual representation learning ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 9729-9738.
37	WANG F， LIU H. Understanding the behaviour of contrastive loss ［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 2495-2504.
38	CHEN T， KORNBLITH S， NOROUZI M， et al. A simple framework for contrastive learning of visual representations ［C］// Proceedings of the 37th International Conference on Machine Learning. New York： JMLR.org， 2020： 1597-1607.
39	CHEN Y， LIU Z， LI J， et al. Intent contrastive learning for sequential recommendation ［C］// Proceedings of the ACM Web Conference 2022. New York： ACM， 2022： 2172-2182.
40	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. ［2024-01-30］..

数据集	用户数	项目数	交互次数	用户平均长度	稠密度/%
Sports	35 598	18 357	296 337	8.3	99.95
Beauty	22 363	12 101	198 502	8.8	99.93
Toys	19 412	11 924	167 597	8.6	99.93

数据集	用户数	项目数	交互次数	用户平均长度	稠密度/%
Sports	35 598	18 357	296 337	8.3	99.95
Beauty	22 363	12 101	198 502	8.8	99.93
Toys	19 412	11 924	167 597	8.6	99.93

超参数	值
Batch_size	256
学习率Learning rate	0.000 1
训练迭代次数Epoch	200
权重衰减weight_decay BI-GGNN缩放范围scale	0.000 01 15
序列推荐任务占比α	0.9
嵌入层对比学习任务占比β	0.03
节点层对比学习任务占比γ	0.03
序列层对比学习任务占比δ	0.04
BI-GGNN层数	3
BI-GGNN的丢弃率dropout	0.3
嵌入维度大小	512

超参数	值
Batch_size	256
学习率Learning rate	0.000 1
训练迭代次数Epoch	200
权重衰减weight_decay BI-GGNN缩放范围scale	0.000 01 15
序列推荐任务占比α	0.9
嵌入层对比学习任务占比β	0.03
节点层对比学习任务占比γ	0.03
序列层对比学习任务占比δ	0.04
BI-GGNN层数	3
BI-GGNN的丢弃率dropout	0.3
嵌入维度大小	512

数据集	评价指标	对比模型					本文模型	本文模型与对比模型最优值的差异/%
数据集	评价指标	CL4SRec	CoSeRec	ICLRec	DuoRec	MCLRec	本文模型	本文模型与对比模型最优值的差异/%
Sports	HR@5	0.023 1	0.025 6	0.029 0	0.0312	0.032 8	0.035 3	7.6
	HR@10	0.036 9	0.040 2	0.043 7	0.0466	0.050 1	0.051 0	1.8
	HR@20	0.055 7	0.060 5	0.064 6	0.0696	0.073 4	0.073 9	0.7
	NDCG@5	0.014 6	0.017 0	0.019 1	0.0192	0.020 4	0.024 6	20.6
	NDCG@10	0.019 1	0.021 7	0.023 8	0.0244	0.026 0	0.029 7	14.2
	NDCG@20	0.023 8	0.026 8	0.029 1	0.0302	0.031 9	0.035 5	11.3
Beauty	HR@5	0.040 1	0.046 6	0.050 0	0.0559	0.058 1	0.061 4	5.7
	HR@10	0.064 2	0.069 9	0.074 4	0.0825	0.087 1	0.087 6	0.6
	HR@20	0.097 4	0.102 3	0.105 8	0.1193	0.124 3	0.124 7	0.3
	NDCG@5	0.026 8	0.030 9	0.032 6	0.0340	0.035 2	0.044 6	26.7
	NDCG@10	0.034 5	0.038 4	0.040 3	0.0425	0.044 6	0.053 1	19.1
	NDCG@20	0.042 8	0.046 5	0.048 3	0.0518	0.053 9	0.061 4	13.9
Toys	HR@5	0.049 7	0.055 5	0.059 8	0.0642	0.065 0	0.071 1	9.4
	HR@10	0.071 6	0.079 3	0.083 4	0.0927	0.093 0	0.096 3	3.5
	HR@20	0.093 5	0.108 2	0.113 8	0.1251	0.126 2	0.128 3	1.7
	NDCG@5	0.025 4	0.0377	0.040 4	0.037 3	0.037 2	0.051 0	26.2
	NDCG@10	0.032 1	0.045 4	0.048 0	0.044 3	0.0461	0.059 1	23.1
	NDCG@20	0.039 8	0.052 7	0.055 7	0.052 1	0.0545	0.067 2	20.6