Two-stage recommendation algorithm of Siamese graph convolutional neural network

doi:10.11772/j.issn.1001-9081.2023020180

Abstract

Abstract:

To solve the problem that the two-tower neural network in the recommendation system is difficult to learn the interaction information between the user side and the item side and the graph connection information， a new algorithm TSN （Two-stage Siamese graph convolutional Neural network recommendation algorithm） was proposed. First， a heterogeneous graph based on user behavior was built. Then， a graph convolutional Siamese network was designed between the two-tower neural networks， so as to achieve information interaction while learning the connection information of the heterogeneous graph. Finally， by designing a special structure of two-stage information sharing mechanism， the neural networks on the user side and the item side could transmit information dynamically and bidirectionally during the training process， and neural network cascading was effectively avoided. In comparative experiments on MovieLens and Douban movie datasets， the NDCG@10， NDCG@50， NDCG@100 of the proposed algorithm are 11.39% to 23.98% higher than those of the optimal benchmark algorithm DAT （Dual Augmented Two-tower model for online large-scale recommendation）. The results show that the proposed algorithm can alleviate the problem of lack of information interaction in the two-tower neural network； and significantly improves the recommendation performance compared with the previous algorithms.

Key words: recommendation system, two-tower model, Siamese network, deep learning, convolutional neural network

摘要：

针对推荐系统中双塔型神经网络难以学习用户侧和商品侧交互信息以及图连接信息的问题，提出一种二阶段孪生卷积神经网络推荐算法（TSN）。首先，以用户行为构建异质图；然后，在双塔型神经网络之间设计图卷积孪生网络，从而在学习异质图连接信息的同时进行信息交互；最后，通过设计特殊结构的二阶段孪生信息共享机制，使得用户侧和商品侧的神经网络在训练过程中能够动态地、双向地传输信息，且有效避免神经网络串联。在基于MovieLens和豆瓣电影数据集的对比实验中，NDCG@10、NDCG@50、NDCG@100相较于最优基准算法DAT（Dual Augmented Two-tower model for online large-scale recommendation）提升了11.39%~23.98%。结果表明，所提算法能够缓解双塔型神经网络缺乏信息交互的问题，较对比算法推荐性能提升显著。

关键词: 推荐系统, 双塔模型, 孪生网络, 深度学习, 卷积神经网络

CLC Number:

TP391.3

Zhiwen JING, Yujia ZHANG, Boting SUN, Hao GUO. Two-stage recommendation algorithm of Siamese graph convolutional neural network[J]. Journal of Computer Applications, 2024, 44(2): 469-476.

荆智文, 张屿佳, 孙伯廷, 郭浩. 二阶段孪生图卷积神经网络推荐算法[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 469-476.

Figures/Tables 9

Fig. 1 Structure of Siamese neural network

Fig. 2 Architecture of two-stage Siamese graph convolutional neural network recommendation algorithm

Tab. 1 Definition of important notations

符号	定义
$U$	用户集合
$V$	商品集合
$u f i$	用户 $u i$ 的画像文档
$D = {d 1, d 2, …, d m + n}$	用户及商品文档集合
$u d i ∈ U D$	用户 $u i$ 的特征密集向量
$v d j ∈ V D$	商品 $v j$ 的特征密集向量
$U E$	用户侧深度神经网络输出的特征矩阵
$V E$	商品侧深度神经网络输出的特征矩阵
$R ∈ R m × n$	评分矩阵
$G ̂ i n t e r u$	单个用户子图
$W D N N$	深度神经网络参数矩阵
$W S i a$	孪生卷积神经网络参数矩阵

Tab. 1 Definition of important notations

符号	定义
$U$	用户集合
$V$	商品集合
$u f i$	用户 $u i$ 的画像文档
$D = {d 1, d 2, …, d m + n}$	用户及商品文档集合
$u d i ∈ U D$	用户 $u i$ 的特征密集向量
$v d j ∈ V D$	商品 $v j$ 的特征密集向量
$U E$	用户侧深度神经网络输出的特征矩阵
$V E$	商品侧深度神经网络输出的特征矩阵
$R ∈ R m × n$	评分矩阵
$G ̂ i n t e r u$	单个用户子图
$W D N N$	深度神经网络参数矩阵
$W S i a$	孪生卷积神经网络参数矩阵

Tab. 2 Statistics of experimental datasets

数据集	用户数	商品数	评分数	评分范围
MovieLens	10 000	9 395	1 462 905	［0.5， 5］
豆瓣电影	2 712	34 893	1 278 401	［1， 5］

Tab. 3 Experimental results of different models on MovieLens dataset

模型	HR@N			NDCG@N			MRR@N
模型	N=10	N=50	N=100	N=10	N=50	N=100	N=10	N=50	N=100
TF⁃IDF	0.013 4	0.024 2	0.035 9	0.021 3	0.044 2	0.052 2	0.035 1	0.047 9	0.068 9
FM	0.018 9	0.040 6	0.063 4	0.031 2	0.050 3	0.065 9	0.058 9	0.063 6	0.071 2
YoutubeDNN	0.021 2	0.037 8	0.059 9	0.030 3	0.048 7	0.059 3	0.052 4	0.059 1	0.072 3
DSSM	0.029 4	0.046 4	0.073 2	0.037 5	0.058 7	0.067 3	0.062 5	0.069 7	0.082 1
STAN	0.033 2	0.048 8	0.077 5	0.043 0	0.062 9	0.073 3	0.067 5	0.075 8	0.087 4
HIRS	0.034 9	0.049 9	0.080 3	0.045 7	0.064 6	0.076 9	0.068 6	0.078 8	0.089 4
DAT	0.038 8	0.051 9	0.083 6	0.053 6	0.068 8	0.082 4	0.071 1	0.081 4	0.092 3
TSN（GC）	0.040 3	0.055 2	0.087 9	0.056 8	0.072 2	0.092 9	0.074 1	0.080 2	0.093 6

Tab. 4 Experimental results of different models on Douban movie dataset

模型	HR@N			NDCG@N			MRR@N
模型	N=10	N=50	N=100	N=10	N=50	N=100	N=10	N=50	N=100
TF⁃IDF	0.016 7	0.023 2	0.031 7	0.012 1	0.020 4	0.028 9	0.015 2	0.022 7	0.031 1
FM	0.022 9	0.030 4	0.042 2	0.019 7	0.027 3	0.040 2	0.022 9	0.030 2	0.043 1
YoutubeDNN	0.034 4	0.046 7	0.050 3	0.024 4	0.035 6	0.047 7	0.033 4	0.041 4	0.050 7
DSSM	0.051 1	0.060 3	0.065 9	0.033 2	0.046 5	0.052 6	0.049 7	0.050 3	0.055 8
STAN	0.052 9	0.066 7	0.068 7	0.039 0	0.051 6	0.060 9	0.050 8	0.054 4	0.062 9
HIRS	0.054 9	0.068 1	0.071 1	0.041 3	0.053 9	0.062 2	0.052 7	0.056 6	0.064 7
DAT	0.059 4	0.071 1	0.077 2	0.046 7	0.067 6	0.068 6	0.056 6	0.067 6	0.069 6
TSN（GC）	0.065 3	0.078 7	0.083 9	0.057 9	0.075 3	0.078 8	0.067 0	0.074 5	0.079 8

Fig. 3 Scatter diagrams （t-SNE） of user and item features for TSN and optimal benchmark algorithm DAT

Tab. 5 Experimental results of TSN（GC） and its variants on Douban movie dataset

模型	HR@N			NDCG@N			MRR@N
模型	N=10	N=50	N=100	N=10	N=50	N=100	N=10	N=50	N=100
TSN（w/o TS，GC）	0.032 2	0.044 5	0.049 6	0.022 1	0.033 9	0.043 5	0.031 2	0.038 7	0.047 5
TSN（U⁃GC）	0.057 3	0.067 4	0.071 2	0.045 5	0.060 6	0.062 3	0.056 4	0.057 6	0.064 1
TSN（I⁃GC）	0.055 2	0.063 3	0.070 2	0.045 9	0.061 7	0.063 3	0.054 8	0.057 9	0.062 7
TSN（TS）	0.065 0	0.079 4	0.082 1	0.057 6	0.075 0	0.077 5	0.066 7	0.074 1	0.080 7
TSN（GC）	0.065 3	0.078 7	0.083 9	0.057 9	0.075 3	0.078 8	0.067 0	0.074 5	0.079 8

Fig. 4 Effect of HS-GCN training intensity on performance of TSN and its variants

References 22

1	赵俊逸，庄福振，敖翔，等. 协同过滤推荐系统综述［J］. 信息安全学报， 2021， 6（5）：17-34. 10.19363/J.cnki.cn10-1380/tn.2021.09.02
	ZHAO J Y， ZHUANG F Z， AO X， et al. Survey of collaborative filtering recommender systems［J］. Journal of Cyber Security， 2021， 6（5）：17-34. 10.19363/J.cnki.cn10-1380/tn.2021.09.02
2	KO H， LEE S， PARK Y， et al. A survey of recommendation systems： recommendation models， techniques， and application fields［J］. Electronics， 2022， 11（1）： No.141. 10.3390/electronics11010141
3	HUANG P S， HE X， GAO J， et al. Learning deep structured semantic models for web search using clickthrough data［C］// Proceedings of the 22nd ACM International Conference on Information & Knowledge Management. New York： ACM， 2013： 2333-2338. 10.1145/2505515.2505665
4	YU Y， WANG W， FENG Z， et al. A dual augmented two-tower model for online large-scale recommendation［EB/OL］. ［2023-01-20］..
5	BROMLEY J， GUYON I， LeCUN Y， et al. Signature verification using a “Siamese” time delay neural network［C］// Proceedings of the 6th International Conference on Neural Information Processing Systems. San Francisco： Morgan Kaufmann Publishers Inc.， 1993： 737-744. 10.1142/9789812797926_0003
6	王梦亭，杨文忠，武雍智. 基于孪生网络的单目标跟踪算法综述［J］. 计算机应用， 2023， 43（3）：661-673. 10.11772/j.issn.1001-9081.2022010150
	WANG M T， YANG W Z， WU Y Z. Survey of single target tracking algorithms based on Siamese networks［J］. Journal of Computer Applications， 2023， 43（3）：661-673. 10.11772/j.issn.1001-9081.2022010150
7	VINEELA A， LAVANYA DEVI G， NELATURI N， et al. A comprehensive study and evaluation of recommender systems［C］// Proceedings of the 5th ICMEET 2019. Singapore： Springer， 2021： 45-53. 10.1007/978-981-15-3828-5_5
8	SANTOS C F G D， PAPA J P. Avoiding overfitting： a survey on regularization methods for convolutional neural networks［J］. ACM Computing Surveys， 2022， 54（10s）： No.213. 10.1145/3510413
9	CONG S， ZHOU Y. A review of convolutional neural network architectures and their optimizations［J］. Artificial Intelligence Review， 2023， 56（3）： 1905-1969. 10.1007/s10462-022-10213-5
10	ABBAS S， ALHWAITI Y， FATIMA A， et al. Convolutional neural network based intelligent handwritten document recognition［J］. Computers， Materials and Continua， 2022， 70（3）： 4563-4581. 10.32604/cmc.2022.021102
11	MA C， MA L， ZHANG Y， et al. Memory augmented graph neural networks for sequential recommendation［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 5045-5052. 10.1609/aaai.v34i04.5945
12	YANG Y， RAO Y， YU M， et al. Multi-layer information fusion based on graph convolutional network for knowledge-driven herb recommendation［J］. Neural Networks， 2022， 146： 1-10. 10.1016/j.neunet.2021.11.010
13	LIU Z， YANG L， FAN Z， et al. Federated social recommendation with graph neural network［J］. ACM Transactions on Intelligent Systems and Technology， 2022， 13（4）： No.55. 10.1145/3501815
14	SU Y， ZHAO Y， ERFANI S， et al. Detecting arbitrary order beneficial feature interactions for recommender systems［C］// Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York： ACM， 2022： 1676-1686. 10.1145/3534678.3539238
15	ALAM M T， UBAID S， SOHAIL S S， et al. Comparative analysis of machine learning based filtering techniques using MovieLens dataset［J］. Procedia Computer Science， 2021， 194： 210-217. 10.1016/j.procs.2021.10.075
16	YU H， FU M. DbRMP： predicting Douban rating of movies with high-dimensional features by comprehensive machine learning algorithms［C］// Proceedings of the 2022 IEEE International Conference on Artificial Intelligence and Computer Applications. Piscataway： IEEE， 2022： 540-544. 10.1109/icaica54878.2022.9844633
17	AIZAWA A. An information-theoretic perspective of TF-IDF measures［J］. Information Processing and Management， 2003， 39（1）： 45-65. 10.1016/s0306-4573(02)00021-3
18	RENDLE S. Factorization machines［C］// Proceedings of the 2010 IEEE International Conference on Data Mining. Piscataway： IEEE， 2010： 995-1000. 10.1109/icdm.2010.127
19	COVINGTON P， ADAMS J， SARGIN E. Deep neural networks for YouTube recommendations［C］// Proceedings of the 10th ACM Conference on Recommender Systems. New York： ACM， 2016： 191-198. 10.1145/2959100.2959190
20	LUO Y， LIU Q， LIU Z. STAN： spatio-temporal attention network for next location recommendation［C］// Proceedings of the Web Conference 2021. Republic and Canton of Geneva： International World Wide Web Conferences Steering Committee， 2021： 2177-2185. 10.1145/3442381.3449998
21	JANNACH D， MANZOOR A， CAI W， et al. A survey on conversational recommender systems［J］. ACM Computing Surveys， 2022， 54（5）： No.105. 10.1145/3453154
22	KO H， LEE S， PARK Y， et al. A survey of recommendation systems： recommendation models， techniques， and application fields［J］. Electronics， 2022， 11（1）： No.141. 10.3390/electronics11010141

[1]	Jiawei ZHANG, Guandong GAO, Ke XIAO, Shengzun SONG. Violent crime hierarchy algorithm by joint modeling of improved hierarchical attention network and TextCNN [J]. Journal of Computer Applications, 2024, 44(2): 403-410.
[2]	Junhong ZHU, Junyu LAI, Lianqiang GAN, Zhiyong CHEN, Huashuo LIU, Guoyao XU. Video prediction model combining involution and convolution operators [J]. Journal of Computer Applications, 2024, 44(1): 113-122.
[3]	Yuning ZHANG, Abudukelimu ABULIZI, Tisheng MEI, Chun XU, Maierdana MAIMAITIREYIMU, Halidanmu ABUDUKELIMU, Yutao HOU. Anomaly detection method for skeletal X-ray images based on self-supervised feature extraction [J]. Journal of Computer Applications, 2024, 44(1): 175-181.
[4]	Yunyun GAO, Lasheng ZHAO, Qiang ZHANG. Acoustic word embedding model based on Bi-LSTM and convolutional-Transformer [J]. Journal of Computer Applications, 2024, 44(1): 123-128.
[5]	Hao CHEN, Zhenping XIA, Cheng CHENG, Xing LIN-LI, Bowen ZHANG. Lightweight image super-resolution reconstruction network based on Transformer-CNN [J]. Journal of Computer Applications, 2024, 44(1): 292-299.
[6]	Tong CHEN, Jiwei WEI, Shiyuan HE, Jingkuan SONG, Yang YANG. Adversarial training method with adaptive attack strength [J]. Journal of Computer Applications, 2024, 44(1): 94-100.
[7]	Ziyi HE, Yan YANG, Yiling ZHANG. Multi-view clustering network with deep fusion [J]. Journal of Computer Applications, 2023, 43(9): 2651-2656.
[8]	Kunting LU, Rongrong FEI, Xuande ZHANG. Remote sensing image pansharpening by convolutional neural network [J]. Journal of Computer Applications, 2023, 43(9): 2963-2969.
[9]	Juntao CHEN, Ziqi ZHU. Image copy-move forgery detection based on multi-scale feature extraction and fusion [J]. Journal of Computer Applications, 2023, 43(9): 2919-2924.
[10]	Shaofa SHANG, Lin JIANG, Yuancheng LI, Yun ZHU. Adaptive partitioning and scheduling method of convolutional neural network inference model on heterogeneous platforms [J]. Journal of Computer Applications, 2023, 43(9): 2828-2835.
[11]	Xiang GUO, Wengang JIANG, Yuhang WANG. Encrypted traffic classification method based on improved Inception-ResNet [J]. Journal of Computer Applications, 2023, 43(8): 2471-2476.
[12]	Yumeng CUI, Jingya WANG, Xiaowen LIU, Shangyi YAN, Zhizhong TAO. General text classification model combining attention and cropping mechanism [J]. Journal of Computer Applications, 2023, 43(8): 2396-2405.
[13]	Kun ZHANG, Fengyu YANG, Fa ZHONG, Guangdong ZENG, Shijian ZHOU. Source code vulnerability detection based on hybrid code representation [J]. Journal of Computer Applications, 2023, 43(8): 2517-2526.
[14]	Doudou LI, Wanggen LI, Yichun XIA, Yang SHU, Kun GAO. Skeleton-based action recognition based on feature interaction and adaptive fusion [J]. Journal of Computer Applications, 2023, 43(8): 2581-2587.
[15]	Xiaolin LI, Songjia YANG. Hybrid beamforming for multi-user mmWave relay networks using deep learning [J]. Journal of Computer Applications, 2023, 43(8): 2511-2516.