Personalized recommendation service system based on cloud-client-convergence

doi:10.11772/j.issn.1001-9081.2021111992

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (11): 3506-3512.DOI: 10.11772/j.issn.1001-9081.2021111992

Special Issue: CCF 2021中国数字服务大会

• ChinaService 2021 • Previous Articles Next Articles

Personalized recommendation service system based on cloud-client-convergence

Jialiang HAN¹, Yudong HAN¹, Xuanzhe LIU¹, Yaoshuai ZHAO²^,³, Di FENG²^,³()

^1.Key Laboratory of High Confidence Software Technologies of Ministry of Education （Peking University），Beijing 100871，China
^2.TravelSky Technology Limited，Beijing 101318，China
^3.Key Laboratory of Intelligent Application Technology for Civil Aviation Passenger Services，Civil Aviation Administration of China，Beijing 101318，China

Received:2021-11-23 Revised:2022-01-12 Accepted:2022-01-17 Online:2022-03-02 Published:2022-11-10
Contact: Di FENG
About author:HAN Jialiang， born in 1997， Ph. D. candidate. His research interests include distributed machine learning， federated learning， recommender system.
HAN Yudong， born in 2000， Ph. D. candidate. His research interests include distributed machine learning， Web system.
LIU Xuanzhe， born in 1980， Ph. D.， associate professor. His research interests include service computing， system software.
ZHAO Yaoshuai， born in 1977， M. S.， senior engineer. His research interests include big data， artificial intelligence.
FENG Di， born in 1981， M. S.， engineer. Her research interests include civil aviation passenger behavior analysis， data analysis.
Supported by:
PKU-Baidu Fund(2020BD007)

基于云‒端融合的个性化推荐服务系统

韩佳良¹, 韩宇栋¹, 刘譞哲¹, 赵耀帅²^,³, 冯迪²^,³()

^1.高可信软件技术教育部重点实验室(北京大学), 北京 100871
^2.中国民航信息网络股份有限公司, 北京 101318
^3.中国民用航空局民航旅客服务智能化应用技术重点实验室, 北京 101318

通讯作者: 冯迪
作者简介:韩佳良（1997—），男，北京人，博士研究生，CCF会员，主要研究方向：分布式机器学习、联邦学习、推荐系统
韩宇栋（2000—），男，山东东营人，博士研究生，主要研究方向：分布式机器学习、Web系统
刘譞哲（1980—），男，甘肃兰州人，副教授，博士，CCF会员，主要研究方向：服务计算、系统软件
赵耀帅（1977—），男，山东嘉祥人，高级工程师，硕士，主要研究方向：大数据、人工智能
冯迪（1981—），女，湖北潜江人，工程师，硕士，主要研究方向：民航旅客行为分析、数据分析。fengdi@travelsky.com.cn
基金资助:
北大百度基金资助项目(2020BD007)

Abstract

Abstract:

Mainstream personalized recommendation systems usually use models deployed in the cloud to perform recommendation， so the private data such as user interaction behaviors need to be uploaded to the cloud， which may cause potential risks of user privacy leakage. In order to protect user privacy， user-sensitive data can be processed on the client， however， there are communication bottleneck and computation resource bottleneck in clients. Aiming at the above challenges， a personalized recommendation service system based on cloud-client-convergence was proposed. In this system， the cloud-based recommendation model was divided into a user representation model and a sorting model. After being pre-trained on the cloud， the user representation model was deployed to the client， while the sorting model was deployed to the cloud. A small-scale Recurrent Neural Network （RNN） was used to model the user behavior characteristics by extracting temporal information from user interaction logs， and the Lasso （Least absolute shrinkage and selection operator） algorithm was used to compress user representations， thereby preventing a drop in recommendation accuracy while reducing the communication overhead between the cloud and the client as well as the computation overhead of the client. Experiments were conducted on RecSys Challenge 2015 dataset， and the results show that the recommendation accuracy of the proposed system is comparable to that of the GRU4REC model， while the volume of the compressed user representations is only 34.8% of that before compression， with a lower computational overhead.

Key words: personalized recommendation service system, cloud-client-convergence, user representation model, privacy-preserving, Recurrent Neural Network (RNN)

摘要：

主流个性化推荐服务系统通常利用部署在云端的模型进行推荐，因此需要将用户交互行为等隐私数据上传到云端，这会造成隐私泄露的隐患。为了保护用户隐私，可以在客户端处理用户敏感数据，然而，客户端存在通信瓶颈和计算资源瓶颈。针对上述挑战，设计了一个基于云?端融合的个性化推荐服务系统。该系统将传统的云端推荐模型拆分成用户表征模型和排序模型，在云端预训练用户表征模型后，将其部署到客户端，排序模型则部署到云端；同时，采用小规模的循环神经网络（RNN）抽取用户交互日志中的时序信息来训练用户表征，并通过Lasso算法对用户表征进行压缩，从而在降低云端和客户端之间的通信量以及客户端的计算开销的同时防止推荐准确率的下跌。基于RecSys Challenge 2015数据集进行了实验，结果表明，所设计系统的推荐准确率和GRU4REC模型相当，而压缩后的用户表征体积仅为压缩前的34.8%，计算开销较低。

CLC Number:

TP311

Jialiang HAN, Yudong HAN, Xuanzhe LIU, Yaoshuai ZHAO, Di FENG. Personalized recommendation service system based on cloud-client-convergence[J]. Journal of Computer Applications, 2022, 42(11): 3506-3512.

韩佳良, 韩宇栋, 刘譞哲, 赵耀帅, 冯迪. 基于云‒端融合的个性化推荐服务系统[J]. 《计算机应用》唯一官方网站, 2022, 42(11): 3506-3512.

Figures/Tables 9

References 19

1	GOLDBERG D， NICHOLS D， OKI B M， et al. Using collaborative filtering to weave an information tapestry［J］. Communications of the ACM， 1992， 35（12）： 61-70. 10.1145/138859.138867
2	SARWAR B， KARYPIS G， KONSTAN J， et al. Item-based collaborative filtering recommendation algorithms［C］// Proceedings of the 10th International Conference on World Wide Web. New York： ACM， 2001： 285-295. 10.1145/371920.372071
3	CHENG H T， KOC L， HARMSEN J， et al. Wide & deep learning for recommender systems［C］// Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. New York： ACM， 2016： 7-10. 10.1145/2988450.2988454
4	GUO H， TANG R， YE Y， et al. DeepFM： a factorization-machine based neural network for CTR prediction［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 1725-1731. 10.24963/ijcai.2017/239
5	HIDASI B， KARATZOGLOU A， BALTRUNAS L， et al. Session-based recommendations with recurrent neural networks［EB/OL］. （2016-03-29）［2021-09-08］..
6	HIDASI B， QUADRANA M， KARATZOGLOU A， et al. Parallel recurrent neural network architectures for feature-rich session-based recommendations［C］// Proceedings of the 10th ACM Conference on Recommender Systems. New York： ACM， 2016： 241-248. 10.1145/2959100.2959167
7	BOGINA V， KUFLIK T. Incorporating dwell time in session-based recommendations with recurrent neural networks［C］// Proceedings of the 1st Workshop on Temporal Reasoning in Recommender Systems co-located with 11th International Conference on Recommender Systems. Aachen： CEUR-WS.org， 2017： 57-59. 10.1145/3109859.3109952
8	JANNACH D， LUDEWIG M. When recurrent neural networks meet the neighborhood for session-based recommendation［C］// Proceedings of the 11th ACM Conference on Recommender Systems. New York： ACM， 2017： 306-310. 10.1145/3109859.3109872
9	XU M W， QIAN F， MEI Q Z， et al. DeepType： on-device deep learning for input personalization service with minimal privacy concern［J］. Proceedings of the ACM on Interactive， Mobile， Wearable and Ubiquitous Technologies， 2018， 2（4）： No.197. 10.1145/3287075
10	RENDLE S， FREUDENTHALER C， GANTNER Z， et al. BPR： Bayesian personalized ranking from implicit feedback［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington， VA： AUAI Press， 2009： 452-461.
11	OKURA S， TAGAMI Y， ONO S， et al. Embedding-based news recommendation for millions of users［C］// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2017： 1933-1942. 10.1145/3097983.3098108
12	BURGES C J C. From RankNet to LambdaRank to LambdaMART： an overview： MSR-TR-2010-82［R/OL］. （2010-06）［2021-09-08］..
13	DAVIDSON J， LIEBALD B， LIU J N， et al. The YouTube video recommendation system［C］// Proceedings of the 4th ACM Conference on Recommender Systems. New York： ACM， 2010： 293-296. 10.1145/1864708.1864770
14	McMAHAN B， MOORE E， RAMAGE D， et al. Communication-efficient learning of deep networks from decentralized data［C］// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York： JMLR.org， 2017： 1273-1282.
15	BONAWITZ K， EICHNER H， GRIESKAMP W， et al. Towards federated learning at scale： system design［C/OL］// Proceedings of the 2nd Machine Learning and Systems. ［2021-11-01］..
16	HAN S， MAO H Z， DALLY W J. Deep compression： compressing deep neural network with pruning， trained quantization and Huffman coding［EB/OL］. （2016-02-15）［2021-09-12］.. 10.1109/isca.2016.30
17	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network［EB/OL］. （2015-03-09）［2021-10-16］..
18	ZHU M H， GUPTA S. To prune， or not to prune： exploring the efficacy of pruning for model compression［EB/OL］. （2018-02-13）［2021-10-08］..
19	CHEN T， SUN Y Z， SHI Y， et al. On sampling strategies for neural network-based collaborative filtering［C］// Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2017： 767-776. 10.1145/3097983.3098202

功能模块	服务器	个人笔记本电脑
CPU	Intel E5-2650 v3	Intel i7-8950H
GPU	Nvidia Tesla M40	Nvidia GeForce GTX 1070
CPU内存	128 819 MB	7 822 MB
GPU内存	11 448 MiB	8 119 MiB

功能模块	服务器	个人笔记本电脑
CPU	Intel E5-2650 v3	Intel i7-8950H
GPU	Nvidia Tesla M40	Nvidia GeForce GTX 1070
CPU内存	128 819 MB	7 822 MB
GPU内存	11 448 MiB	8 119 MiB

基线	Recall@20	MRR@20
POP	0.005	0.001
S-POP	0.267	0.178
Item‑KNN	0.507	0.205
BPR‑MF	0.257	0.062
GRU4REC （CE）	0.621	0.253
GRU4REC （BPR）	0.604	0.243
GRU4REC （TOP1）	0.631	0.264

基线	Recall@20	MRR@20
POP	0.005	0.001
S-POP	0.267	0.178
Item‑KNN	0.507	0.205
BPR‑MF	0.257	0.062
GRU4REC （CE）	0.621	0.253
GRU4REC （BPR）	0.604	0.243
GRU4REC （TOP1）	0.631	0.264

模型	Recall@20	MRR@20
CE w/o Lasso	0.621	0.253
CE w/ Lasso	0.618	0.266
BPR w/o Lasso	0.604	0.243
BPR w/ Lasso	0.581	0.249
TOP1 w/o Lasso	0.631	0.264
TOP1 w/ Lasso	0.582	0.234

Personalized recommendation service system based on cloud-client-convergence

基于云‒端融合的个性化推荐服务系统

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 19

Related Articles 15

Recommended Articles

Metrics

模型	Recall@20	MRR@20
GRU4REC （TOP1）	0.631	0.264
Small GRU + Lasso （CE）	0.618	0.266
Small GRU + Lasso （BPR）	0.581	0.249
Small GRU + Lasso （TOP1）	0.582	0.234

模型	稠密度	压缩率
CE w/o Lasso	0.996	1
CE w/ Lasso	0.232	0.348
BPR w/o Lasso	0.996	1
BPR w/ Lasso	0.235	0.352
TOP1 w/o Lasso	0.996	1
TOP1 w/ Lasso	0.404	0.603

评测指标	实验结果
CPU占用率	62.0%
CPU内存占用	1 030.6 MB
GPU内存占用	661 MiB
单次用户表征更新所需时间	53 s

[1]	Jianyang CUI, Ying CAI, Yu ZHANG, Yanfang FAN. Authenticatable privacy-preserving scheme based on signcryption from lattice for vehicular ad hoc network [J]. Journal of Computer Applications, 2024, 44(1): 233-241.
[2]	Yunyun GAO, Lasheng ZHAO, Qiang ZHANG. Acoustic word embedding model based on Bi-LSTM and convolutional-Transformer [J]. Journal of Computer Applications, 2024, 44(1): 123-128.
[3]	Mengjie LAN, Jianping CAI, Lan SUN. Self-regularization optimization methods for Non-IID data in federated learning [J]. Journal of Computer Applications, 2023, 43(7): 2073-2081.
[4]	Wanzhen CHEN, En ZHANG, Leiyong QIN, Shuangxi HONG. Privacy-preserving federated learning algorithm based on blockchain in edge computing [J]. Journal of Computer Applications, 2023, 43(7): 2209-2216.
[5]	Cong YIN, Hanping HU. Parameter identification model for time-delay chaotic systems based on temporal attention mechanism [J]. Journal of Computer Applications, 2023, 43(3): 842-847.
[6]	Nanfan LI, Wenwen SI, Siyuan DU, Zhiyong WANG, Chongyang ZHONG, Shihong XIA. Hidden state initialization method for recurrent neural network-based human motion model [J]. Journal of Computer Applications, 2023, 43(3): 723-727.
[7]	Teng WANG, Zheng HUO, Yaxin HUANG, Yilin FAN. Review on privacy-preserving technologies in federated learning [J]. Journal of Computer Applications, 2023, 43(2): 437-449.
[8]	Guishuang XU, Xinchun YIN. Certificateless conditional privacy-preserving authentication scheme for VANET [J]. Journal of Computer Applications, 2023, 43(11): 3358-3367.
[9]	Meng YU, Wentao HE, Xuchuan ZHOU, Mengtian CUI, Keqi WU, Wenjie ZHOU. Review of recommendation system [J]. Journal of Computer Applications, 2022, 42(6): 1898-1913.
[10]	Xueqin CHEN, Tao TAO, Zhongwang ZHANG, Yilei WANG. Long short-term session-based recommendation algorithm combining paired coding scheme and two-dimensional conventional neural network [J]. Journal of Computer Applications, 2022, 42(5): 1347-1354.
[11]	Yang ZHONG, Renwan BI, Xishan YAN, Zuobin YING, Jinbo XIONG. Efficient homomorphic neural network supporting privacy-preserving training [J]. Journal of Computer Applications, 2022, 42(12): 3792-3800.
[12]	Jiana MENG, Pin LYU, Yuhai YU, Shichang SUN, Hongfei LIN. Aspect-level cross-domain sentiment analysis based on capsule network [J]. Journal of Computer Applications, 2022, 42(12): 3700-3707.
[13]	LIU Zichen, LI Xiaojuan, WEI Wei. Automatic patent price evaluation based on recurrent neural network [J]. Journal of Computer Applications, 2021, 41(9): 2532-2538.
[14]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[15]	ZHAO Xiaohu, LI Xiao. Image captioning algorithm based on multi-feature extraction [J]. Journal of Computer Applications, 2021, 41(6): 1640-1646.