Staged variational autoencoder for heterogeneous one-class collaborative filtering

doi:10.11772/j.issn.1001-9081.2021060894

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (12): 3499-3507.DOI: 10.11772/j.issn.1001-9081.2021060894

Special Issue: 第十八届中国机器学习会议(CCML 2021)

• The 18th China Conference on Machine Learning • Previous Articles Next Articles

Staged variational autoencoder for heterogeneous one-class collaborative filtering

Xiancong CHEN¹^,²^,³, Weike PAN¹^,²^,³(), Zhong MING¹^,²^,³

^1.National Engineering Laboratory for Big Data System Computing Technology （Shenzhen University），Shenzhen Guangdong 518060，China
^2.Guangdong Laboratory for Artificial Intelligence and Digital Economy （Shenzhen）（Shenzhen University），Shenzhen Guangdong 518060，China
^3.College of Computer Science and Software Engineering，Shenzhen University，Shenzhen Guangdong 518060，China

Received:2021-05-12 Revised:2021-06-14 Accepted:2021-06-23 Online:2021-08-20 Published:2021-12-10
Contact: Weike PAN
About author:CHEN Xiancong， born in 1995， M. S. candidate. His research interests include personalized recommendation， transfer learning.
MING Zhong， born in 1967， Ph. D.， professor. His research interests include artificial intelligence， Web intelligence.
Supported by:
the Key Program of National Natural Science Foundation of China(61836005);the Surface Program of National Natural Science Foundation of China(61872249)

面向异构单类协同过滤的阶段式变分自编码器

陈宪聪¹^,²^,³, 潘微科¹^,²^,³(), 明仲¹^,²^,³

^1.大数据系统计算技术国家工程实验室（深圳大学），广东深圳 518060
^2.人工智能与数字经济广东省实验室（深圳）（深圳大学），广东深圳 518060
^3.深圳大学计算机与软件学院，广东深圳 518060

通讯作者: 潘微科
作者简介:陈宪聪（1995—），男，广东茂名人，硕士研究生，主要研究方向：个性化推荐、迁移学习
明仲（1967—），男，江西宁都人，教授，博士，CCF高级会员，主要研究方向：人工智能、网络智能。
基金资助:
国家自然科学基金重点项目(61836005);国家自然科学基金面上项目(61872249)

Abstract

Abstract:

In recommender system field， most of the existing works mainly focus on the One-Class Collaborative Filtering （OCCF） problem with only one type of users’ feedback， e.g.， purchasing feedback. However， users’ feedback is usually heterogeneous in real applications， so it has become a new challenge to model the users’ heterogeneous feedback to capture their true preferences. Focusing on the Heterogeneous One-Class Collaborative Filtering （HOCCF） problem （including users’ purchasing feedback and browsing feedback）， a transfer learning solution named Staged Variational AutoEncoder （SVAE） model was proposed. Firstly， the latent feature vectors were generated via the Multinomial Variational AutoEncoder （Multi-VAE） with users’ browsing feedback auxiliary data. Then， the obtained latent feature vectors were transferred to another Multi-VAE to assist the modeling of users’ target data， i.e.， purchasing feedback by this Multi-VAE. Experimental results on three real-world datasets show that the performance of SVAE model on the important metrics such as Precision@5 and Normalized Discounted Cumulative Gain@5 （NDCG@5） is significantly better than the performance of the state-of-the-art recommendation algorithms in most cases， demonstrating the effectiveness of the proposed model.

Key words: recommender system, users’ feedback, Heterogeneous One-Class Collaborative Filtering (HOCCF), transfer learning, Variational AutoEncoder (VAE)

摘要：

在推荐系统领域，大部分现有的工作主要关注仅有一种类型的用户反馈（如购买反馈）的单类协同过滤（OCCF）问题。然而，在现实的应用中，用户的反馈往往是异构的，因此如何对用户的异构反馈进行建模从而准确刻画用户的真实偏好成为了一个新的挑战。围绕异构单类协同过滤（HOCCF）问题（包含了用户的购买反馈和浏览反馈），提出了一个迁移学习解决方案——阶段式变分自编码器（SVAE）模型。首先，将用户的浏览反馈当作辅助数据，以多项式变分自编码器（Multi-VAE）为基础模型学习并生成隐特征向量；然后迁移该隐特征向量到另一路Multi-VAE，用于帮助该Multi-VAE对用户的目标数据（即购买反馈）进行建模。三个真实数据集上的实验结果显示，在多数情况下，SVAE模型在精确度（Precision@5）、归一化折损累计增益（NDCG@5）等重要指标上的表现显著优于其他流行的推荐算法，验证了所提模型的有效性。

CLC Number:

TP391

Xiancong CHEN, Weike PAN, Zhong MING. Staged variational autoencoder for heterogeneous one-class collaborative filtering[J]. Journal of Computer Applications, 2021, 41(12): 3499-3507.

陈宪聪, 潘微科, 明仲. 面向异构单类协同过滤的阶段式变分自编码器[J]. 《计算机应用》唯一官方网站, 2021, 41(12): 3499-3507.

Figures/Tables 11

Tab. 1 Symbol description

符号	含义
$n$	用户数
$m$	物品数
$u = {1,2, …, n}$	用户ID
$i = {1,2, …, m}$	物品ID
$U = {u}, U = n$	用户集合
$I = {i}, I = m$	物品集合
$R A = {(u, i)}$	所有浏览记录集合
$R T = {(u, i)}$	所有购买记录集合
$r u ⋅ A ∈ R 1 × m$	用户 $u$ 对所有物品的浏览向量
$z u ⋅ A ∈ R 1 × h$	浏览模型编码器生成的隐向量
$r u ⋅ T ∈ R 1 × m$	用户 $u$ 对所有物品的购买向量
$z u ⋅ T ∈ R 1 × h$	购买模型编码器生成的隐向量
$ϕ A, θ A$	浏览模型变分自编码器参数
$ϕ T, θ T$	购买模型变分自编码器参数
$T$	迭代次数
$γ$	学习率
$β$	正则化参数

Tab. 1 Symbol description

符号	含义
$n$	用户数
$m$	物品数
$u = {1,2, …, n}$	用户ID
$i = {1,2, …, m}$	物品ID
$U = {u}, U = n$	用户集合
$I = {i}, I = m$	物品集合
$R A = {(u, i)}$	所有浏览记录集合
$R T = {(u, i)}$	所有购买记录集合
$r u ⋅ A ∈ R 1 × m$	用户 $u$ 对所有物品的浏览向量
$z u ⋅ A ∈ R 1 × h$	浏览模型编码器生成的隐向量
$r u ⋅ T ∈ R 1 × m$	用户 $u$ 对所有物品的购买向量
$z u ⋅ T ∈ R 1 × h$	购买模型编码器生成的隐向量
$ϕ A, θ A$	浏览模型变分自编码器参数
$ϕ T, θ T$	购买模型变分自编码器参数
$T$	迭代次数
$γ$	学习率
$β$	正则化参数

Fig. 1 Model structure of variational autoencoder

Fig. 2 Model of SVAE

Tab. 2 Statistics of three datasets after processing

数据集	用户数	物品数	购买记录数	浏览记录数
ML10M	71 567	10 681	309 317	4 000 024
Netflix	480 189	17 770	4 554 888	39 628 846
Rec15	36 917	9 621	159 429	213 332

Tab. 3 Experimental results of each model on three datasets

数据集	方法	Precision@5	Recall@5	F1@5	NDCG@5	1-call@5
ML10M	BPR	0.068 0±0.000 2	0.091 5±0.000 3	0.065 4±0.000 1	0.093 3±0.000 4	0.283 7±0.002 3
	MFLogLoss	0.073 6±0.000 5	0.099 5±0.001 0	0.070 5±0.000 7	0.101 9±0.000 4	0.303 4±0.001 7
	Multi-VAE	0.074 4±0.000 6	0.099 6±0.000 9	0.070 9±0.000 5	0.102 9±0.000 8	0.306 2±0.002 3
	RoToR	0.087 2±0.000 1	0.123 9±0.000 7	0.085 7±0.000 1	0.123 5±0.000 6	0.356 2±0.000 8
	SVAE（B+P）	0.080 4±0.000 5	0.114 6±0.000 1	0.079 1±0.000 4	0.112 6±0.000 6	0.331 8±0.001 3
	SVAE（B）	0.093 2±0.000 5	0.137 3±0.000 9	0.093 6±0.000 5	0.137 1±0.001 0	0.374 9±0.002 6
Netflix	BPR	0.075 5±0.000 4	0.050 3±0.000 5	0.048 1±0.000 3	0.085 4±0.000 4	0.299 4±0.001 3
	MFLogLoss	0.078 5±0.000 3	0.054 9±0.000 6	0.051 4±0.000 4	0.090 0±0.000 4	0.310 3±0.001 4
	Multi-VAE	0.085 8±0.000 3	0.059 2±0.000 2	0.055 1±0.000 2	0.099 4±0.000 2	0.331 5±0.001 2
	RoToR	0.094 1±0.000 3	0.075 0±0.000 3	0.064 7±0.000 3	0.111 9±0.000 4	0.367 4±0.001 0
	SVAE（B+P）	0.087 7±0.000 3	0.068 3±0.000 6	0.059 5±0.000 4	0.102 9±0.000 4	0.345 8±0.001 6
	SVAE（B）	0.098 2±0.000 6	0.079 3±0.000 6	0.068 5±0.000 5	0.118 3±0.000 7	0.378 7±0.001 6
Rec15	BPR	0.045 7	0.228 6	0.076 2	0.147 3	0.228 6
	MFLogLoss	0.049 0	0.245 1	0.081 7	0.158 6	0.245 1
	Multi-VAE	0.051 0	0.255 2	0.085 1	0.167 0	0.255 2
	RoToR	0.053 4	0.266 9	0.089 0	0.173 4	0.266 9
	SVAE（B+P）	0.049 7	0.248 5	0.082 8	0.157 0	0.248 5
	SVAE（B）	0.053 3	0.266 7	0.088 9	0.177 1	0.266 7

Fig. 3 Experimental results on ML10M with different values of K

Fig. 4 Experimental results on Netflix with different values of K

Fig. 5 Experimental results on Rec15 with different values of K

Fig. 6 Experimental results with different values of hidden dimension h

Tab. 4 Characteristics of three datasets and model with the optimal recommendation results

数据集

购买记录数∶

浏览记录数

Tab. 5 Experimental results of SVAE（alt.） and SVAE（seq.） on three datasets

数据集	方法	Precision@5	NDCG@5
ML10M	SVAE（alt.）	0.093 2±0.000 5	0.137 1±0.001 0
ML10M	SVAE（seq.）	0.090 1±0.000 7	0.132 5±0.001 0
Netflix	SVAE（alt.）	0.098 2±0.000 6	0.118 3±0.000 7
Netflix	SVAE（seq.）	0.101 1±0.000 6	0.121 9±0.000 7
Rec15	SVAE（alt.）	0.053 3	0.177 1
Rec15	SVAE（seq.）	0.053 0	0.176 4

References 33

1	SCHAFER J B， KONSTAN J A， RIEDL J. E-commerce recommendation applications［J］. Data Mining and Knowledge Discovery， 2001， 5（1/2）： 115-153. 10.1007/978-1-4615-1627-9_6
2	OORD A V D， DIELEMAN S， SCHRAUWEN B. Deep content-based music recommendation［C］// Proceeding of the 26th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2013： 2643-2651.
3	LIU J H， DOLAN P， PEDERSEN E R. Personalized news recommendation based on click behavior［C］// Proceedings of the 15th International Conference on Intelligent User Interfaces. New York： ACM， 2010： 31-40. 10.1145/1719970.1719976
4	PAN R， ZHOU Y H， CAO B， et al. One-class collaborative filtering［C］// Proceedings of the 8th IEEE International Conference on Data Mining. Piscataway： IEEE， 2008： 502-511. 10.1109/icdm.2008.16
5	PAN W K， ZHONG H， XU C F， et al. Adaptive Bayesian personalized ranking for heterogeneous implicit feedbacks［J］. Knowledge-Based Systems， 2015， 73： 173-180. 10.1016/j.knosys.2014.09.013
6	QIU H H， LIU Y， GUO G B， et al. BPRH： Bayesian personalized ranking for heterogeneous implicit feedback［J］. Information Sciences， 2018， 453： 80-98. 10.1016/j.ins.2018.04.027
7	DING J T， YU G H， HE X N， et al. Improving implicit recommender systems with view data［C］// Proceedings of the 27th International Joint Conference on Artificial Intelligence. ［S.l.］： IJCAI Organization， 2018： 3343-3349. 10.24963/ijcai.2018/464
8	PAN W K， LIU M S， MING Z. Transfer learning for heterogeneous one-class collaborative filtering［J］. IEEE Intelligent Systems， 2016， 31（4）： 43-49. 10.1109/mis.2016.19
9	PAN W K， YANG Q， CAI W L， et al. Transfer to rank for heterogeneous one-class collaborative filtering［J］. ACM Transactions on Information Systems， 2019， 37（1）： No.10. 10.1145/3243652
10	HE X N， LIAO L Z， ZHANG H W， et al. Neural collaborative filtering［C］// Proceedings of the 26th International Conference on World Wide Web. Republic and Canton of Geneva： International World Wide Web Conferences Steering Committee， 2017： 173-182. 10.1145/3038912.3052569
11	ZHOU M Z， DING Z Y， TANG J L， et al. Micro behaviors： a new perspective in e-commerce recommender systems［C］// Proceedings of the 11th ACM International Conference on Web Search and Data Mining. New York： ACM， 2018： 727-735. 10.1145/3159652.3159671
12	LI Z， ZHAO H K， LIU Q， et al. Learning from history and present： next-item recommendation via discriminatively exploiting user behaviors［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2018： 1734-1743. 10.1145/3219819.3220014
13	SALAKHUTDINOV R R， MNIH A. Probabilistic matrix factorization［C］// Proceeding of the 20th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2007： 1257-1264. 10.1145/1390156.1390267
14	LIANG D W， KRISHNAN R G， HOFFMAN M D， et al. Variational autoencoders for collaborative filtering［C］// Proceedings of the 2018 World Wide Web Conference. Republic and Canton of Geneva： International World Wide Web Conferences Steering Committee， 2018： 689-698. 10.1145/3178876.3186150
15	PAN S J， YANG Q. A survey on transfer learning［J］. IEEE Transactions on Knowledge and Data Engineering， 2010， 22（10）： 1345-1359. 10.1109/tkde.2009.191
16	RENDLE S， FREUDENTHALER C， GANTNER Z， et al. BPR： Bayesian personalized ranking from implicit feedback［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington， VA： AUAI Press， 2009： 452-461.
17	PENG X G， CHEN Y F， DUAN Y C， et al. RBPR： role-based Bayesian personalized ranking for heterogeneous one-class collaborative filtering［C］// Proceedings of the 2016 Workshop on Multi-dimension Information Fusion for Modeling and Personalisation at the 24th ACM Conference on User Modeling， Adaptation and Personalisation. Aachen： CEUR-WS， 2016： 1-4.
18	黄立威，江碧涛，吕守业，等. 基于深度学习的推荐系统研究综述［J］. 计算机学报， 2018， 41（7）： 1619-1647. 10.11897/SP.J.1016.2018.01619
	HUANG L W， JIANG B T， LYU S Y， et al. Survey on deep leaning based recommender systems［J］. Chinese Journal of Computers， 2018， 41（7）：1619-1647. 10.11897/SP.J.1016.2018.01619
19	SEDHAIN S， MENON A K， SANNER S， et al. AutoRec： autoencoders meet collaborative filtering［C］// Proceedings of the 24th International Conference on World Wide Web. New York： ACM， 2015：111-112. 10.1145/2740908.2742726
20	WU Y， DuBOIS C， ZHENG A X， et al. Collaborative denoising auto-encoders for top-N recommender systems［C］// Proceedings of the 9th ACM International Conference on Web Search and Data Mining. New York： ACM， 2016： 153-162. 10.1145/2835776.2835837
21	LEE W， SONG K， MOON I C. Augmented variational autoencoders for collaborative filtering with auxiliary information［C］// Proceedings of the 2017 ACM International Conference on Information and Knowledge Management. New York： ACM， 2017： 1139-1148. 10.1145/3132847.3132972
22	PANDEY G， DUKKIPATI A. Variational methods for conditional multimodal deep learning［C］// Proceedings of the 2017 International Joint Conference on Neural Networks. Piscataway： IEEE， 2017： 308-315. 10.1109/ijcnn.2017.7965870
23	SUZUKI M， NAKAYAMA K， MATSUO Y. Joint multimodal learning with deep generative models［EB/OL］. （2016-11-07）［2021-04-03］..
24	CUI K N， CHEN X， YAO J C， et al. Variational collaborative learning for user probabilistic representation［EB/OL］. （2018-09-22）［2021-04-03］.. 10.1109/icde51399.2021.00051
25	IQBAL M， ARYAFAR K， ANDERTON T. Style conditioned recommendations［C］// Proceedings of the 13th ACM Conference on Recommender Systems. New York： ACM， 2019： 128-136. 10.1145/3298689.3347007
26	BLEI D M， KUCUKELBIR A， McAULIFFE J D. Variational inference： a review for statisticians［EB/OL］. （2018-05-09）［2021-04-03］.. 10.1080/01621459.2017.1285773
27	SACHDEVA N， MANCO G， RITACCO E， et al. Sequential variational autoencoders for collaborative filtering［C］// Proceedings of the 12th ACM International Conference on Web Search and Data Mining. New York： ACM， 2019： 600-608. 10.1145/3289600.3291007
28	翟正利，梁振明，周炜，等. 变分自编码器模型综述［J］. 计算机工程与应用， 2019， 55（3）：1-9. 10.3778/j.issn.1002-8331.1810-0284
	ZHAI Z L， LIANG Z M， ZHOU W， et al. Research overview of variational auto-encoders models［J］. Computer Engineering and Applications， 2019， 55（3）：1-9. 10.3778/j.issn.1002-8331.1810-0284
29	EBESU T， SHEN B， FANG Y. Collaborative memory network for recommendation systems［C］// Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2018： 515-524. 10.1145/3209978.3209991
30	RICCI F， ROKACH L， SHAPIRA B. Recommender Systems Handbook［M］. 2 ed. Boston： Springer， 2015： 176-199. 10.1007/978-1-4899-7637-6_1
31	McMAHAN B， MOORE E， RAMAGE D， et al. Communication-efficient learning of deep networks from decentralized data［C］// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York： JMLR， 2017： 1273-1282.
32	YANG Q， LIU Y， CHEN T J， et al. Federated machine learning： Concept and applications［J］. ACM Transactions on Intelligent Systems and Technology， 2019， 10（2）： No.12. 10.1145/3298981
33	熊平，朱天清，王晓峰. 差分隐私保护及其应用［J］. 计算机学报， 2014， 37（1）： 101-122. 10.3724/SP.J.1016.2014.00101
	XIONG P， ZHU T Q， WANG X F. A survey on differential privacy and applications［J］. Chinese Journal of Computers， 2014， 37（1）： 101-122. 10.3724/SP.J.1016.2014.00101

[1]	Tingjie TANG, Jiajin HUANG, Jin QIN. Session-based recommendation with graph auxiliary learning [J]. Journal of Computer Applications, 2024, 44(9): 2711-2718.
[2]	Feiyu ZHAI, Handa MA. Hybrid classical-quantum classification model based on DenseNet [J]. Journal of Computer Applications, 2024, 44(6): 1905-1910.
[3]	Wangjun SHI, Jing WANG, Xiaojun NING, Youfang LIN. Sleep stage classification model by meta transfer learning in few-shot scenarios [J]. Journal of Computer Applications, 2024, 44(5): 1445-1451.
[4]	Hongtian LI, Xinhao SHI, Weiguo PAN, Cheng XU, Bingxin XU, Jiazheng YUAN. Few-shot object detection via fusing multi-scale and attention mechanism [J]. Journal of Computer Applications, 2024, 44(5): 1437-1444.
[5]	Haoran WANG, Dan YU, Yuli YANG, Yao MA, Yongle CHEN. Domain transfer intrusion detection method for unknown attacks on industrial control systems [J]. Journal of Computer Applications, 2024, 44(4): 1158-1165.
[6]	Zongyu LI, Siwei QIANG, Xiaobo GUO, Zhenfeng ZHU. Re-weighted adversarial variational autoencoder and its application in industrial causal effect estimation [J]. Journal of Computer Applications, 2024, 44(4): 1099-1106.
[7]	Li ZENG, Jingru YANG, Gang HUANG, Xiang JING, Chaoran LUO. Survey on hypergraph application methods： issues， advances， and challenges [J]. Journal of Computer Applications, 2024, 44(11): 3315-3326.
[8]	Qiujie LIU, Yuan WAN, Jie WU. Deep bi-modal source domain symmetrical transfer learning for cross-modal retrieval [J]. Journal of Computer Applications, 2024, 44(1): 24-31.
[9]	Kezheng CHEN, Xiaoran GUO, Yong ZHONG, Zhenping LI. Relation extraction method based on negative training and transfer learning [J]. Journal of Computer Applications, 2023, 43(8): 2426-2430.
[10]	Zexi JIN, Lei LI, Ji LIU. Transfer learning model based on improved domain separation network [J]. Journal of Computer Applications, 2023, 43(8): 2382-2389.
[11]	Yuan LIU, Yongquan DONG, Rui JIA, Haolin YANG. Hierarchical and phased attention network model for personalized course recommendation [J]. Journal of Computer Applications, 2023, 43(8): 2358-2363.
[12]	Bona XUAN, Jin LI, Yafei SONG, Zexuan MA. Malicious code classification method based on improved MobileNetV2 [J]. Journal of Computer Applications, 2023, 43(7): 2217-2225.
[13]	Huibin ZHANG, Liping FENG, Yaojun HAO, Yining WANG. Ancient mural dynasty identification based on attention mechanism and transfer learning [J]. Journal of Computer Applications, 2023, 43(6): 1826-1832.
[14]	Hao SUN, Jian CAO, Haisheng LI, Dianhui MAO. Session-based recommendation model based on enhanced capsule network [J]. Journal of Computer Applications, 2023, 43(4): 1043-1049.
[15]	Qing JIA, Laihua WANG, Weisheng WANG. Anomaly detection in video via independently recurrent neural network and variational autoencoder network [J]. Journal of Computer Applications, 2023, 43(2): 507-513.

Staged variational autoencoder for heterogeneous one-class collaborative filtering

面向异构单类协同过滤的阶段式变分自编码器

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 33

Related Articles 15

Recommended Articles

Metrics