Sequential recommendation based on hierarchical filter and temporal convolution enhanced self-attention network

doi:10.11772/j.issn.1001-9081.2023091352

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (10): 3090-3096.DOI: 10.11772/j.issn.1001-9081.2023091352

• Data science and technology • Previous Articles Next Articles

Sequential recommendation based on hierarchical filter and temporal convolution enhanced self-attention network

Xingyao YANG¹(), Hongtao SHEN¹, Zulian ZHANG², Jiong YU¹, Jiaying CHEN¹, Dongxiao WANG¹

^1.School of Software，Xinjiang University，Urumqi Xinjiang 830091，China
^2.Xinjiang Xinnong Network Information Center，Meteorological Bureau of Xinjiang Uygur Autonomous Region，Urumqi Xinjiang 830002，China

Received:2023-10-07 Revised:2023-12-04 Accepted:2023-12-05 Online:2023-12-20 Published:2024-10-10
Contact: Xingyao YANG
About author:SHEN Hongtao， born in 2001， M. S. candidate. His research interests include recommender system.
ZHANG Zulian， born in 1984， M. S.， senior engineer. Her research interests include numerical prediction， information retrieval.
YU Jiong， born in 1964， Ph. D.， professor. His research interests include grid computing， parallel computing.
CHEN Jiaying， born in 1988， Ph. D.， associate professor. Her research interests include recommender system， social network， data mining.
WANG Dongxiao， born in 1970， experimentalist. His research interests include cyber security.
Supported by:
Natural Science Foundation of Xinjiang Uygur Autonomous Region(2023D01C17);National Natural Science Foundation of China(62262064);Xinjiang Meteorological Bureau Guidance Project(YD202212);Xinjiang Uygur Autonomous Region Science and Technology Program(2023D4012)

基于层级过滤器和时间卷积增强自注意力网络的序列推荐

杨兴耀¹(), 沈洪涛¹, 张祖莲², 于炯¹, 陈嘉颖¹, 王东晓¹

^1.新疆大学软件学院，乌鲁木齐 830091
^2.新疆维吾尔自治区气象局新疆兴农网信息中心，乌鲁木齐 830002

通讯作者: 杨兴耀
作者简介:杨兴耀（1984—），男，湖北襄阳人，副教授，博士，CCF会员，主要研究方向：推荐系统、大数据、信任计算 yangxy@xju.edu.cn
沈洪涛（2001—），男，安徽淮南人，硕士研究生，主要研究方向：推荐系统
张祖莲（1984—），女，湖北襄阳人，高级工程师，硕士，主要研究方向：数值预报、信息检索
于炯（1964—），男，新疆乌鲁木齐人，教授，博士，CCF会员，主要研究方向：网格计算、并行计算
陈嘉颖（1988—），女，新疆沙湾人，副教授，博士，CCF会员，主要研究方向：推荐系统、社交网络、数据挖掘
王东晓（1970—），男，黑龙江绥化人，实验师，主要研究方向：网络安全。
基金资助:
新疆维吾尔自治区自然科学基金资助项目(2023D01C17);国家自然科学基金资助项目(62262064);新疆气象局引导项目(YD202212);新疆维吾尔自治区科技计划项目(2023D4012)

Abstract

Abstract:

Aiming at the problem of noise arising from user’s unexpected interactions in practical recommendation scenarios and the challenge of capturing short-term demand biases due to the dispersed attention in self-attention mechanism， a model namely FTARec （sequential Recommendation based on hierarchical Filter and Temporal convolution enhanced self-Attention network） was proposed. Firstly， hierarchical filter was used to filter noise in the original data. Then， user embeddings were obtained by combining temporal convolution enhanced self-attention networks with decoupled hybrid location encoding. The deficiencies in modeling short-term dependencies among items were supplemented by enhancing the self-attention network with temporal convolution in this process. Finally， contrastive learning was incorporated to refine user embeddings and predictions were made based on the final user embeddings. Compared to existing sequential recommendation models such as the Self-Attentive Sequential Recommendation （SASRec） and the Filter-enhanced Multi-Layer Perceptron approach for sequential Recommendation （FMLP-Rec）， FTARec achieves higher Hit Rate （HR） and Normalized Discounted Cumulative Gain （NDCG） on three publicly available datasets： Beauty， Clothing， and Sports. Compared with the suboptimal DuoRec， FTARec has the HR@10 increased by 7.91%， 13.27%， 12.84%， and the NDCG@10 increased by 5.52%， 8.33%， 9.88%， respectively， verifying the effectiveness of the proposed model.

Key words: self-attention mechanism, filtering algorithm, temporal convolutional network, sequential recommendation, contrastive learning

摘要：

针对实际推荐场景中用户意外交互产生的噪声问题，以及自注意力机制中注意力分布分散导致用户短期需求偏移难以捕获的问题，提出一种基于层级过滤器和时间卷积增强自注意力网络的序列推荐（FTARec）模型。首先，通过层级过滤器过滤原始数据中的噪声；其次，结合时间卷积增强自注意力网络和解耦混合位置编码获取用户嵌入，该过程通过时间卷积增强补充自注意力网络在项目短期依赖建模上的不足；最后，结合对比学习改善用户嵌入，并根据最终用户嵌入进行预测。相较于自注意力序列推荐（SASRec）、过滤增强的多层感知器序列推荐方法（FMLP-Rec）等现有序列推荐模型，FTARec在3个公开数据集Beauty、Clothing和Sports上取得了更高的命中率（HR）和归一化折损累计增益（NDCG），相较于次优的DuoRec，HR@10分别提高了7.91%、13.27%和12.84%，NDCG@10分别提高了5.52%、8.33%和9.88%，验证了所提模型的有效性。

关键词: 自注意力机制, 过滤算法, 时间卷积网络, 序列推荐, 对比学习

CLC Number:

TP391.3

Xingyao YANG, Hongtao SHEN, Zulian ZHANG, Jiong YU, Jiaying CHEN, Dongxiao WANG. Sequential recommendation based on hierarchical filter and temporal convolution enhanced self-attention network[J]. Journal of Computer Applications, 2024, 44(10): 3090-3096.

杨兴耀, 沈洪涛, 张祖莲, 于炯, 陈嘉颖, 王东晓. 基于层级过滤器和时间卷积增强自注意力网络的序列推荐[J]. 《计算机应用》唯一官方网站, 2024, 44(10): 3090-3096.

Figures/Tables 10

References 26

1	黄立威，江碧涛，吕守业，等.基于深度学习的推荐系统研究综述［J］.计算机学报，2018，41（7）：1619-1647.
	HUANG L W， JIANG B T， LYU S Y， et al. Survey on deep learning based recommender systems ［J］. Chinese Journal of Computers， 2018， 41（7）： 1619-1647.
2	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2017： 6000-6010.
3	KANG W-C， McAULEY J. Self-attentive sequential recommendation ［C］// Proceedings of the 2018 IEEE International Conference on Data Mining. Piscataway： IEEE， 2018： 197-206.
4	SUN F， LIU J， WU J， et al. BERT4Rec： sequential recommendation with bidirectional encoder representations from Transformer ［C］// Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York： ACM， 2019： 1441-1450.
5	FAN X， LIU Z， LIAN J， et al. Lighter and better： low-rank decomposed self-attention networks for next-item recommendation ［C］// Proceedings of the 44th International ACM SIGIR Conference on Research and Development of Information Retrieval. New York： ACM， 2021： 1733-1737.
6	LI W， GOU J， FAN Z. Session-based recommendation with temporal convolutional network to balance numerical gaps ［J］. Neurocomputing， 2022， 493： 166-175.
7	BAI S， KOLTER J Z， KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling ［EB/OL］. ［2023-08-23］. .
8	FENG S， LI X， ZENG Y， et al. Personalized ranking metric embedding for next new POI recommendation ［C］// Proceedings of the 24th International Joint Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2015： 2069-2075.
9	HIDASI B， KARATZOGLOU A， BALTRUNAS L， et al. Session-based recommendations with recurrent neural networks ［EB/OL］. ［2021-06-30］. .
10	TANG J， WANG K. Personalized top-N sequential recommendation via convolutional sequence embedding ［C］// Proceedings of the 11th ACM International Conference on Web Search and Data Mining. New York： ACM， 2018： 565-573.
11	XIE X， SUN F， LIU Z， et al. Contrastive learning for sequential recommendation ［C］// Proceedings of the 2022 IEEE 38th International Conference on Data Engineering. Piscataway： IEEE， 2022： 1259-1273.
12	QIU R， HUAN Z， YIN， H， et al. Contrastive learning for representation degeneration problem in sequential recommendation ［C］// Proceedings of the 15th ACM International Conference on Web Search and Data Mining. New York： ACM， 2022： 813-823.
13	YANG X-Y， XU F， YU J， et al. Graph neural network-guided contrastive learning for sequential recommendation ［J］. Sensors， 2023， 23（12）： 5572.
14	夏玉杰，时永鹏，高雅，等.降低滤波器组多载波信号峰均比的边信息嵌入选择性映射方法［J］.计算机应用，2021，41（5）：1425-1431.
	XIA Y J， SHI Y P， GAO Y， et al. Selected mapping method with embedded side information to reduce PAPR of FBMC signals ［J］. Journal of Computer Applications， 2021， 41（5）： 1425-1431.
15	张少东，杨兴耀，于炯，等.基于对比学习和傅里叶变换的序列推荐算法［J］.电子科技大学学报，2023，52（4）：610-619.
	ZHANG S D， YANG X Y， YU J， et al. Sequence recommendation based on contrast learning and Fourier transform ［J］. Journal of University of Electronic Science and Technology of China， 2023， 52（4）： 610-619.
16	CHEN B， LUO W， LUO D. Identification of audio processing operations based on convolutional neural network ［C］// Proceedings of the 6th ACM Workshop on Information Hiding and Multimedia Security. New York： ACM， 2018： 73-77.
17	ZHOU K， YU H， ZHAO W X， et al. Filter-enhanced MLP is all you need for sequential recommendation ［C］// Proceedings of the ACM Web Conference 2022. New York： ACM， 2022： 2388-2399.
18	DU X， YUAN H， ZHAO P， et al. Contrastive enhanced slide filter mixer for sequential recommendation ［C］// Proceedings of the 2023 IEEE 39th International Conference on Data Engineering. Piscataway： IEEE， 2023： 2673-2685.
19	ZHOU K， WANG， H， ZHAO W X， et al. S3-Rec： self-supervised learning for sequential recommendation with mutual information maximization ［C］// Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York： ACM， 2020： 1893-1902.
20	BIAN Y， HUANG J， CAI X， et al. On attention redundancy： a comprehensive study ［C］// Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics： Human Language Technologies. Stroudsburg： ACL， 2021： 930-945.
21	BA J L， KIROS J R， HINTON G E. Layer normalization ［EB/OL］. ［2023-09-01］. .
22	SRIVASTAVA N， HINTON G， KRIZHEVSKY A， et.al. Dropout： a simple way to prevent neural networks from overfitting ［J］. The Journal of Machine Learning Research， 2014， 15： 1929-1958.
23	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778.
24	刘华锋，景丽萍，于剑.融合社交信息的矩阵分解推荐方法研究综述［J］.软件学报，2018，29（2）：340-362.
	LIU H F， JING L P， YU J. Survey of matrix factorization based recommendation methods by integrating social information ［J］. Journal of Software， 2018， 29（2）： 340-362.
25	RENDLE S， FREUDENTHALER C， GANTER Z， et al. BPR： Bayesian personalized ranking from implicit feedback ［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington， VA： AUAI Press， 2009： 452-461.
26	ZHAO W X， MU S， HOU Y， et al. RecBole： towards a unified， comprehensive and efficient framework for recommendation algorithms ［C］// Proceedings of the 30th ACM International Conference on Information & Knowledge Management. New York： ACM， 2021： 4653-4664.

子数据集	用户数	项目数	交互数	稀疏性/%
Beauty	22 363	12 101	198 502	0.07
Clothing	39 387	23 033	278 677	0.30
Sports	35 598	18 357	296 337	0.05

子数据集	用户数	项目数	交互数	稀疏性/%
Beauty	22 363	12 101	198 502	0.07
Clothing	39 387	23 033	278 677	0.30
Sports	35 598	18 357	296 337	0.05

模型	HR@5	HR@10	NDCG@5	NDCG@10
BPR-MF	0.012 0	0.029 9	0.004 0	0.005 3
GRU4Rec	0.016 4	0.036 5	0.008 6	0.014 2
SASRec	0.036 5	0.062 7	0.023 6	0.028 1
BERT4Rec	0.019 3	0.040 1	0.018 7	0.025 4
FMLP-Rec	0.039 8	0.063 2	0.025 8	0.033 3
CL4Rec	0.040 1	0.068 3	0.022 3	0.031 7
DuoRec	0.054 7	0.083 4	0.034 6	0.043 5
FTARec	0.058 1	0.090 0	0.035 7	0.045 9

模型	HR@5	HR@10	NDCG@5	NDCG@10
BPR-MF	0.012 0	0.029 9	0.004 0	0.005 3
GRU4Rec	0.016 4	0.036 5	0.008 6	0.014 2
SASRec	0.036 5	0.062 7	0.023 6	0.028 1
BERT4Rec	0.019 3	0.040 1	0.018 7	0.025 4
FMLP-Rec	0.039 8	0.063 2	0.025 8	0.033 3
CL4Rec	0.040 1	0.068 3	0.022 3	0.031 7
DuoRec	0.054 7	0.083 4	0.034 6	0.043 5
FTARec	0.058 1	0.090 0	0.035 7	0.045 9

模型	HR@5	HR@10	NDCG@5	NDCG@10
BPR-MF	0.006 7	0.009 4	0.005 2	0.006 9
GRU4Rec	0.009 5	0.016 5	0.006 1	0.008 3
SASRec	0.016 8	0.027 2	0.009 1	0.012 4
BERT4Rec	0.012 5	0.020 8	0.007 5	0.010 2
FMLP-Rec	0.012 6	0.020 6	0.008 2	0.010 7
CL4Rec	0.016 8	0.026 6	0.009 0	0.012 1
DuoRec	0.018 8	0.029 4	0.010 8	0.014 4
FTARec	0.020 8	0.033 3	0.011 6	0.015 6

Sequential recommendation based on hierarchical filter and temporal convolution enhanced self-attention network

基于层级过滤器和时间卷积增强自注意力网络的序列推荐

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 26

Related Articles 15

Recommended Articles

Metrics

模型	HR@5	HR@10	NDCG@5	NDCG@10
BPR-MF	0.009 2	0.018 8	0.004 0	0.005 1
GRU4Rec	0.013 7	0.027 4	0.009 6	0.013 7
SASRec	0.021 8	0.033 6	0.012 7	0.016 9
BERT4Rec	0.017 6	0.032 6	0.010 5	0.015 3
FMLP-Rec	0.021 8	0.034 4	0.014 4	0.018 5
CL4Rec	0.022 7	0.037 4	0.012 9	0.019 7
DuoRec	0.030 9	0.047 5	0.019 0	0.024 3
FTARec	0.034 0	0.053 6	0.020 5	0.026 7

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[3]	Xingyao YANG, Yu CHEN, Jiong YU, Zulian ZHANG, Jiaying CHEN, Dongxiao WANG. Recommendation model combining self-features and contrastive learning [J]. Journal of Computer Applications, 2024, 44(9): 2704-2710.
[4]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[5]	Zexin XU, Lei YANG, Kangshun LI. Shorter long-sequence time series forecasting model [J]. Journal of Computer Applications, 2024, 44(6): 1824-1831.
[6]	Xiaoxia JIANG, Ruizhang HUANG, Ruina BAI, Lina REN, Yanping CHEN. Deep event clustering method based on event representation and contrastive learning [J]. Journal of Computer Applications, 2024, 44(6): 1734-1742.
[7]	Yue LIU, Fang LIU, Aoyun WU, Qiuyue CHAI, Tianxiao WANG. 3D object detection network based on self-attention mechanism and graph convolution [J]. Journal of Computer Applications, 2024, 44(6): 1972-1977.
[8]	Juxiang ZHOU, Jinsheng LIU, Jianhou GAN, Di WU, Zijie LI. Classroom speech emotion recognition method based on multi-scale temporal-aware network [J]. Journal of Computer Applications, 2024, 44(5): 1636-1643.
[9]	Jiong WANG, Taotao TANG, Caiyan JIA. PAGCL： positive augmentation graph contrastive learning recommendation method without negative sampling [J]. Journal of Computer Applications, 2024, 44(5): 1485-1492.
[10]	Jie GUO, Jiayu LIN, Zuhong LIANG, Xiaobo LUO, Haitao SUN. Recommendation method based on knowledge‑awareness and cross-level contrastive learning [J]. Journal of Computer Applications, 2024, 44(4): 1121-1127.
[11]	Rong HUANG, Junjie SONG, Shubo ZHOU, Hao LIU. Image aesthetic quality evaluation method based on self-supervised vision Transformer [J]. Journal of Computer Applications, 2024, 44(4): 1269-1276.
[12]	Xinran LUO, Tianrui LI, Zhen JIA. Chinese medical named entity recognition based on self-attention mechanism and lexicon enhancement [J]. Journal of Computer Applications, 2024, 44(2): 385-392.
[13]	Weichao DANG, Lei ZHANG, Gaimei GAO, Chunxia LIU. Weakly supervised action localization method with snippet contrastive learning [J]. Journal of Computer Applications, 2024, 44(2): 548-555.
[14]	Ziqi HUANG, Jianpeng HU. Entity category enhanced nested named entity recognition in automotive domain [J]. Journal of Computer Applications, 2024, 44(2): 377-384.
[15]	Liqing QIU, Xiaopan SU. Personalized multi-layer interest extraction click-through rate prediction model [J]. Journal of Computer Applications, 2024, 44(11): 3411-3418.