《计算机应用》唯一官方网站 ›› 2024, Vol. 44 ›› Issue (11): 3411-3418.DOI: 10.11772/j.issn.1001-9081.2023111681
收稿日期:
2023-12-05
修回日期:
2024-05-01
接受日期:
2024-05-10
发布日期:
2024-05-30
出版日期:
2024-11-10
通讯作者:
苏小盼
作者简介:
仇丽青(1978—),女,山东德州人,副教授,博士,主要研究方向:社交网络、推荐系统
基金资助:
Received:
2023-12-05
Revised:
2024-05-01
Accepted:
2024-05-10
Online:
2024-05-30
Published:
2024-11-10
Contact:
Xiaopan SU
About author:
QIU Liqing, born in 1978, Ph. D., associate professor. Her research interests include social network, recommender system.
Supported by:
摘要:
目前,点击率(CTR)预测最常用的方法是利用特征交互技术提取兴趣特征,但这些方法大多忽视了用户与项目之间的内在联系,同时也未能充分发掘项目间所蕴含的用户潜在兴趣。针对该问题,提出一种个性化多层兴趣提取点击率预测模型(PMIC),旨在从不同角度深入挖掘用户在同一时间内展现的多层兴趣。首先,采用召回匹配的方法,从项目学习模块和用户学习模块两个角度学习并建模用户与项目之间的联系,捕捉用户多样化的兴趣;其次,利用多头自注意力机制,在项目学习模块中提取同一时间内隐含的多个潜在兴趣;最后,通过内积计算,进一步细化和加强用户与项目之间的特征表达。在多个公共数据集上的实验结果表明,与基线模型相比,PMIC的受试者特征工作曲线下面积(AUC)最少提高了2.3%。
中图分类号:
仇丽青, 苏小盼. 个性化多层兴趣提取点击率预测模型[J]. 计算机应用, 2024, 44(11): 3411-3418.
Liqing QIU, Xiaopan SU. Personalized multi-layer interest extraction click-through rate prediction model[J]. Journal of Computer Applications, 2024, 44(11): 3411-3418.
数据集 | 用户数 | 项目数 | 产品类别数 | 样本数 |
---|---|---|---|---|
Taobao | 1 141 729 | 846 811 | 12 960 | 1 366 056 |
MovieLens | 943 | 1 447 | 19 | 108 866 |
Book | 603 668 | 367 982 | 1 600 | 603 668 |
Electronics | 192 403 | 63 001 | 801 | 192 403 |
Auazu | 4 904 | 4 737 | 26 | 6 865 066 |
表1 实验数据集
Tab. 1 Experimental datasets
数据集 | 用户数 | 项目数 | 产品类别数 | 样本数 |
---|---|---|---|---|
Taobao | 1 141 729 | 846 811 | 12 960 | 1 366 056 |
MovieLens | 943 | 1 447 | 19 | 108 866 |
Book | 603 668 | 367 982 | 1 600 | 603 668 |
Electronics | 192 403 | 63 001 | 801 | 192 403 |
Auazu | 4 904 | 4 737 | 26 | 6 865 066 |
模型 | Taobao | MovieLens | Book | ||||||
---|---|---|---|---|---|---|---|---|---|
GAUC | AUC | RelaImpr | GAUC | AUC | RelaImpr | GAUC | AUC | RelaImpr | |
BaseModel | 0.625 7 | 0.627 9 | 0.000 0 | 0.599 8 | 0.619 7 | 0.000 0 | 0.628 9 | 0.631 5 | 0.000 0 |
Wide & Deep | 0.632 1 | 0.633 4 | 0.050 9 | 0.619 9 | 0.621 0 | 0.201 4 | 0.614 9 | 0.634 7 | 0.108 6 |
PNN | 0.634 5 | 0.632 4 | 0.070 0 | 0.622 3 | 0.625 5 | 0.225 4 | 0.648 4 | 0.648 9 | 0.151 2 |
DIN | 0.652 1 | 0.654 2 | 0.210 0 | 0.631 0 | 0.631 2 | 0.312 6 | 0.651 3 | 0.651 4 | 0.173 7 |
DIEN | 0.654 1 | 0.656 4 | 0.225 9 | 0.631 8 | 0.632 0 | 0.320 6 | 0.661 2 | 0.662 4 | 0.250 5 |
DMIN | 0.693 4 | 0.695 7 | 0.538 5 | 0.645 1 | 0.652 3 | 0.453 9 | 0.694 8 | 0.695 5 | 0.511 2 |
DMR | 0.684 7 | 0.688 7 | 0.469 3 | 0.635 2 | 0.643 8 | 0.354 7 | 0.692 4 | 0.696 4 | 0.492 6 |
FAT | 0.684 9 | 0.697 8 | 0.470 9 | 0.631 0 | 0.632 5 | 0.312 6 | 0.662 4 | 0.663 7 | 0.259 8 |
NIIN | 0.685 2 | 0.698 9 | 0.473 3 | 0.632 0 | 0.642 5 | 0.322 6 | 0.665 4 | 0.664 7 | 0.283 1 |
PMIC | 0.736 8 | 0.742 0 | 0.883 8 | 0.639 8 | 0.667 5 | 0.400 8 | 0.705 9 | 0.712 4 | 0.597 3 |
模型 | Electronics | Auazu | |||||||
GAUC | AUC | RelaImpr | GAUC | AUC | RelaImpr | ||||
BaseModel | 0.600 1 | 0.619 9 | 0.000 0 | 0.626 7 | 0.631 0 | 0.000 0 | |||
Wide & Deep | 0.620 0 | 0.621 5 | 0.198 8 | 0.638 1 | 0.640 1 | 0.089 9 | |||
PNN | 0.624 3 | 0.627 5 | 0.241 7 | 0.638 5 | 0.642 3 | 0.093 1 | |||
DIN | 0.632 3 | 0.634 2 | 0.321 6 | 0.657 9 | 0.658 4 | 0.246 2 | |||
DIEN | 0.640 1 | 0.642 2 | 0.399 6 | 0.658 6 | 0.659 3 | 0.251 7 | |||
DMIN | 0.645 5 | 0.652 4 | 0.453 5 | 0.689 4 | 0.699 1 | 0.494 8 | |||
DMR | 0.638 2 | 0.644 0 | 0.380 6 | 0.688 7 | 0.699 9 | 0.489 3 | |||
FAT | 0.639 0 | 0.645 1 | 0.388 6 | 0.687 2 | 0.688 4 | 0.477 5 | |||
NIIN | 0.632 4 | 0.644 3 | 0.322 6 | 0.687 5 | 0.680 0 | 0.479 8 | |||
PMIC | 0.663 5 | 0.672 1 | 0.633 3 | 0.743 5 | 0.752 3 | 0.936 0 |
表2 模型对比实验结果
Tab.2 Model comparison experimental results
模型 | Taobao | MovieLens | Book | ||||||
---|---|---|---|---|---|---|---|---|---|
GAUC | AUC | RelaImpr | GAUC | AUC | RelaImpr | GAUC | AUC | RelaImpr | |
BaseModel | 0.625 7 | 0.627 9 | 0.000 0 | 0.599 8 | 0.619 7 | 0.000 0 | 0.628 9 | 0.631 5 | 0.000 0 |
Wide & Deep | 0.632 1 | 0.633 4 | 0.050 9 | 0.619 9 | 0.621 0 | 0.201 4 | 0.614 9 | 0.634 7 | 0.108 6 |
PNN | 0.634 5 | 0.632 4 | 0.070 0 | 0.622 3 | 0.625 5 | 0.225 4 | 0.648 4 | 0.648 9 | 0.151 2 |
DIN | 0.652 1 | 0.654 2 | 0.210 0 | 0.631 0 | 0.631 2 | 0.312 6 | 0.651 3 | 0.651 4 | 0.173 7 |
DIEN | 0.654 1 | 0.656 4 | 0.225 9 | 0.631 8 | 0.632 0 | 0.320 6 | 0.661 2 | 0.662 4 | 0.250 5 |
DMIN | 0.693 4 | 0.695 7 | 0.538 5 | 0.645 1 | 0.652 3 | 0.453 9 | 0.694 8 | 0.695 5 | 0.511 2 |
DMR | 0.684 7 | 0.688 7 | 0.469 3 | 0.635 2 | 0.643 8 | 0.354 7 | 0.692 4 | 0.696 4 | 0.492 6 |
FAT | 0.684 9 | 0.697 8 | 0.470 9 | 0.631 0 | 0.632 5 | 0.312 6 | 0.662 4 | 0.663 7 | 0.259 8 |
NIIN | 0.685 2 | 0.698 9 | 0.473 3 | 0.632 0 | 0.642 5 | 0.322 6 | 0.665 4 | 0.664 7 | 0.283 1 |
PMIC | 0.736 8 | 0.742 0 | 0.883 8 | 0.639 8 | 0.667 5 | 0.400 8 | 0.705 9 | 0.712 4 | 0.597 3 |
模型 | Electronics | Auazu | |||||||
GAUC | AUC | RelaImpr | GAUC | AUC | RelaImpr | ||||
BaseModel | 0.600 1 | 0.619 9 | 0.000 0 | 0.626 7 | 0.631 0 | 0.000 0 | |||
Wide & Deep | 0.620 0 | 0.621 5 | 0.198 8 | 0.638 1 | 0.640 1 | 0.089 9 | |||
PNN | 0.624 3 | 0.627 5 | 0.241 7 | 0.638 5 | 0.642 3 | 0.093 1 | |||
DIN | 0.632 3 | 0.634 2 | 0.321 6 | 0.657 9 | 0.658 4 | 0.246 2 | |||
DIEN | 0.640 1 | 0.642 2 | 0.399 6 | 0.658 6 | 0.659 3 | 0.251 7 | |||
DMIN | 0.645 5 | 0.652 4 | 0.453 5 | 0.689 4 | 0.699 1 | 0.494 8 | |||
DMR | 0.638 2 | 0.644 0 | 0.380 6 | 0.688 7 | 0.699 9 | 0.489 3 | |||
FAT | 0.639 0 | 0.645 1 | 0.388 6 | 0.687 2 | 0.688 4 | 0.477 5 | |||
NIIN | 0.632 4 | 0.644 3 | 0.322 6 | 0.687 5 | 0.680 0 | 0.479 8 | |||
PMIC | 0.663 5 | 0.672 1 | 0.633 3 | 0.743 5 | 0.752 3 | 0.936 0 |
模型 | Taobao | MovieLens | Book | Electronics | Auazu | |||||
---|---|---|---|---|---|---|---|---|---|---|
GAUC | AUC | GAUC | AUC | GAUC | AUC | GAUC | AUC | GAUC | AUC | |
PMIC-type | 0.712 4 | 0.718 9 | 0.637 7 | 0.649 6 | 0.701 7 | 0.702 3 | 0.637 9 | 0.650 1 | 0.720 1 | 0.723 4 |
PMIC-Aux | 0.685 0 | 0.688 9 | 0.636 9 | 0.644 1 | 0.693 3 | 0.697 4 | 0.625 1 | 0.633 3 | 0.693 1 | 0.698 4 |
PMIC-selfattention-Aux | 0.653 0 | 0.657 8 | 0.601 7 | 0.602 4 | 0.664 7 | 0.665 8 | 0.612 5 | 0.621 3 | 0.660 1 | 0.673 2 |
PMIC | 0.736 8 | 0.742 0 | 0.639 8 | 0.667 5 | 0.705 9 | 0.712 4 | 0.642 1 | 0.645 5 | 0.743 5 | 0.745 1 |
表3 消融实验结果
Tab.3 Ablation experimental results
模型 | Taobao | MovieLens | Book | Electronics | Auazu | |||||
---|---|---|---|---|---|---|---|---|---|---|
GAUC | AUC | GAUC | AUC | GAUC | AUC | GAUC | AUC | GAUC | AUC | |
PMIC-type | 0.712 4 | 0.718 9 | 0.637 7 | 0.649 6 | 0.701 7 | 0.702 3 | 0.637 9 | 0.650 1 | 0.720 1 | 0.723 4 |
PMIC-Aux | 0.685 0 | 0.688 9 | 0.636 9 | 0.644 1 | 0.693 3 | 0.697 4 | 0.625 1 | 0.633 3 | 0.693 1 | 0.698 4 |
PMIC-selfattention-Aux | 0.653 0 | 0.657 8 | 0.601 7 | 0.602 4 | 0.664 7 | 0.665 8 | 0.612 5 | 0.621 3 | 0.660 1 | 0.673 2 |
PMIC | 0.736 8 | 0.742 0 | 0.639 8 | 0.667 5 | 0.705 9 | 0.712 4 | 0.642 1 | 0.645 5 | 0.743 5 | 0.745 1 |
1 | 罗凯耀,孙伟智,唐云. 融合注意力机制的广告点击率预测模型[J].微型电脑应用, 2023, 39(5):36-38. |
LUO K Y, SUN W Z, TANG Y. Advertising click-through rate prediction model based on attention mechanism[J]. Microcomputer Applications, 2023, 39(5): 36-38. | |
2 | 龚雪鸾,陈艳姣,王帅.在线广告点击率预测方法的研究综述[J].中文信息学报,2023,37(4):1-17. |
GONG X L, CHEN Y J, WANG S. A review of click-through rate prediction approaches for online advertising[J]. Journal of Chinese Information Processing, 2023, 37(4):1-17. | |
3 | McMAHAN H B, HOLT G, SCULLEY D, et al. Ad click prediction: a view from the trenches[C]// Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2013: 1222-1230. |
4 | GAI K, ZHU X, LI H, et al. Learning piece-wise linear models from large scale data for ad click prediction[EB/OL]. [2024-03-19].. |
5 | RENDLE S. Factorization machines[C]// Proceedings of the 2010 IEEE International Conference on Data Mining. Piscataway: IEEE, 2010: 995-1000. |
6 | HE X, PAN J, JIN O, et al. Practical lessons from predicting clicks on ads at Facebook[C]// Proceedings of the 8th International Workshop on Data Mining for Online Advertising. New York: ACM, 2014: 1-9. |
7 | ZHANG W, DU T, WANG J. Deep learning over multi-field categorical data — a case study on user response prediction[C]// Proceedings of the 2016 European Conference on Information Retrieval, LNCS 9626. Cham: Springer, 2016: 45-57. |
8 | 林晓颖.人工智能时代的互联网广告:基于深度学习技术的广告推荐系统[J].吉林广播电视大学学报,2020(8):143-145. |
LIN X Y. Internet advertising in the age of artificial intelligence: an advertising recommendation system based on deep learning technology[J]. Journal of Jilin Radio and Television University, 2020(8): 143-145. | |
9 | 李俊毅,陈碧欢,彭鑫,等.DeepLabel:基于深度学习的问题单分类方法研究[J].计算机应用与软件,2022,39(4):170-178. |
LI J Y, CHEN B H, PENG X, et al. DeepLabel: labeling issues based on deep learning[J]. Computer Applications and Software, 2022, 39(4):170-178. | |
10 | COVINGTON P, ADAMS J, SARGIN E. Deep neural networks for YouTube recommendations[C]// Proceedings of the 10th ACM Conference on Recommender Systems. New York: ACM, 2016: 191-198. |
11 | ZHOU G, ZHU X, SONG C, et al. Deep interest network for click-through rate prediction[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2018: 1059-1068. |
12 | ZHOU G, MOU N, FAN Y, et al. Deep interest evolution network for click-through rate prediction[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2019: 5941-5948. |
13 | LYU Z, DONG Y, HUO C, et al. Deep match to rank model for personalized click-through rate prediction[C]// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 2020: 156-163. |
14 | 王淼,李大为.基于内容与协同过滤的混合推荐算法在数字科技馆中的应用[J].网络安全技术与应用,2023(8):37-39. |
WANG M, LI D W. Application of hybrid recommendation algorithm based on content and collaborative filtering in digital science museum[J]. Network Security Technology and Application, 2023(8):37-39. | |
15 | HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[EB/OL]. [2024-01-22]. . |
16 | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778. |
17 | BA J L, KIROS J R, HINTON G E. Layer normalization[EB/OL]. [2024-02-01]. . |
18 | DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional Transformers for language understanding[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Stroudsburg, PA: ACL, 2019: 4171-4186. |
19 | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2017: 6000-6010. |
20 | LU Y, ZHANG S, HUANG Y, et al. Future-aware diverse trends framework for recommendation[C]// Proceedings of the Web Conference 2021. New York: ACM, 2021: 2992-3001. |
21 | ZHAO K, ZHAO X, CAO Q, et al. A non-sequential approach to deep user interest model for CTR prediction[C]// Proceedings of the 2022 SIAM International Conference on Data Mining. Philadelphia, PA: SIAM, 2022: 531-539. |
22 | CHENG H T, KOC L, HARMSEN J, et al. Wide & Deep learning for recommender systems[C]// Proceedings of the 1st Workshop on Deep Learning for Recommender Systems. New York: ACM, 2016: 7-10. |
23 | QU Y, CAI H, REN K, et al. Product-based neural networks for user response prediction[C]// Proceedings of the IEEE 16th International Conference on Data Mining. Piscataway: IEEE, 2016: 1149-1154. |
24 | XIAO Z, YANG L, JIANG W, et al. Deep multi-interest network for click-through rate prediction[C]// Proceedings of the 29th ACM International Conference on Information and Knowledge Management. New York: ACM, 2020: 2265-2268. |
[1] | 黄云川, 江永全, 黄骏涛, 杨燕. 基于元图同构网络的分子毒性预测[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2964-2969. |
[2] | 潘烨新, 杨哲. 基于多级特征双向融合的小目标检测优化模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2871-2877. |
[3] | 李顺勇, 李师毅, 胥瑞, 赵兴旺. 基于自注意力融合的不完整多视图聚类算法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2696-2703. |
[4] | 秦璟, 秦志光, 李发礼, 彭悦恒. 基于概率稀疏自注意力神经网络的重性抑郁疾患诊断[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2970-2974. |
[5] | 王熙源, 张战成, 徐少康, 张宝成, 罗晓清, 胡伏原. 面向手术导航3D/2D配准的无监督跨域迁移网络[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2911-2918. |
[6] | 张春雪, 仇丽青, 孙承爱, 荆彩霞. 基于两阶段动态兴趣识别的购买行为预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2365-2371. |
[7] | 刘禹含, 吉根林, 张红苹. 基于骨架图与混合注意力的视频行人异常检测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2551-2557. |
[8] | 顾焰杰, 张英俊, 刘晓倩, 周围, 孙威. 基于时空多图融合的交通流量预测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2618-2625. |
[9] | 石乾宏, 杨燕, 江永全, 欧阳小草, 范武波, 陈强, 姜涛, 李媛. 面向空气质量预测的多粒度突变拟合网络[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2643-2650. |
[10] | 唐廷杰, 黄佳进, 秦进, 陆辉. 基于图共现增强多层感知机的会话推荐[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2357-2364. |
[11] | 孙明皓, 余瀚, 陈雨青, 陆恺. 基于U形多层感知机网络的地震波初至拾取与反演[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2301-2309. |
[12] | 吴筝, 程志友, 汪真天, 汪传建, 王胜, 许辉. 基于深度学习的患者麻醉复苏过程中的头部运动幅度分类方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2258-2263. |
[13] | 李欢欢, 黄添强, 丁雪梅, 罗海峰, 黄丽清. 基于多尺度时空图卷积网络的交通出行需求预测[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2065-2072. |
[14] | 张郅, 李欣, 叶乃夫, 胡凯茜. 基于暗知识保护的模型窃取防御技术DKP[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2080-2086. |
[15] | 赵亦群, 张志禹, 董雪. 基于密集残差物理信息神经网络的各向异性旅行时计算方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2310-2318. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||