Learning path recommendation model integrating multi-behavior modeling and reinforcement learning

doi:10.11772/j.issn.1001-9081.2025070864

Journal of Computer Applications ›› 2026, Vol. 46 ›› Issue (8): 2485-2493.DOI: 10.11772/j.issn.1001-9081.2025070864

• Data science and technology • Previous Articles Next Articles

Learning path recommendation model integrating multi-behavior modeling and reinforcement learning

Pengyu CHEN¹, Baojun TIAN¹^,²(), Lichang ZHAO¹, Jiandong FANG²^,³

^1.College of Intelligent Science and Technology，Inner Mongolia University of Technology，Hohhot Inner Mongolia 010080，China
^2.Inner Mongolia Key Laboratory of Intelligent Perception and System Engineering （Inner Mongolia University of Technology），Hohhot Inner Mongolia 010080，China
^3.College of Information Engineering，Inner Mongolia University of Technology，Hohhot Inner Mongolia 010080，China

Received:2025-07-31 Revised:2025-10-11 Accepted:2025-10-14 Online:2025-11-05 Published:2026-08-10
Contact: Baojun TIAN
About author:CHEN Pengyu， born in 2000， M. S. candidate. His research interests include recommendation system， machine learning.
ZHAO Lichang， born in 2001， M. S. candidate. His research interests include machine learning， recommendation system.
FANG Jiandong， born in 1966， Ph. D.， professor. Her research interests include information processing， intelligent control.
Supported by:
Inner Mongolia Scientific and Technological Program(2025KYPT0088);Natural Science Foundation of Inner Mongolia Autonomous Region(2023LHMS06008);“Artificial Intelligence+” Special Project under “Unveiling and Commanding” Major Initiatives of Ordos City’s 2025 Technology Breakthrough Engineering(JB20251436)

融合多行为建模与强化学习的学习路径推荐模型

陈鹏宇¹, 田保军¹^,²(), 赵利畅¹, 房建东²^,³

^1.内蒙古工业大学智能科学与技术学院，呼和浩特 010080
^2.内蒙古自治区智能感知与系统工程重点实验室（内蒙古工业大学），呼和浩特 010080
^3.内蒙古工业大学信息工程学院，呼和浩特 010080

通讯作者: 田保军
作者简介:陈鹏宇（2000—），男，重庆人，硕士研究生，主要研究方向：推荐系统、机器学习
田保军（1971—），男，内蒙古呼和浩特人，教授，硕士，主要研究方向：机器学习、推荐系统
赵利畅（2001—），男，山东德州人，硕士研究生，主要研究方向：机器学习、推荐系统
房建东（1966—），女，内蒙古呼和浩特人，教授，博士，主要研究方向：信息处理、智能控制。
基金资助:
内蒙古自治区科技计划项目(2025KYPT0088);内蒙古自治区自然科学基金资助项目(2023LHMS06008);鄂尔多斯市2025年科技突围工程“揭榜挂帅”重大项目“人工智能+”专项(JB20251436)

Abstract

Abstract:

To address issues of sparse interaction data and unreasonable learning resource planning in learning path recommendation， a Learning Path Recommendation Model integrating Multi-behavior Modeling and Reinforcement Learning （RL）， LPRMMMRL， was proposed. First， user behavior data were classified and fused into a course knowledge graph as entity nodes， and a Graph Convolutional Network （GCN） with attention mechanism was used to capture multi-source heterogeneous features. Second， three data augmentation strategies were designed from both behavioral and learning resource perspectives， and contrastive learning was applied to carry out representation learning to the augmented data， which were integrated into RL as supplementary information. Finally， after fully considering individual differences of the users， a personalized learning path recommendation model was built using a Deep Q-Network （DQN）， and a dual reward function of both knowledge points and paths was designed to guide model convergence. Experiments were conducted for comparing LPRMMMRL with learning path recommendation models such as LPG （Learning Path Generator）， KTKDM （Knowledge Tracing based Knowledge Demand Model） on four real-world courses from MOOPer and MOOCCubeX datasets. The results show that LPRMMMRL achieves at least 5.53% and 5.74% improvements in precision， recall， and F1-score on the MOOPer dataset， respectively， and at least 4.56% and 3.41% improvements on the MOOCCubeX dataset， respectively. Besides， parameter sensitivity analysis and ablation studies validate the effectiveness of the proposed model.

Key words: learning path recommendation, multi-behavior modeling, Reinforcement Learning (RL), heterogeneous knowledge graph, Graph Convolutional Network (GCN)

摘要：

针对学习路径推荐任务中的交互数据稀疏和学习资源规划不合理问题，提出一种融合多行为建模与强化学习（RL）的学习路径推荐模型LPRMMMRL。首先，将用户行为数据分类并作为实体节点融入课程知识图谱，使用带有注意力机制的图卷积网络（GCN）捕捉多源异构特征；其次，从行为与学习资源维度设计3种数据增强策略，并采用对比学习方法对增强后的数据进行表征学习且把它们作为补充信息纳入RL；最后，充分考虑用户的个体差异，通过深度Q网络（DQN）构建个性化学习路径推荐模型，并设计知识点与路径双重奖励函数机制以引导模型收敛。在真实数据集MOOPer和MOOCCubeX的4门课程上，将LPRMMMRL与LPG（Learning Path Generator）、KTKDM（Knowledge Tracing based Knowledge Demand Model）等学习路径推荐模型进行对比。实验结果表明，LPRMMMRL的精确率、召回率和F1分数在MOOPer数据集上分别至少提升了5.53%和5.74%，在MOOCCubeX数据集上分别至少提升了4.56%和3.41%。此外，通过参数敏感性分析实验与消融实验结果，验证了所提模型的有效性。

CLC Number:

TP391.3

Pengyu CHEN, Baojun TIAN, Lichang ZHAO, Jiandong FANG. Learning path recommendation model integrating multi-behavior modeling and reinforcement learning[J]. Journal of Computer Applications, 2026, 46(8): 2485-2493.

陈鹏宇, 田保军, 赵利畅, 房建东. 融合多行为建模与强化学习的学习路径推荐模型[J]. 《计算机应用》唯一官方网站, 2026, 46(8): 2485-2493.

Figures/Tables 11

References 27

[1]	Rahayu N W， Ferdiana R， Kusumawardani S S. A systematic review of learning path recommender systems［J］. Education and Information Technologies， 2023， 28（6）： 7437-7460.
[2]	云岳，代欢，张育培，等. 个性化学习路径推荐综述［J］. 软件学报， 2022， 33（12）： 4590-4615.
	Yun Yue， Dai Huan， Zhang Yupei， et al. State-of-the-art survey on personalized learning path recommendation［J］. Journal of Software， 2022， 33（12）： 4590-4615.
[3]	Wu S， Wang J， Zhang W. Contrastive personalized exercise recommendation with reinforcement learning［J］. IEEE Transactions on Learning Technologies， 2024， 17： 691-703.
[4]	Zheng Y， Wang D， Xu Y， et al. A multigranularity learning path recommendation framework based on knowledge graph and improved ant colony optimization algorithm for e-learning［J］. IEEE Transactions on Computational Social Systems， 2025， 12（2）： 586-607.
[5]	Zhang F， Feng X， Wang Y. Personalized process-type learning path recommendation based on process mining and deep knowledge tracing［J］. Knowledge-Based Systems， 2024， 303： No.112431.
[6]	Ngo H， Vo K， Nguyen T. Personalized learning path recommendations： fusing knowledge graph embedding， sequence mining， and collaborative filtering［C］// BigData 2024. Piscataway： IEEE， 2024： 8145-8153.
[7]	Sageengrana S， Selvakumar S， Srinivasan S. Optimized RB-RNN： development of hybrid deep learning for analyzing student’s behaviours in online-learning using brain waves and chatbots［J］. Expert Systems with Applications， 2024， 248： No.123267.
[8]	Li X. Learning behavior analysis and learning effect evaluation in open online courses［J］. Creative Education， 2022， 13（4）： 1337-1352.
[9]	Sofyan Z， Staubitz T， Meinel C. Expanding learning analytics in MOOCs to track learners’ engagement with interactive content［C］// LWMOOCS 2023. Piscataway： IEEE， 2023： 1-6.
[10]	Ma D， Zhu H， Liao S， et al. Learning path recommendation with multi-behavior user modeling and cascading deep Q networks［J］. Knowledge-Based Systems， 2024， 294： No.111743.
[11]	Wang S， Xu Y， Li Q， et al. Learning path planning algorithm based on learner behavior analysis［C］// ICBDE 2021. New York： ACM， 2021： 26-33.
[12]	Shou Z， Lu X， Wu Z， et al. On learning path planning algorithm based on collaborative analysis of learning behavior［J］. IEEE Access， 2020， 8： 119863-119879.
[13]	Afsar M M， Crump T， Far B. Reinforcement learning based recommender systems： a survey［J］. ACM Computing Surveys， 2023， 55（7）： No.145.
[14]	Haldar S， Sengupta S， Das A K. Personalized learning path recommendation using graph reinforcement learning［J］. Procedia Computer Science， 2025， 258： 3480-3489.
[15]	Yun Y， Dai H， An R， et al. Doubly constrained offline reinforcement learning for learning path recommendation［J］. Knowledge-Based Systems， 2024， 284： No.111242.
[16]	Amin S， Uddin M I， Alarood A A， et al. Smart E-learning framework for personalized adaptive learning and sequential path recommendations using reinforcement learning［J］. IEEE Access， 2023， 11： 89769-89790.
[17]	Lin Y， Liu Z， Sun M， et al. Learning entity and relation embeddings for knowledge graph completion［J］. Proceedings of the AAAI Conference on Artificial Intelligence， 2015， 29（1）： 2181-2187.
[18]	Devlin J， Chang M W， Lee K， et al. BERT： pre-training of deep bidirectional transformers for language understanding［C］// NAACL-HLT 2019， Volume 1 （Long and Short Papers）. Stroudsburg： ACL， 2019： 4171-4186.
[19]	Wu Z. An efficient recommendation model based on Knowledge Graph ATtention-Assisted network （KGATAX）［PP/OL］. V1. arXiv （2024-09-05）［2025-05-20］..
[20]	Kipf T N， Welling M. Semi-supervised classification with graph convolutional networks［PP/OL］. V4. arXiv （2017-02-22）［2025-05-29］..
[21]	Li H， Gong R， Zhong Z， et al. Research on personalized learning path planning model based on knowledge network［J］. Neural Computing and Applications， 2023， 35（12）： 8809-8821.
[22]	Liu K， Zhao X， Tang J， et al. MOOPer： a large-scale dataset of practice-oriented online learning［C］// CCKS 2021， CCIS 1466 . Singapore： Springer， 2021： 281-287.
[23]	Yu J， Wang Y， Zhong Q， et al. MOOCCubeX： a large knowledge-centered repository for adaptive learning in MOOCs［C］// CIKM 2021. New York： ACM， 2021： 4643-4652.
[24]	Gao J， Liu Q， Huang W B. Learning path generator based on knowledge graph［C］// IC4E 2021. New York： ACM， 2021： 27-33.
[25]	Cai D， Zhang Y， Dai B. Learning path recommendation based on knowledge tracing model and reinforcement learning［C］// ICCC 2019. Piscataway： IEEE， 2019： 1881-1885.
[26]	Chen Y-H， Huang N-F， Tzeng J-W， et al. A personalized learning path recommender system with LINE bot in MOOCs based on LSTM［C］// ICEIT 2022. Piscataway： IEEE， 2022： 40-45.
[27]	Zhang S， Hui N， Zhai P， et al. A fine-grained and multi-context-aware learning path recommendation model over knowledge graphs for online learning communities［J］. Information Processing and Management， 2023， 60（5）： No.103464.

数据集	课程	知识点数	学习资源数	交互数据数	用户数	平均路径长度	稀疏性
MOOPer	JavaScript	121	78	61 858	2 689	23.0	0.71
MOOPer	Linux	96	50	32 854	2 493	13.2	0.74
MOOCCubeX	DS	1 484	511	26 961	2 844	9.5	0.98
MOOCCubeX	OS	1 563	207	28 719	2 940	9.8	0.95

数据集	课程	知识点数	学习资源数	交互数据数	用户数	平均路径长度	稀疏性
MOOPer	JavaScript	121	78	61 858	2 689	23.0	0.71
MOOPer	Linux	96	50	32 854	2 493	13.2	0.74
MOOCCubeX	DS	1 484	511	26 961	2 844	9.5	0.98
MOOCCubeX	OS	1 563	207	28 719	2 940	9.8	0.95

模型	JavaScript			Linux			DS			OS
模型	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
LPG	0.188 2	0.034 3	0.058 0	0.226 3	0.022 3	0.040 6	0.069 7	0.014 9	0.024 6	0.122 3	0.020 4	0.035 0
KTKDM	0.206 3	0.035 8	0.061 0	0.228 8	0.023 4	0.042 5	0.072 3	0.015 8	0.025 9	0.126 6	0.026 4	0.043 7
文献［26］模型	0.250 3	0.039 1	0.067 6	0.260 6	0.028 5	0.051 4	0.074 3	0.017 7	0.028 6	0.130 4	0.030 9	0.050 0
文献［27］模型	0.274 4	0.046 1	0.078 9	0.259 2	0.034 5	0.060 9	0.077 6	0.024 4	0.037 1	0.136 3	0.037 8	0.059 2
DCQN	0.281 4	0.046 8	0.080 3	0.261 8	0.034 8	0.061 4	0.078 9	0.024 0	0.036 8	0.137 4	0.038 1	0.059 7
LPRMMMRL	0.307 6	0.049 7	0.085 6	0.276 3	0.036 8	0.064 9	0.082 5	0.026 8	0.040 5	0.144 2	0.039 4	0.061 9

模型	JavaScript			Linux			DS			OS
模型	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
LPG	0.188 2	0.034 3	0.058 0	0.226 3	0.022 3	0.040 6	0.069 7	0.014 9	0.024 6	0.122 3	0.020 4	0.035 0
KTKDM	0.206 3	0.035 8	0.061 0	0.228 8	0.023 4	0.042 5	0.072 3	0.015 8	0.025 9	0.126 6	0.026 4	0.043 7
文献［26］模型	0.250 3	0.039 1	0.067 6	0.260 6	0.028 5	0.051 4	0.074 3	0.017 7	0.028 6	0.130 4	0.030 9	0.050 0
文献［27］模型	0.274 4	0.046 1	0.078 9	0.259 2	0.034 5	0.060 9	0.077 6	0.024 4	0.037 1	0.136 3	0.037 8	0.059 2
DCQN	0.281 4	0.046 8	0.080 3	0.261 8	0.034 8	0.061 4	0.078 9	0.024 0	0.036 8	0.137 4	0.038 1	0.059 7
LPRMMMRL	0.307 6	0.049 7	0.085 6	0.276 3	0.036 8	0.064 9	0.082 5	0.026 8	0.040 5	0.144 2	0.039 4	0.061 9

L	JavaScript			Linux			DS			OS
L	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1	Precision	Recall	F1
1	0.274 2	0.045 6	0.078 2	0.254 9	0.032 1	0.057 0	0.075 0	0.019 8	0.031 3	0.138 3	0.031 9	0.051 8
2	0.295 3	0.047 6	0.082 0	0.272 0	0.034 0	0.060 4	0.078 3	0.023 7	0.036 4	0.142 3	0.037 2	0.059 0
3	0.307 6	0.049 7	0.085 6	0.276 3	0.036 8	0.064 9	0.082 5	0.026 8	0.040 5	0.144 2	0.039 4	0.061 9
4	0.284 9	0.046 0	0.079 2	0.260 1	0.032 3	0.057 5	0.077 4	0.022 8	0.035 2	0.140 8	0.035 1	0.056 2
5	0.260 1	0.043 3	0.074 2	0.253 0	0.028 4	0.051 1	0.074 3	0.021 0	0.032 7	0.139 2	0.032 3	0.052 4

Learning path recommendation model integrating multi-behavior modeling and reinforcement learning

融合多行为建模与强化学习的学习路径推荐模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 27

Related Articles 15

Recommended Articles

Metrics

[1]	Musheng CHEN, Wenqing FU, Xiaohong QIU, Junhua WU, Qiang WEN. Aspect sentiment triplet extraction based on graph convolutional network and cross-domain data augmentation [J]. Journal of Computer Applications, 2026, 46(8): 2432-2439.
[2]	Huihui LYU, Zhaoman ZHONG, Yu ZHANG, Jidong FAN. Syntactically enhanced aspect-based sentiment analysis via mutual information maximization and contrastive learning [J]. Journal of Computer Applications, 2026, 46(8): 2440-2446.
[3]	Huijie GUO, Tianfeng DOU, Zhenlin ZHANG, Kaiyuan QI, Dong WU, Zhijian QU, Zhao LI, Chongguang REN. Time-interdependency-aware dynamic Bayesian network for traffic prediction [J]. Journal of Computer Applications, 2026, 46(5): 1507-1517.
[4]	Baoyuan ZHENG, Chaobo HE. Graph convolutional network enhanced by graph diffusion and dual-view feature learning [J]. Journal of Computer Applications, 2026, 46(5): 1370-1377.
[5]	Wenhao LI, Yinzhang GUO. Urban traffic flow prediction based on dual-layer multi-scale dynamic graph convolutional network model [J]. Journal of Computer Applications, 2026, 46(4): 1323-1333.
[6]	Chao SHI, Yuxin ZHOU, Qian FU, Wanyu TANG, Ling HE, Yuanyuan LI. Action recognition algorithm for ADHD patients using skeleton and 3D heatmap [J]. Journal of Computer Applications, 2025, 45(9): 3036-3044.
[7]	Ziliang LI, Guangli ZHU, Yulei ZHANG, Jiajia LIU, Yixuan JIAO, Shunxiang ZHANG. Aspect-based sentiment analysis model integrating syntax and sentiment knowledge [J]. Journal of Computer Applications, 2025, 45(6): 1724-1731.
[8]	Quan WANG, Qixiang LU, Pei SHI. Multi-graph diffusion attention network for traffic flow prediction [J]. Journal of Computer Applications, 2025, 45(5): 1472-1479.
[9]	Man CHEN, Xiaojun YANG, Huimin YANG. Pedestrian trajectory prediction based on graph convolutional network and endpoint induction [J]. Journal of Computer Applications, 2025, 45(5): 1480-1487.
[10]	Yufei LONG, Yuchen MOU, Ye LIU. Multi-source data representation learning model based on tensorized graph convolutional network and contrastive learning [J]. Journal of Computer Applications, 2025, 45(5): 1372-1378.
[11]	Weichao DANG, Chujun SONG, Gaimei GAO, Chunxia LIU. Multi-behavior recommendation based on cascading residual graph convolutional network [J]. Journal of Computer Applications, 2025, 45(4): 1223-1231.
[12]	Kun FU, Shicong YING, Tingting ZHENG, Jiajie QU, Jingyuan CUI, Jianwei LI. Graph data augmentation method for few-shot node classification [J]. Journal of Computer Applications, 2025, 45(2): 392-402.
[13]	Zhengyue ZHANG, Juhong PENG, Zixu DING, Xinyu FAN, Changyu HU. Aspect sentiment triplet extraction model with multi-view linguistic features and sentiment lexicon [J]. Journal of Computer Applications, 2025, 45(12): 3779-3785.
[14]	Yuqi ZHANG, Ying SHA. Chinese semantic error recognition model based on hierarchical information enhancement [J]. Journal of Computer Applications, 2025, 45(12): 3771-3778.
[15]	Mengnan XU, Hailiang YE, Feilong CAO. Neighborhood-attention and topology-aware graph convolution method for robust point cloud registration [J]. Journal of Computer Applications, 2025, 45(11): 3573-3582.