Multi-objective exam paper generation guided by reinforcement learning and matrix completion

doi:10.11772/j.issn.1001-9081.2024010010

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (1): 48-58.DOI: 10.11772/j.issn.1001-9081.2024010010

• Artificial intelligence • Previous Articles Next Articles

Multi-objective exam paper generation guided by reinforcement learning and matrix completion

Changzheng XING¹, Junfeng LIANG¹(), Haibo JIN¹, Jiayu XU¹, Hairong WU²

^1.School of Electronic and Information Engineering，Liaoning Technical University，Huludao Liaoning 125105，China
^2.Fuxin Big Data Management Center，Fuxin Liaoning 123000，China

Received:2024-01-11 Revised:2024-03-26 Accepted:2024-03-26 Online:2024-05-09 Published:2025-01-10
Contact: Junfeng LIANG
About author:XING Changzheng， born in 1967， Ph. D.， professor. His research interests include artificial intelligence， data mining.
JIN Haibo， born in 1983， Ph. D.， associate professor. His research interests include random process， decision theory.
XU Jiayu， born in 1999， M. S candidate. Her research interests include data mining， machine learning.
WU Hairong， born in 1977， senior engineer. Her research interests include information system project management.
Supported by:
National Natural Science Foundation of China(62173171)

强化学习和矩阵补全引导的多目标试卷生成

邢长征¹, 梁浚锋¹(), 金海波¹, 徐佳玉¹, 乌海荣²

^1.辽宁工程技术大学电子与信息工程学院，辽宁葫芦岛 125105
^2.阜新市大数据管理中心，辽宁阜新 123000

通讯作者: 梁浚锋
作者简介:邢长征（1967—），男，辽宁阜新人，教授，博士，CCF高级会员，主要研究方向：人工智能、数据挖掘；
金海波（1983—），男，辽宁葫芦岛人，副教授，博士，主要研究方向：随机过程、决策理论；
徐佳玉（1999—），女，辽宁锦州人，硕士研究生，主要研究方向：数据挖掘、机器学习；
乌海荣（1977—），女，辽宁阜新人，高级工程师，主要研究方向：信息系统项目管理。
基金资助:
国家自然科学基金资助项目(62173171)

Abstract

Abstract:

In view of the problem that the existing exam paper generation technologies pay too much attention to the difficulty of generated exam papers， while ignoring other related objectives， such as quality， score distribution， and skill coverage， a multi-objective exam paper generation method guided by reinforcement learning and matrix completion was proposed to optimize the specific objectives in the field of exam paper generation. Firstly， deep knowledge tracking method was used to model the interaction information among students and response logs in order to obtain the skill proficiency of the student group. Secondly， matrix factorization and matrix completion methods were used to predict the scores of students' undone exercises. Finally， based on the multi-objective exam paper generation strategy， in order to improve the Q network update efficiency， an Exam Q-Network function approximator was designed to select the appropriate question set automatically for update of the exam paper composition. Experimental results show that compared with the models such as DEGA （Diseased-Enhanced Genetic Algorithm） and SSA-GA （Sparrow Search Algorithm - Genetic Algorithm）， it is verified that the proposed model has significant effect in solving multiple dilemmas of exam paper generation scenarios in terms of three indicators — difficulty， rationality and accuracy. The effect of verifying the models mentioned in the solution of the test papers is significantly effective.

Key words: multi-objective exam paper generation, deep knowledge tracking, Q-Network, matrix decomposition, matrix completion

摘要：

针对现有的试卷生成技术存在过多关注生成试卷的难易程度，而忽略了其他相关目标，例如质量、分数分布和技能覆盖范围的问题，提出一种强化学习和矩阵补全引导的多目标试卷生成方法，以优化试卷生成领域的特定目标。首先，运用深度知识追踪方法对学生之间的交互信息和响应日志进行建模以获取学生群体的技能熟练程度；其次，运用矩阵分解和矩阵补全方法对学生未做的习题进行得分预测；最后，基于多目标试卷生成策略，为提升Q网络的更新效率，设计一个Exam Q-Network函数逼近器以自动地选择合适的问题集来更新试卷组成。实验结果表明，相较于DEGA （Diseased-Enhanced Genetic Algorithm）、SSA-GA （Sparrow Search Algorithm - Genetic Algorithm）等模型，在试卷难度、合理性、准确性这3个指标上验证了所提模型在解决试卷生成场景的多重困境方面上效果显著。

关键词: 多目标试卷生成, 深度知识追踪, Q网络, 矩阵分解, 矩阵补全

CLC Number:

TP181

Changzheng XING, Junfeng LIANG, Haibo JIN, Jiayu XU, Hairong WU. Multi-objective exam paper generation guided by reinforcement learning and matrix completion[J]. Journal of Computer Applications, 2025, 45(1): 48-58.

邢长征, 梁浚锋, 金海波, 徐佳玉, 乌海荣. 强化学习和矩阵补全引导的多目标试卷生成[J]. 《计算机应用》唯一官方网站, 2025, 45(1): 48-58.

Figures/Tables 12

References 40

1	CUI J， ZHOU Y， HUANG G. A test paper generation algorithm based on diseased enhanced genetic algorithm ［J］. Heliyon， 2023， 9（6）： No.e17187.
2	王凤蕊，王文宏，董金新.基于自适应果蝇优化算法的平行试卷自动生成［J］.计算工程与设计， 2015， 36（10）： 2807-2812.
	WANG F R， WANG W H， DONG J X. Fruit fly optimization algorithm with adaptive mutation for intelligent parallel test-sheets composition problem ［J］. Computer Engineering and Design， 2015， 36（10）： 2807-2812.
3	秦川，祝恒书，庄福振，等.基于知识图谱的推荐系统研究综述［J］.中国科学：信息科学， 2020， 50（7）： 937-956.
	QIN C， ZHU H S， ZHUANG F Z， et al. A survey on knowledge graph-based recommender systems ［J］. SCIENTIA SINICA Informationis， 2020， 50（7）： 937-956.
4	THOMPSON Z， DOWNEY A R J， BAKOS J D， et al. Multi-modal generative adversarial networks for synthesizing time-series structural impact responses ［J］. Mechanical Systems and Signal Processing， 2023， 204： No.110725.
5	SHAO J， YANG R. Controllable image caption based on adaptive weight and optimization strategy ［J］. Pattern Recognition and Image Analysis， 2023， 33（1）： 1-10.
6	SUN X， ZHAO X， LI B， et al. Dynamic key-value memory networks with rich features for knowledge tracing ［J］. IEEE Transactions on Cybernetics， 2022， 52（8）： 8239-8245.
7	LIU Y， YANG Y， CHEN X， et al. Improving knowledge tracing via pre-training question embeddings ［C］// Proceedings of the 29th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2020： 1577-1583.
8	PANDEY S， KARYPIS G. A self-attentive model for knowledge tracing ［EB/OL］. ［2023-03-27］. .
9	GHOSH A， HEFFERNAN N， LAN A S. Context-aware attentive knowledge tracing ［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2020： 2330-2339.
10	YANG Z P， ZHAO Y. Hybrid SGD algorithms to solve stochastic composite optimization problems with application in sparse portfolio selection problems ［J］. Journal of Computational and Applied Mathematics， 2024， 436： No.115424.
11	NAHAR M， KAMAL A H M， HOSSAIN G. Protecting health data in the cloud through steganography： a table-driven， blind method using neural networks and bit-shuffling algorithm ［J］. Journal of Network and Computer Applications， 2023， 217： No.103689.
12	YANG J H， LIU W Y， AN Y H， et al. Enhanced adaptive sequential Monte Carlo method for Bayesian model class selection by quantifying data fit and information gain ［J］. Mechanical Systems and Signal Processing， 2024， 206： No.110792.
13	ABDELFATAH R I. A color image authenticated encryption using conic curve and Mersenne twister ［J］. Multimedia Tools and Applications， 2020， 79（33/34）： 24731-24756.
14	REN M， XU R， ZHU T. Double deep Q-network decoder based on EEG brain-computer interface ［J］. ZTE Communications， 2023， 21（3）： 3-10.
15	PATEL H R， SHAH V A. Type-2 fuzzy logic applications designed for active parameter adaptation in metaheuristic algorithm for fuzzy fault-tolerant controller ［J］. International Journal of Intelligent Computing and Cybernetics， 2023， 16（2）： 198-222.
16	CHEN Y. Study on non-iterative algorithms for center-of-sets type-reduction of Takagi-Sugeno-Kang type general type-2 fuzzy logic systems ［J］. Complex and Intelligent Systems， 2022， 9： 4015-4023.
17	JOODAKI M， DOWLATSHAHI M B， JOODAKI N Z. An ensemble feature selection algorithm based on PageRank centrality and fuzzy logic ［J］. Knowledge-Based Systems， 2021， 233： No.107538.
18	SALTO C， MINETTI G， ALBA E， et al. Big optimization with genetic algorithms： Hadoop， Spark， and MPI ［J］. Soft Computing， 2023， 27（16）： 11469-11484.
19	GAO M， YANG X. APSO-SL： an adaptive particle swarm optimization with state-based learning strategy ［J］. Processes， 2024， 12（2）： No.400.
20	YAN L. Design of 3D animation color rendering system supported by cloud computing based on genetic algorithm ［J］. Soft Computing， 2024，28（）：77-77.
21	YU Z， KESKINOCAK P， ORENSTEIN W A， et al. A mixed integer programming model for vaccine pricing within a group purchasing organization ［J］. Vaccine， 2024， 42（8）： 1892-1898.
22	LI G， WEI Y， CHEN Y， et al. Softmax policy gradient methods can take exponential time to converge ［J］. Mathematical Programming， 2023， 201（1/2）： 707-802.
23	ECCLES B J， RODGERS P， KILPATRICK P， et al. DNNShifter： an efficient DNN pruning system for edge computing ［J］. Future Generation Computer Systems， 2024， 152： 43-54.
24	MA Y， SHEN Y， GUAN H， et al. A novel approach to detect the spring corn phenology using layered strategy ［J］. International Journal of Applied Earth Observation and Geoinformation， 2023， 122： No.103422.
25	REN H， REN H， SUN Z. HSFA： a novel firefly algorithm based on a hierarchical strategy ［J］. Knowledge-Based Systems， 2023， 279： No.110950.
26	ZHAO J， SHAO M， TANG H， et al. RHGNN： fake reviewer detection based on reinforced heterogeneous graph neural networks ［J］. Knowledge-Based Systems， 2023， 280： No.111029.
27	DENG C， ZHANG Q， ZHANG H， et al. Research on rapid congestion identification method based on TSNE-FCM and LightGBM ［J］. Sustainability， 2023， 15（14）： No.11322.
28	BELTRAN-ROYO C， LLOPIS-IBOR L， PANTRIGO J J， et al. DC Neural Networks avoid over fitting in one-dimensional nonlinear regression ［J］. Knowledge-Based Systems， 2024， 283： No.111154.
29	郑道博，宁慧，张汝波.基于遗传算法的自动组卷系统的策略优化设计及实现［J］.应用科技， 2023， 50（3）： 17-21.
	ZHENG D B， NING H， ZHANG R B. Strategy optimization design and implementation of automatic test paper generation system based on genetic algorithm ［J］. Applied Science and Technology， 2023， 50（3）： 17-21.
30	RAHUL， CHOUDHARY B. An advanced genetic algorithm with improved support vector machine for multi-class classification of real power quality events ［J］. Electric Power Systems Research， 2021， 191： No.106879.
31	LIANG Y， BO K， MEYYAPPAN S， et al. Decoding fMRI data with support vector machines and deep neural networks ［J］. Journal of Neuroscience Methods， 2024， 401： No.110004.
32	张锟滨，陈玉明，吴克寿，等.粒向量驱动的随机森林分类算法研究［J］.计算机工程与应用， 2024， 60（3）： 148-156.
	ZHANG K B， CHEN Y M， WU K S， et al. Research on granule vectors random forest classification algorithm ［J］. Computer Engineering and Applications， 2024， 60（3）： 148-156.
33	夏文博，范威，高莉.基于卷积神经网络的水下多目标方位估计方法［J］.声学技术， 2023， 42（3）： 290-296.
	XIA W B， FAN W， GAO L. Underwater multi-target azimuth estimation method based on convolutional neural network ［J］. Technical Acoustics， 2023， 42（3）： 290-296.
34	SONKAR S， WATERS A E， LAN A S， et al. qDKT： question-centric deep knowledge tracing ［EB/OL］. ［2023-03-27］. .
35	LIU D， ZHANG Y， ZHANG J， et al. Multiple features fusion attention mechanism enhanced deep knowledge tracing for student performance prediction ［J］. IEEE Access， 2020， 8： 194894-194903.
36	HAN Y， LIU Z， LYU Y， et al. Deep learning-based visual ensemble method for high-speed railway catenary clevis fracture detection ［J］. Neurocomputing， 2020， 396： 556-568.
37	ABIDI S M R， HUSSAIN M， XU Y， et al. Prediction of confusion attempting algebra homework in an intelligent tutoring system through machine learning techniques for educational sustainable development ［J］. Sustainability， 2019， 11（1）： No.105.
38	CHENG X. A multi-objective optimization approach for question routing in community question answering services ［J］. IEEE Transactions on Knowledge and Data Engineering， 2017， 29（9）： 1779-1792.
39	NGUYEN M L， HUI S C， FONG A C M. Large-scale multiobjective static test generation for web-based testing with integer programing ［J］. IEEE Transactions on Learning Technologies， 2013， 6（1）： 46-59.
40	HAN L M， GAO H， ZHAI R J. Modelling and Simulation of Intelligent English Paper Generating Based on SSA‐GA［J］. Mathematical Problems in Engineering， 2023， 2023（1）：2277185.

数据集	题目数	学生数	问题数	答题次数
ASSISTments0910	110	4 151	16 891	325 637
Statics2011	1 223	333	300	189 287

数据集	题目数	学生数	问题数	答题次数
ASSISTments0910	110	4 151	16 891	325 637
Statics2011	1 223	333	300	189 287

模型	难度	合理性	准确性	均值
P-value	9.722 5E-7	3.204 9E-14	0.011 5	1.629 5E-17
RSF	0.882 6±0.014 2	0.895 9±0.011 6	0.861 9±0.017 0	0.880 1±0.014 3
MOCPSO	0.897 2±0.009 0	0.912 2±0.011 6	0.872 4±0.012 5	0.893 9±0.011 0
PGA-EG	0.919 9±0.007 2	0.925 7±0.007 4	0.903 0±0.009 0	0.916 2±0.007 9
MMGA	0.916 9±0.008 6	0.920 5±0.008 5	0.898 3±0.010 4	0.911 9±0.009 2
DEGA	0.920 4±0.006 1	0.921 4±0.006 3	0.903 5±0.008 2	0.918 9±0.007 3
SSA-GA	0.923 5±0.004 6	0.934 1±0.038 0	0.910 7±0.009 5	0.920 3±0.006 7
MOEPG-r1	0.959 3±0.005 1	0.961 1±0.006 2	0.489 8±0.009 1	0.838 0±0.006 8
MOEPG-r2	0.931 9±0.003 8	0.973 3±0.003 9	0.421 9±0.011 2	0.782 0±0.006 3
MOEPG-r3	0.788 6±0.008 1	0.810 3±0.009 3	0.988 4±0.005 3	0.859 4±0.007 6
MOEPG	0.931 9±0.004 2	0.955 8±0.004 1	0.921 8±0.006 7	0.932 2±0.005 0

模型	难度	合理性	准确性	均值
P-value	9.722 5E-7	3.204 9E-14	0.011 5	1.629 5E-17
RSF	0.882 6±0.014 2	0.895 9±0.011 6	0.861 9±0.017 0	0.880 1±0.014 3
MOCPSO	0.897 2±0.009 0	0.912 2±0.011 6	0.872 4±0.012 5	0.893 9±0.011 0
PGA-EG	0.919 9±0.007 2	0.925 7±0.007 4	0.903 0±0.009 0	0.916 2±0.007 9
MMGA	0.916 9±0.008 6	0.920 5±0.008 5	0.898 3±0.010 4	0.911 9±0.009 2
DEGA	0.920 4±0.006 1	0.921 4±0.006 3	0.903 5±0.008 2	0.918 9±0.007 3
SSA-GA	0.923 5±0.004 6	0.934 1±0.038 0	0.910 7±0.009 5	0.920 3±0.006 7
MOEPG-r1	0.959 3±0.005 1	0.961 1±0.006 2	0.489 8±0.009 1	0.838 0±0.006 8
MOEPG-r2	0.931 9±0.003 8	0.973 3±0.003 9	0.421 9±0.011 2	0.782 0±0.006 3
MOEPG-r3	0.788 6±0.008 1	0.810 3±0.009 3	0.988 4±0.005 3	0.859 4±0.007 6
MOEPG	0.931 9±0.004 2	0.955 8±0.004 1	0.921 8±0.006 7	0.932 2±0.005 0

模型	难度	合理性	准确性	均值
P-value	7.354 1E-12	1.167 4E-11	6.885 4E-9	5.556 3E-21
RSF	0.938 9±0.025 1	0.836 4±0.001 8	0.682 9±0.013 8	0.819 4±0.013 6
MOCPSO	0.954 3±0.013 4	0.857 5±0.001 0	0.694 0±0.011 7	0.835 3±0.008 7
PGA-EG	0.971 2±0.006 7	0.895 3±0.000 8	0.719 9±0.007 0	0.862 1±0.004 8
MMGA	0.967 6±0.010 5	0.884 9±0.001 6	0.705 1±0.009 6	0.852 5±0.007 2
DEGA	0.972 2±0.005 7	0.892 4±0.001 1	0.718 2±0.007 0	0.861 1±0.004 2
SSA-GA	0.977 4±0.004 9	0.902 5±0.000 9	0.774 9±0.007 7	0.863 1±0.004 3
MOEPG-r1	0.990 5±0.002 7	0.906 5±0.003 1	0.327 8±0.013 9	0.745 5±0.006 8
MOEPG-r2	0.985 2±0.004 3	0.923 4±0.001 0	0.276 1±0.012 5	0.729 2±0.006 1
MOEPG-r3	0.479 0±0.005 1	0.418 9±0.002 5	0.889 5±0.004 6	0.598 3±0.004 2
MOEPG	0.988 9±0.004 9	0.909 9±0.000 7	0.734 9±0.005 4	0.887 6±0.003 7

Multi-objective exam paper generation guided by reinforcement learning and matrix completion

强化学习和矩阵补全引导的多目标试卷生成

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 40

Related Articles 14

Recommended Articles

Metrics

[1]	Yu WANG, Tianjun REN, Zilin FAN. Air combat maneuver decision-making of unmanned aerial vehicle based on guided Minimax-DDQN [J]. Journal of Computer Applications, 2023, 43(8): 2636-2643.
[2]	Yi ZHANG, Zhenmei WANG. circRNA-disease association prediction by two-stage fusion on graph auto-encoder [J]. Journal of Computer Applications, 2023, 43(6): 1979-1986.
[3]	Zhenliang LI, Bo LI. Improved method of convolution neural network based on matrix decomposition [J]. Journal of Computer Applications, 2023, 43(3): 685-691.
[4]	YAO Huayong, YE Dongyi, CHEN Zhaojiong. Multi-round conversational reinforcement learning recommendation algorithm via multi-granularity feedback [J]. Journal of Computer Applications, 2023, 43(1): 15-21.
[5]	HU Xuemin, CHENG Yu, CHEN Guowen, ZHANG Ruohan, TONG Xiuchi. Motion planning for autonomous driving with directional navigation based on deep spatio-temporal Q-network [J]. Journal of Computer Applications, 2020, 40(7): 1919-1925.
[6]	YANG Jianxi, ZHANG Yuanli, JIANG Hua, ZHU Xiaochen. Detection method of physical-layer impersonation attack based on deep Q-network in edge computing [J]. Journal of Computer Applications, 2020, 40(11): 3229-3235.
[7]	ZHANG Li, KONG Xu, SUN Zhonggui. Matrix completion algorithm based on nonlocal self-similarity and low-rank matrix approximation [J]. Journal of Computer Applications, 2020, 40(11): 3327-3331.
[8]	SHEN Xueli, LI Zijian, HE Chenhao. Hybrid recommendation algorithm based on rating filling and trust information [J]. Journal of Computer Applications, 2020, 40(10): 2789-2794.
[9]	YANG Yanlin, YE Zhonglin, ZHAO Haixing, MENG Lei. Link prediction algorithm based on high-order proximity approximation [J]. Journal of Computer Applications, 2019, 39(8): 2366-2373.
[10]	ZHOU Wan, HU Xuemin, SHI Chenyin, WEI Jieling, TONG Xiuchi. Motion planning algorithm of robot for crowd evacuation based on deep Q-network [J]. Journal of Computer Applications, 2019, 39(10): 2876-2882.
[11]	SUN Wange, XIA Kewen, LAN Pu. Regularized weighted incomplete robust principal component analysis method and its application in fitting trajectory of wireless sensor network nodes [J]. Journal of Computer Applications, 2018, 38(6): 1709-1714.
[12]	REN Xiaoxu, LYU Liangfu, CUI Guangtai. Data fusion algorithm of coupled images [J]. Journal of Computer Applications, 2018, 38(10): 2862-2868.
[13]	LI Lina, LI Wenhao, YOU Hongxiang, WANG Yue. High efficient construction of location fingerprint database based on matrix completion improved by backtracking search optimization [J]. Journal of Computer Applications, 2017, 37(7): 1893-1899.
[14]	HAO Liyan WANG Jing. Collaborative filtering recommendation algorithm based on filling and similarity confidence factor [J]. Journal of Computer Applications, 2013, 33(03): 834-837.