Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (10): 3021-3031.DOI: 10.11772/j.issn.1001-9081.2023091313

• Artificial intelligence • Previous Articles     Next Articles

Agent model for hyperparameter self-optimization of deep classification model

Rui ZHANG1(), Junming PAN1, Xiaolu BAI2, Jing HU1, Rongguo ZHANG1, Pengyun ZHANG1   

  1. 1.School of Computer Science and Technology,Taiyuan University of Science and Technology,Taiyuan Shanxi 030024,China
    2.Faculty of Information Technology,Beijing University of Technology,Beijing 100124,China
  • Received:2023-09-25 Revised:2024-03-07 Accepted:2024-03-19 Online:2024-04-01 Published:2024-10-10
  • Contact: Rui ZHANG
  • About author:PAN Junming, born in 1996, M. S. candidate. His research interests include intelligent information processing.
    BAI Xiaolu, born in 1999, Ph. D. candidate. His research interests include intelligent information processing.
    HU Jing, born in 1977, Ph. D., professor. Her research interests include deep learning, image processing.
    ZHANG Rongguo, born in 1964, Ph. D., professor. His research interests include image processing, computer vision, pattern recognition.
    ZHANG Pengyun, born in 1999, M. S. candidate. His research interests include intelligent information processing.
  • Supported by:
    Humanities and Social Sciences Research Project of Ministry of Education(23YJCZH299);Shanxi Basic Research Program(20210302123216);Graduate Joint Training Demonstration Base Project of Taiyuan University of Science and Technology(JD2022004);Graduate Education Innovation Project of Taiyuan University of Science and Technology(SY2023040)

面向深度分类模型超参数自优化的代理模型

张睿1(), 潘俊铭1, 白晓露2, 胡静1, 张荣国1, 张鹏云1   

  1. 1.太原科技大学 计算机科学与技术学院,太原 030024
    2.北京工业大学 信息学部,北京 100124
  • 通讯作者: 张睿
  • 作者简介:张睿(1987—),男,山西太原人,副教授,博士,CCF高级会员,主要研究方向:自动机器学习、智能信息处理 zhangrui@tyust.edu.cn
    潘俊铭(1996—),男,山西太原人,硕士研究生,主要研究方向:智能信息处理
    白晓露(1999—),男,山西大同人,博士研究生,主要研究方向:智能信息处理
    胡静(1977—),女,山西大同人,教授,博士,CCF会员,主要研究方向:深度学习、图像处理
    张荣国(1964—),男,山西太原人,教授,博士,CCF高级会员,主要研究方向:图像处理、计算机视觉、模式识别
    张鹏云(1999—),男,山西太原人,硕士研究生,主要研究方向:智能信息处理。
  • 基金资助:
    教育部人文社会科学研究项目(23YJCZH299);山西省基础研究计划项目(20210302123216);太原科技大学研究生联合培养示范基地项目(JD2022004);太原科技大学研究生教育创新项目(SY2023040)

Abstract:

To further improve the efficiency of hyperparameter multi-objective adaptive optimization of deep classification models, a Filter Enhanced Dropout Agent (FEDA) model was proposed. Firstly, a dual-channel Dropout neural network with enhanced point-to-point mutual information constraint was constructed, to enhance the fitting of high-dimensional hyperparameter deep classification model, and the selection of candidate solution sets was accelerated by combining the aggregation solution selection strategy. Secondly, an FEDA model-A novel preference-based dominance Relation for Multi-Objective Evolutionary Algorithm (FEDA-ARMOEA) combined with model management strategy was designed to balance the convergence and diversity of population individuals, and to assist FEDA in improving the efficiency of deep classification model training and hyperparameter self optimization. Comparative experiments were conducted between FEDA-ARMOEA, EDN-ARMOEA (Efficient Dropout neural Network-assisted AR-MOEA), HeE-MOEA (Heterogeneous Ensemble-based infill criterion for Multi-Objective Evolutionary Algorithm), and other algorithms. Experimental results show that FEDA-ARMOEA performs well on 41 sets in all 56 sets of testing problems. Experiments on industrial application weld data set MTF and public data set CIFAR-10 show that the accuracy of FEDA-ARMOEA optimized classification model is 96.16% and 93.79%, respectively, and the training time is decreased by 6.94%-47.04% and 4.44%-39.07% compared with the contrast algorithms, respectively. All of them are superior to those of the contrast algorithms, which verifies the effectiveness and generalization of the proposed algorithm.

Key words: deep convolutional neural network, classification model, hyperparameter optimization, agent model, model optimization

摘要:

为进一步提高深度分类模型超参数多目标自适应寻优效率,提出一种筛选式增强Dropout代理(FEDA)模型。首先,构建点对互信息约束增强的双通道Dropout神经网络,增强对高维超参数深度分类模型的拟合,并结合聚集选解策略加速候选解集的选取;其次,设计一种结合模型管理策略的算法FEDA-ARMOEA(FEDA model-A novel preference-based dominance Relation for Multi-Objective Evolutionary Algorithm)均衡种群个体的收敛性和多样性,协助FEDA提高深度分类模型训练及超参数自优化效率。将FEDA-ARMOEA与EDN-ARMOEA(Efficient Dropout neural Network-assisted AR-MOEA)、HeE-MOEA(Heterogeneous Ensemble-based infill criterion for Multi-Objective Evolutionary Algorithm)等算法进行对比实验,实验结果表明,FEDA-ARMOEA在56组测试问题中的41组上表现较好。在工业应用焊缝数据集MTF和公共数据集CIFAR-10上实验,FEDA-ARMOEA优化的分类模型的精度分别达到96.16%和93.79%,训练时间相较于对比算法分别降低6.94%~47.04%和4.44%~39.07%,均优于对比算法,验证了所提算法的有效性和泛化性。

关键词: 深度卷积神经网络, 分类模型, 超参数优化, 代理模型, 模型优化

CLC Number: