Journal of Computer Applications ›› 2026, Vol. 46 ›› Issue (2): 395-405.DOI: 10.11772/j.issn.1001-9081.2025030300

• Artificial intelligence • Previous Articles    

Neural network architecture search algorithm guided by hybrid heuristic information

Qianlong XIONG1,2, Jin QIN1,2()   

  1. 1.State Key Laboratory of Public Big Data (Guizhou University),Guizhou Guiyang 550025,China
    2.College of Computer Science and Technology,Guizhou University,Guizhou Guiyang 550025,China
  • Received:2025-03-25 Revised:2025-05-14 Accepted:2025-05-16 Online:2025-05-29 Published:2026-02-10
  • Contact: Jin QIN
  • About author:XIONG Qianlong, born in 1997, M. S. candidate. His research interests include evolutionary neural network architecture search.
    QIN Jin, born in 1978, Ph. D., associate professor. His research interests include computational intelligence, reinforcement learning. Email:jqin1@gzu.edu.cn
  • Supported by:
    National Natural Science Foundation of China(62162007);Scientific and Technological Project in Guizhou (Qiankehe Talents Program KJZY [2025]020)

混合启发信息指导神经网络架构搜索算法

熊前龙1,2, 秦进1,2()   

  1. 1.公共大数据国家重点实验室(贵州大学),贵阳 550025
    2.贵州大学 计算机科学与技术学院,贵阳 550025
  • 通讯作者: 秦进
  • 作者简介:熊前龙(1997—),男,贵州黔西人,硕士研究生,主要研究方向:进化神经网络架构搜索
    秦进(1978—),男,贵州黔西人,副教授,博士,CCF会员,主要研究方向:计算智能、强化学习。Email:jqin1@gzu.edu.cn
  • 基金资助:
    国家自然科学基金资助项目(62162007);贵州省科技计划项目(黔科合人才KJZY[2025]020)

Abstract:

Aiming at Neural network Architecture Search (NAS) tasks, an NAS guided by Hybrid Heuristic Information (GHHI-NAS) algorithm was proposed. Firstly, a heuristic information construction module integrating prior knowledge and local search feedback was designed to generate multi-dimensional dynamic heuristic indicators, and a hybrid update strategy was combined to guide architecture search, thereby solving the problems of global exploration deficiency and local optimal traps caused by unidirectional updates in traditional NAS effectively. Secondly, a Covariance Matrix Adaptation Evolution Strategy (CMA-ES) update framework was adopted, enhanced by a hybrid fitness evaluation function, so that the algorithm was able to escape from small-model traps during early stage. Finally, a fitness sharing strategy was implemented to realize smooth evaluation of noise and improve population diversity. Additionally, a Monte Carlo swap sampling method with penalty mechanism was introduced to further reduce performance degradation caused by sampling. Experimental results demonstrate that GHHI-NAS has the validation accuracies of 97.55% and 83.44% on the CIFAR-10 and CIFAR-100 datasets, respectively, along with a test error rate of 24.7% on the ImageNet dataset and outstanding performance on the NAS-Bench-201 dataset, which is similar to or slightly surpasses those of Evolutionary NAS (ENAS) algorithm while requiring only 0.12 GPU-Days for search time, achieving low search costs and high test performance.

Key words: Evolutionary Algorithm (EA), Neural network Architecture Search (NAS), hybrid heuristic information, Covariance Matrix Adaptation Evolution Strategy (CMA-ES), Monte Carlo swap sampling

摘要:

针对神经网络架构搜索(NAS)任务,提出一种混合启发信息指导NAS(GHHI-NAS)算法。首先,通过设计融合先验知识与局部搜索反馈的启发信息构造模块,生成多维动态启发指标,并配合混合更新策略指导架构搜索,从而有效解决传统NAS因更新方向单一导致的全局探索不足及局部最优陷阱的问题;其次,使用自适应协方差进化策略(CMA-ES)作为更新框架,并辅以混合适应度评价函数,从而指导算法在初期跳出小模型陷阱;最后,通过适应度共享策略平滑地评价噪声并提升种群多样性。此外,为了进一步降低采样带来的性能损失,提出带惩罚机制的蒙特卡洛交换采样方法。实验结果表明,GHHI-NAS算法在CIFAR-10和CIFAR-100数据集上分别取得了97.55%和83.44%的验证正确率,在ImageNet数据集上取得了24.7%的测试错误率,在NAS-Bench-201数据集上也取得了杰出的表现,接近甚至略优于进化NAS(ENAS)算法,同时搜索时间仅为0.12 GPU-Days,实现了较低的搜索开销和较高水平的测试性能。

关键词: 进化算法, 神经网络架构搜索, 混合启发信息, 自适应协方差策略, 蒙特卡洛交换采样

CLC Number: