《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (1): 44-49.DOI: 10.11772/j.issn.1001-9081.2021010170

• 人工智能 • 上一篇    下一篇

优化搜索空间下带约束的可微分神经网络架构搜索

李建明1,2(), 陈斌2,3, 江志伟4, 覃健4   

  1. 1.中国科学院 成都计算机应用研究所, 成都 610041
    2.中国科学院大学, 北京 100049
    3.哈尔滨工业大学(深圳) 国际人工智能研究院, 广东 深圳 518055
    4.中科院广州电子技术有限公司, 广州 510070
  • 收稿日期:2021-01-29 修回日期:2021-03-08 接受日期:2021-03-30 发布日期:2021-04-15 出版日期:2022-01-10
  • 通讯作者: 李建明
  • 作者简介:李建明(1989—),男,四川南充人,博士研究生,主要研究方向:机器视觉、人工智能、神经网络架构设计
    陈斌(1970—),男,四川广汉人,教授,博士生导师,博士,主要研究方向:机器视觉、人工智能
    江志伟(1970—),男,福建永定人,高级工程师,博士,主要研究方向:图形图像处理、人工智能、3D打印
    覃健(1992—),男,广西钟山人,主要研究方向:自动化控制、视觉检测。

Constrained differentiable neural architecture search in optimized search space

Jianming LI1,2(), Bin CHEN2,3, Zhiwei JIANG4, Jian QIN4   

  1. 1.Chengdu Institute of Computer Applications,Chinese Academy of Sciences,Chengdu Sichuan 610041,China
    2.University of Chinese Academy of Sciences,Beijing 100049,China
    3.International Research Institute of Artificial Intelligence,Harbin Institute of Technology,Shenzhen,Shenzhen Guangdong 518055,China
    4.Guangzhou Electronic Technology Company Limited,Chinese Academy of Sciences,Guangzhou Guangdong 510070,China
  • Received:2021-01-29 Revised:2021-03-08 Accepted:2021-03-30 Online:2021-04-15 Published:2022-01-10
  • Contact: Jianming LI
  • About author:LI Jianming, born in 1989, Ph. D. candidate. His research interests include computer vision, artificial intelligence, neural network architecture design.
    CHEN Bin, born in 1970, Ph. D., professor. His research interests include computer vision, artificial intelligence.
    JIANG Zhiwei, born in 1970, Ph. D., senior engineer. His research interests include graphics and image processing, artificial intelligence, 3D printing.
    QIN Jian, born in 1992. His research interests include automatic control, visual inspection.

摘要:

可微分架构搜索(DARTS)可高效、自动地设计神经网络架构,但其超网络的构建方式与派生策略的设计之间存在性能“鸿沟”。针对上述问题,提出了优化搜索空间下带约束的可微分神经网络架构搜索算法。首先,以候选操作关联的架构参数为量化指标来分析超网络的训练过程,发现在派生架构中未生效的候选操作none占据了权重最大的架构参数,从而导致算法搜得的架构表现欠佳,针对该问题设计了优化的搜索空间;然后,分析了DARTS超网络与派生架构之间的差异后,以架构参数为基础定义了架构熵,并把架构熵作为DARTS超网络目标函数的约束项,从而促使超网络缩小与派生架构的差异;最后,在CIFAR-10数据集上进行了实验。实验结果表明,所提算法在其中搜得的架构取得了97.17%的分类准确率,综合准确率、参数量和搜索时间优于对比算法。所提出的算法是有效的,提升了搜得架构在CIFAR-10数据集上的准确率。

关键词: 卷积神经网络, 网络架构搜索, 可微分架构搜索, 架构确定性, 架构熵

Abstract:

Differentiable ARchiTecture Search (DARTS) can design neural network architectures efficiently and automatically. However, there is a performance “wide gap” between the construction method of super network and the design of derivation strategy in it. To solve the above problem, a differentiable neural architecture search algorithm with constraint in optimal search space was proposed. Firstly, the training process of the super network was analyzed by using the architecture parameters associated with the candidate operations as the quantitative indicators, and it was found that the invalid candidate operation none occupied the architecture parameter with the maximum weight in deviation architecture, which caused that architectures obtained by the algorithm had poor performance. Aiming at this problem, an optimized search space was proposed. Then, the difference between the super network of DARTS and derivation architecture was analyzed, the architecture entropy was defined based on architecture parameters, and this architecture entropy was used as the constraint of the objective function of DARTS, so as to promote the super network to narrow the difference with the derivation strategy. Finally, experiments were conducted on CIFAR-10 dataset. The experimental results show that the searched architecture by the proposed algorithm achieved 97.17% classification accuracy in these experiments, better than the comparison algorithms in accuracy, parameter quantity and search time comprehensively. The proposed algorithm is effective and improves classification accuracy of searched architecture on CIFAR-10 dataset.

Key words: convolutional neural network, network architecture search, Differentiable ARchiTecture Search (DARTS), architecture determinacy, architecture entropy

中图分类号: