Abstract: Neural architecture search has been recognized as an automated paradigm for designing neural network structures, through which the limitations of conventional trial-and-error–based manual design have been transcended. By this means, the reliance of deep learning model development on expert knowledge has been substantially reduced, while the overall model performance has been significantly enhanced. Among the diverse approaches, Differentiable Neural Architecture Search (DNAS), which leverages the one-shot framework in conjunction with gradient-based optimization, has drastically shortened the search time and has emerged as the predominant research direction. Nevertheless, a systematic review of differentiable methods and their developmental trajectory remains absent. To bridge this gap and provide a reference for subsequent studies, recent advances in differentiable neural architecture search are synthesized and analyzed. The paper begins by introducing the fundamental theories and benchmark datasets of differentiable neural architecture search, followed by an analysis of its existing challenges. A hierarchical review of current algorithms is then provided, and the paper concludes with a discussion on future research directions.

Key words: Differentiable Neural Architecture Search (DNAS), one-shot architecture, super net, gradient descent, architecture collapse, hardware latency 

摘要： 神经架构搜索通过自动化设计神经网络结构，突破传统手动设计的试错局限，显著降低了深度学习模型开发对专业经验的依赖，大幅提升模型表现。在众多方法中，可微神经架构搜索(DNAS)借助one-shot框架与梯度下降优化，大幅缩短了搜索时间，已成为当前研究的主流方向。然而，现有研究仍缺乏对可微方法及其发展脉络的系统化梳理。为了填补相关空白，并为相关研究提供借鉴参考，对近年来出现的可微神经架构搜索方法进行归纳分析。首先介绍可微神经架构搜索算法的基本理论和实验数据集，梳理了可微神经架构搜索的现存挑战，之后对现有算法进行层次化分析，最后对未来发展方向进行展望。

关键词: 可微神经架构搜索, one-shot架构, 超网, 梯度下降, 架构崩溃, 硬件延迟