关键词: 可微神经架构搜索, one-shot架构, 超网, 梯度下降, 架构崩溃, 硬件延迟

Abstract: Neural architecture search has been recognized as an automated paradigm for designing neural network structures, through which the limitations of conventional trial-and-error–based manual design have been transcended. By this means, the reliance of deep learning model development on expert knowledge has been substantially reduced, while the overall model performance has been significantly enhanced. Among the diverse approaches, Differentiable Neural Architecture Search (DNAS), which leverages the one-shot framework in conjunction with gradient-based optimization, has drastically shortened the search time and has emerged as the predominant research direction. Nevertheless, a systematic review of differentiable methods and their developmental trajectory remains absent. To bridge this gap and provide a reference for subsequent studies, recent advances in differentiable neural architecture search are synthesized and analyzed. The paper begins by introducing the fundamental theories and benchmark datasets of differentiable neural architecture search, followed by an analysis of its existing challenges. A hierarchical review of current algorithms is then provided, and the paper concludes with a discussion on future research directions.

Key words: Differentiable Neural Architecture Search (DNAS), one-shot architecture, super net, gradient descent, architecture collapse, hardware latency