Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Review of mean field theory for deep neural network
Mengmei YAN, Dongping YANG
Journal of Computer Applications    2024, 44 (2): 331-343.   DOI: 10.11772/j.issn.1001-9081.2023020166
Abstract491)   HTML59)    PDF (1848KB)(1249)       Save

Mean Field Theory (MFT) provides profound insights to understand the operation mechanism of Deep Neural Network (DNN), which can theoretically guide the engineering design of deep learning. In recent years, more and more researchers have started to devote themselves into the theoretical study of DNN, and in particular, a series of works based on mean field theory have attracted a lot of attention. To this end, a review of researches related to mean field theory for deep neural networks was presented to introduce the latest theoretical findings in three basic aspects: initialization, training process, and generalization performance of deep neural networks. Specifically, the concepts, properties and applications of edge of chaos and dynamical isometry for initialization were introduced, the training properties of overparameter networks and their equivalence networks were analyzed, and the generalization performance of various network architectures were theoretically analyzed, reflecting that mean field theory is a very important basic theoretical approach to understand the mechanisms of deep neural networks. Finally, the main challenges and future research directions were summarized for the investigation of mean field theory in the initialization, training and generalization phases of DNN.

Table and Figures | Reference | Related Articles | Metrics