关键词: 深度学习, 卷积神经网络, Dropout正则化, 过拟合, 模型平均

Abstract: In order to effectively solve the overfitting problem in deep Convolutional Neural Network (CNN), a model prediction averaging method based on Dropout improved CNN was proposed. Firstly, Dropout was employed in the pooling layers to sparse the unit values of pooling layers in the training phase. Then, in the testing phase, the probability of selecting unit value according to pooling layer Dropout was multiplied by the probability of each unit value in the pooling area as a double probability. Finally, the proposed double-probability weighted model averaging method was applied to the testing phase, so that the sparse effect of the pooling layer Dropout in the training phase was able to be better reflected on the pooling layer in the testing phase, thus achieving the low testing error as training result. The testing error rates of the proposed method in the given size network on MNIST and CIFAR-10 data sets were 0.31% and 11.23% respectively. The experimental results show that the improved method has lower error rate than Prob. weighted pooling and Stochastic Pooling method with only the impact of pooling layer on the results considered. It can be seen that the pooling layer Dropout makes the model more generalized and the pooling unit value is helpful for model generalization and can effectively avoid overfitting.

Key words: deep learning, Convolution Neural Network (CNN), Dropout regularization, overfitting, model averaging