关键词: 卷积神经网络, 矩阵分解, 奇异值分解, 过参数化, 图像分类

Abstract:

Aiming at the difficulty of optimizing the traditional Convolutional Neural Network （CNN） in the training process， an improved method of CNN based on matrix decomposition was proposed. Firstly， the convolution kernel parameter tensor of the model convolution layer during training was converted into the product of multiple parameter matrices through matrix decomposition to form overparameterization. Secondly， these additional linear parameters were added to the back propagation of the network and updated synchronously with other parameters of the model to improve the optimization process of gradient descent. After completing the training， the matrix product was restored to the standard convolution kernel parameters， so that the computational complexity of forward propagation during inference was able to be the same as before the improvement. With thin QR decomposition and reduced Singular Value Decomposition （SVD） applied， the classification effect experiments were carried out on CIFAR-10 （Canadian Institute For Advanced Research， 10 classes） dataset， and further generalization experiments were carried out by using different image classification datasets and different initialization methods. Experimental results show that the classification accuracies of 7 models of different depths of Visual Geometry Group （VGG） and Residual Network （ResNet） based on matrix decomposition are higher than those of the original convolutional neural network models. It can be seen that the matrix decomposition method can make CNN achieve higher classification accuracy， and eventually converge to a better local optimum.

Key words: Convolution Neural Network (CNN), matrix decomposition, Singular Value Decomposition (SVD), overparameterization, image classification