Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (11): 3146-3151.DOI: 10.11772/j.issn.1001-9081.2020030362

• Artificial intelligence • Previous Articles     Next Articles

Convolution neural network model compression method based on pruning and tensor decomposition

GONG Kaiqiang, ZHANG Chunmei, ZENG Guanghua   

  1. College of Computer Science and Engineering, North Minzu University, Yinchuan Ningxia 750021, China
  • Received:2020-03-26 Revised:2020-06-01 Online:2020-11-10 Published:2020-07-17
  • Supported by:
    This work is partially supported by North Minzu University Graduate Innovation Project (YCX19063).

卷积神经网络模型剪枝结合张量分解压缩方法

巩凯强, 张春梅, 曾光华   

  1. 北方民族大学 计算机科学与工程学院, 银川 750021
  • 通讯作者: 巩凯强(1994-),男,甘肃天水人,硕士研究生,CCF会员,主要研究方向:计算机视觉、模型压缩;kq192011@sina.com
  • 作者简介:张春梅(1964-),女,宁夏银川人,教授,硕士,CCF会员,主要研究方向:计算机视觉、模式识别;曾光华(1992-),男,贵州铜仁人,硕士,主要研究方向:图像处理、嵌入式系统
  • 基金资助:
    北方民族大学研究生创新项目(YCX19063)。

Abstract: Focused on the problem that the huge number of parameters and calculations of Convolutional Neural Network (CNN) limit the application of CNN on resource-constrained devices such as embedded systems, a neural network compression method of statistics based network pruning and tensor decomposition was proposed. The core idea was to use the mean and variance as the basis for evaluating the weight contribution. Firstly, Lenet5 was used as a pruning model, the mean and variance distribution of each convolutional layer of the network were clustered to separate filters with weaker extracted features, and the retained filters were used to reconstruct the next convolutional layer. Secondly, the pruning method was combined with tensor decomposition to compress the Faster Region with Convolutional Neural Network (Faster RCNN). The pruning method was adopted for the low-dimensional convolution layers, and the high-dimensional convolutional layers were decomposed into three cascaded convolutional layers. Finally, the compressed model was fine-tuned, making the model be at the convergence state once again on the training set. Experimental results on the PASCAL VOC test set show that the proposed method reduces the storage space of the Faster RCNN model by 54% while the decrease of the accuracy is only 0.58%, at the same time, the method can reach 1.4 times acceleration of forward computing on the Raspberry Pi 4B system, which helpful for the deployment of deep CNN models on resource-constrained embedded devices.

Key words: Convolutional Neural Network (CNN), object detection, Faster Region with Convolutional Neural Network (Faster RCNN), pruning, tensor decomposition

摘要: 针对卷积神经网络(CNN)拥有巨大的参数量及计算量,限制了其在嵌入式系统等资源受限设备上应用的问题,提出了基于统计量的网络剪枝结合张量分解的神经网络压缩方法,其核心思想是以均值和方差作为评判权值贡献度的依据。首先,以Lenet5为剪枝模型,网络各卷积层的均值和方差分布以聚类方式分离出提取特征较弱的滤波器,而使用保留的滤波器重构下一层卷积层;然后,将剪枝方法结合张量分解对更快的区域卷积神经网络(Faster RCNN)进行压缩,低维卷积层采取剪枝方法,而高维卷积层被分解为三个级联卷积层;最后,将压缩后的模型进行微调,使其在训练集上重新达到收敛状态。在PASCAL VOC测试集上的实验结果表明,所提方法降低了Faster RCNN模型54%的存储空间而精确率仅下降了0.58%,同时在树莓派4B系统上达到1.4倍的前向计算加速,有助于深度CNN模型在资源受限的嵌入式设备上的部署。

关键词: 卷积神经网络, 目标检测, 更快的区域卷积神经网络, 剪枝, 张量分解

CLC Number: