Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (10): 2817-2821.DOI: 10.11772/j.issn.1001-9081.2020020222

• Artificial intelligence • Previous Articles     Next Articles

Bio-inspired matrix reduction and quantization method for deep neural network

ZHU Qianqian1,2, LIU Yuan1,2, LI Fu3   

  1. 1. School of Artificial Intelligence and Computer, Jiangnan University, Wuxi Jiangsu 214122, China;
    2. Jiangsu Key Laboratory of Media Design and Software Technology(Jiangnan University), Wuxi Jiangsu 214122, China;
    3. Quantum Cloud New Media Technology Company Limited, Wuxi Jiangsu 214122, China
  • Received:2020-03-05 Revised:2020-05-11 Online:2020-10-10 Published:2020-05-21
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61972182)

深度神经网络的仿生矩阵约简与量化方法

朱倩倩1,2, 刘渊1,2, 李甫3   

  1. 1. 江南大学 人工智能与计算机学院, 江苏 无锡 214122;
    2. 江苏省媒体设计与软件技术重点实验室(江南大学), 江苏 无锡 214122;
    3. 无锡量子云数字新媒体科技有限公司, 江苏 无锡 214122
  • 通讯作者: 朱倩倩
  • 作者简介:朱倩倩(1995-),女,江苏连云港人,硕士研究生,CCF会员,主要研究方向:计算机视觉、人工智能;刘渊(1967-),男,江苏无锡人,教授,博士,CCF高级会员,主要研究方向:网络应用、数字媒体;李甫(1984-),男,湖北武汉人,教授,博士,主要研究方向:量子动力学、量子统计、未来网络、高性能计算。
  • 基金资助:
    国家自然科学基金资助项目(61972182)。

Abstract: Bio-inspired Deep Neural Network (DNN) is a revolutionary breakthrough in artificial intelligent field. However, the lack of storage space as well as computing capacity caused by the explosive increase of the model weights not only keeps DNN apart from its original inspiration, but also makes it difficult to deploy DNN on embedded/mobile devices. In order to solve this problem, the biological selection principle in the evolution was studied, and a novel neural network algorithm based on "evolution" + "randomness" + "selection" was proposed. In this method, the size of the existing models were greatly simplified on the premise of maintaining the basic framework of the existing neural network models. First, the weight parameters were clustered. Then, based on the cluster centroid values of the parameters, the random perturbation was added to reconstruct the parameters. Finally, the image classification and object detection were performed on the reconstructed model to realize the accuracy test and model stability analysis. Experimental results on ImageNet dataset and COCO dataset show that the proposed model reconstruction method can compress the sizes of four models, including Darknet19, ResNet18, ResNet50 and YOLOv3, to 1/4-1/3 of the original ones, and under the condition of 1%-3% performance improvement in the test accuracy of image classification and object detection, there is the possibility of further simplification.

Key words: model compression, Deep Neural Network (DNN), parameter reconstruction, object detection, network dynamics, bio-inspired model

摘要: 基于生物学原理的深度神经网络(DNN)的发展给人工智能领域带来了革命性的突破,然而当前神经网络的发展却越来越脱离生物学原理,DNN越来越臃肿的模型对存储空间和计算力的需求越来越高,并且对于DNN在嵌入式/移动端设备上的部署带来了阻碍。针对这一问题,对生物学进化选择原理进行研究,并提出一种基于“进化”+“随机”+“选择”的全新神经网络算法。该方法在保持现有神经网络模型的基本框架的前提下,能极大简化现有模型的大小。首先对权值参数进行聚类,然后在参数的聚类质心值的基础上添加随机微扰进行参数重构,最后通过对重构模型进行图像分类和目标检测来实现准确度测试以及模型稳定性分析。在ImageNet数据集和COCO数据集上的实验结果表明,提出的模型重构方法在对图像分类和目标检测的测试准确度提升1%~3%的情况下,仍可将Darknet19、ResNet18、ResNet50以及YOLOv3等四种重构模型的体量压缩到原来的1/4~1/3,并还有进一步简化的可能。

关键词: 模型压缩, 深度神经网络, 参数重构, 目标检测, 网络动力学, 仿生模型

CLC Number: