Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (11): 3107-3112.DOI: 10.11772/j.issn.1001-9081.2020020263

• Artificial intelligence • Previous Articles     Next Articles

Deep transfer adaptation network based on improved maximum mean discrepancy algorithm

ZHENG Zongsheng1, HU Chenyu1, JIANG Xiaoyi2   

  1. 1. College of Information and Science, Shanghai Ocean University, Shanghai 201306, China;
    2. National Marine Information Center, Tianjin 300171, China
  • Received:2020-03-14 Revised:2020-06-01 Online:2020-11-10 Published:2020-07-07
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (41671431), the Shanghai Municipal Science and Technology Commission Local Colleges Capacity Building Project (17050501900), the Open Fund of Key Laboratory of Digital Ocean Science and Technology of State Oceanic Administration.

基于改进的最大均值差异算法的深度迁移适配网络

郑宗生1, 胡晨雨1, 姜晓轶2   

  1. 1. 上海海洋大学 信息学院, 上海 201306;
    2. 国家海洋信息中心, 天津 300171
  • 通讯作者: 胡晨雨(1995-),女,江苏盐城人,硕士研究生,主要研究方向:迁移学习、图像处理;1105814265@qq.com
  • 作者简介:郑宗生(1979-),男,河北唐山人,副教授,博士,主要研究方向:深度学习、迁移学习、遥感图像处理;姜晓轶(1973-),男,黑龙江齐齐哈尔人,博士,主要研究方向:时空数据库、地理信息系统
  • 基金资助:
    国家自然科学基金资助项目(41671431);上海市科委地方院校能力建设项目(17050501900);国家海洋局数字海洋科学技术重点实验室开放基金资助项目。

Abstract: In the study of model parameter based transfer learning, both the sample distribution discrepancy between two domains and the co-adaptation between convolutional layers of the source model impact performance of model. In response to these problems, a Multi-Convolution Adaptation (MCA) deep transfer framework was proposed and applied to the grade classification of typhoons in satellite cloud images, and a CE-MMD loss function was defined by adding the improved L-MMD (Maximum Mean Discrepancy) algorithm as a regular term to the cross-entropy function and applying the linear unbiased estimation to the distribution of the samples in Reproducing Kernel Hilbert Space (RKHS). In the back propagation process, the residual error and the distribution discrepancy between the samples in two domains were used as common indexes to update the network parameters, making model converge faster and have higher accuracy. Comparison experimental results of L-MMD and two measurement algorithms-Bregman difference and KL (Kullback-Leibler) divergence on the self-built typhoon dataset show that the precision of the proposed algorithm is improved by 11.76 percentage points and 8.05 percentage points respectively compared to those of the other two algorithms. It verifies that L-MMD is superior to other measurement algorithms and the MCA deep transfer framework is feasible.

Key words: transfer learning, Deep Convolutional Neural Network (DCNN), Maximum Mean Discrepancy (MMD), domain adaptation, typhoon grade

摘要: 在基于模型参数的迁移学习研究中,两域样本的分布差异、源模型卷积层之间的互适应性都是影响模型迁移性能的重要因素。针对上述问题,提出一种多层卷积适配(MCA)深度迁移框架并将其应用于台风卫星云图的等级分类,在交叉熵函数的基础上添加L-最大均值差异(MMD)算法作为正则项,并对样本在再生核希尔伯特空间(RKHS)中的分布进行线性的无偏估计,最终定义了CE-MMD损失函数。在反向传播过程中,将残差和两域样本分布的差异共同作为网络参数更新的指标,使得迁移模型收敛速度更快、精度更高。在自建的台风数据集上对L-MMD算法和Bregman差异、KL散度两种度量算法进行对比实验,结果表明所提算法的精度分别高出11.76个百分点和8.05个百分点。实验结果表明,L-MMD算法优于其他度量算法,而且MCA深度迁移框架具有可行性。

关键词: 迁移学习, 深度卷积神经网络, 最大均值差异, 领域适配, 台风等级

CLC Number: