Abstract:

In recent years， semantic segmentation models based on Convolutional Neural Network （CNN） have shown excellent performance in a variety of applications. However， these models usually do not generalize well when they are applied to new domains， especially from synthetic to real data. The problem of Unsupervised Domain Adaptation （UDA） is attempting to train in a known domain with labeled data （the source domain） while learning in an unknown domain with unlabeled data （the target domain）， in order to improve the generalization ability of the segmentation model trained in the source domain to the target domain. The existing methods have made great progress through training pseudo-labels on unlabeled target domain images by self-training， and various ways have been proposed to reduce low-quality pseudo-labels brought by domain migration， but the above leads to mixed results. Aiming at this problem， a masking enhanced self-training domain adaptation method was proposed to generate pseudo-labels for target domain image masking enhanced processing and correct pseudo-labels generated from unmasked target images， and with the goal of minimizing loss of consistency between the pseudo-labels of masked image and the corrected pseudo-labels of unmasked image， more features of the target domain were learnt by the model and more robust pseudo-labels were generated by the model at the same time. Experimental results show that the proposed method achieves good performance in benchmark experiments of semantic segmentation used commonly in two UDA tasks， GTA5 （Grand Theft AutoV） → Cityscapes and SYNTHIA （SYNTHetic collection of Imagery and Annotations） → Cityscapes. Specifically， compared with the classical DACS （Domain Adaptation Cross-domain Sampling） method， the proposed method improves the mean Intersection over Union （mIoU） by 1.3 percentage points on the GTA5 dataset， and 1.2 percentage points on SYNTHIA dataset. In addition， the ablation experimental results show the effectiveness of the proposed mask enhancement and pseudo-label correction modules. It can be seen that the proposed self-training domain adaptation method learns more target domain context information and improves generalization ability of the segmentation model in target domain.

Key words: mask enhancement, self-training mechanism, semantic segmentation, transfer learning, domain adaptation

摘要：

近年来，基于卷积神经网络（CNN）的语义分割模型在多种应用中表现出了卓越的性能。然而，这些模型在应用于新领域时通常不能很好地泛化，特别是从合成数据应用到真实数据的情况。无监督域适应（UDA）问题旨在尝试在一个带有标记数据的已知领域（源域）上进行模型训练，同时在无标记数据的未知领域（目标域）中学习，以提升源域训练的分割模型在目标域的泛化能力。现有方法虽然通过用自训练对无标记目标域图像进行伪标签训练的方式取得了很大进展，并提出多种方式来减少因领域迁移而产生的低质量伪标签，但效果参差不齐。针对该问题，提出一种基于掩码增强自训练的域适应方法对目标域图像掩码增强处理以生成伪标签，并对未掩码目标图像生成的伪标签进行修正，并且以最小化掩码图像伪标签和未掩码图像修正伪标签的一致性损失为目标，使模型学习到更多目标域特征的同时生成更鲁棒的伪标签。实验结果表明，所提方法在GTA5 （Grand Theft AutoV）→Cityscapes和SYNTHIA（SYNTHetic collection of Imagery and Annotations）→Cityscapes两项UDA任务常用的语义分割基准实验中均取得了不错的性能，比经典的DACS（Domain Adaptation Cross-domain Sampling）方法在GTA5和SYNTHIA数据集上的平均交并比（mIoU）分别提高了1.3和1.2个百分点；另外，消融实验结果也验证了所提掩码增强及伪标签修正模块的有效性。可见，所提出的自训练域适应方法学习到了更多的目标域上下文信息，并且提升了分割模型在目标域的泛化能力。

关键词: 掩码增强, 自训练机制, 语义分割, 迁移学习, 域适应