Abstract:

The images collected in rainy days usually have some phenomena that affect the image quality， such as the background object blocked by rain streaks and the image deformation， which have serious impact on the subsequent image analysis and application. Recently， numerous de-raining algorithms based on deep learning have been proposed and achieve good results. Most algorithms adopt supervised learning， that is， training the model on the synthetic rainy image dataset with paired labels due to the difficulty in acquiring clean background images without rain streaks from real-world rainy images. However， there are differences between synthetic and real-world rainy images on brightness， transparency， and shape of rain streaks. Thus， most de-raining algorithms based on supervised learning have poor generalization ability to real-world rainy images. Therefore， in order to improve the rain removal effect of de-raining models on the real-world rainy images， a single image de-raining algorithm based on semi-supervised learning was proposed. In the model training process of the proposed algorithm， the synthetic and real-world rainy images were added， and the difference of the first-order and second-order statistic information of feature vectors converted from the both input images were minimized to make the features of the both have same distribution. Meanwhile， in view of the complex and diverse characteristics of rain streaks， a multi-scale network was introduced to obtain richer image features and improve the performance of model. Experimental results show that， on the Rain100H dataset of synthetic rainy images， compared with Joint Deraining Network （JDNet）， Synthetic-to-Real transfer learning （Syn2Real）， the proposed algorithm improves the Peak Signal-to-Noise Ratio （PSNR） and Structural SIMilarity （SSIM） by at least 0.66 dB and 0.01 respectively. While removing rain streaks， the proposed algorithm can retain image details and color information to the greatest extent. At the same time， with the reduction of distribution discrepancy， the proposed algorithm achieves better performance on the real-world rainy images with strong generalization ability， compared with the de-raining algorithms such as JDNet and Syn2Real. The proposed algorithm is highly independent， can be applied to the existing de-raining algorithms based on supervised learning and significantly improve their de-raining effects.

Key words: single image de-raining, semi-supervised learning, multi-scale network, deep learning, dense residual connection

摘要：

在雨天采集的图像通常存在背景物体被雨纹遮挡、图像变形等影响图像质量的现象，对后续图像分析及应用造成严重影响。近年来，已经提出了许多基于深度学习的去雨算法并获得了较好的效果。由于真实雨图的无雨纹干净背景图采集非常困难，大多数算法都采用监督学习即在含有配对标签的合成雨图数据集上进行模型训练。由于合成雨图和真实雨图中雨纹的亮度、透明度、形状等存在巨大差异，基于监督学习的去雨算法对真实雨图的泛化能力普遍较差。为提高去雨模型对真实雨图的去雨效果，提出了一种基于半监督学习的单幅图像去雨算法。该算法在模型训练过程中加入合成雨图和真实雨图并最小化两个输入图像转换成的特征向量的一阶信息和二阶统计信息差异，使两者特征分布一致。同时，针对雨纹复杂多样的特点，引入多尺度网络以获取更丰富的图像特征，并提高模型性能。实验结果表明，所提算法在Rain100H合成雨图测试集上相较JDNet、Syn2Real等算法在峰值信噪比（PSNR）和结构相似度（SSIM）上分别至少提升了0.66 dB、0.01，在去除雨纹的同时能最大限度保留图像细节和颜色信息；并且由于减少了分布差异，该算法在真实雨图测试集上的去雨效果明显优于现有的JDNet、Syn2Real等去雨算法，具有较强的泛化能力。所提算法可以应用于现有的基于监督学习的去雨算法并显著提高其去雨效果，拥有较高的独立性。

关键词: 单幅图像去雨, 半监督学习, 多尺度网络, 深度学习, 密集残差连接