Abstract:

Aiming at the problem that the deraining methods based on tensor product wavelet cannot capture high-frequency rain streaks in all directions， a Dual U-Former Network （DUFN） based on non-separable lifting wavelet was proposed. Firstly， the isotropic non-separable lifting wavelet was used to capture high-frequency rain streaks in all directions. In this way， compared with tensor product wavelets such as Haar wavelet， which can only capture high-frequency rain streaks in three directions， DUFN was able to obtain more comprehensive rain streak information. Secondly， two U-Nets composed of Transformer Blocks （TBs） were connected in series at various scales， so that the semantic features of the shallow decoder were transferred to the deep stage， and the rain streaks were removed more thoroughly. At the same time， the scale-guide encoder was used to guide the coding stage by using the information of various scales in the shallow layer， and Gated Fusion Module （GFM） based on CBAM （Convolutional Block Attention Module） was used to make the fusion process put more focus on the rain area. Experimental results on Rain200H， Rain200L， Rain1200 and Rain12 synthetic datasets show that the Structure SIMilarity （SSIM） of DUFN is improved by 0.009 7 on average compared to that of the advanced method SPDNet （Structure-Preserving Deraining Network）. And on Rain200H， Rain200L and Rain12 synthetic datasets， the Peak Signal-to-Noise Ratio （PSNR） of DUFN is improved by 0.657 dB averagely. On real-world dataset SPA-Data， PSNR and SSIM of DUFN are improved by 0.976 dB and 0.003 1 respectively compared with those of the advanced method ECNetLL （Embedding Consistency Network+Layered Long short-term memory）. The above verifies that DUFN can improve the rain removal performance by enhancing the ability to capture high-frequency information.

Key words: image deraining, non-separable lifting wavelet, multi-scale, Transformer, scale-guide

摘要：

针对基于张量积小波的去雨方法无法捕获所有方向的高频雨纹的问题，提出基于不可分提升小波的双U-Former网络（DUFN）。首先，利用各向同性的不可分提升小波捕捉各个方向的高频雨纹，相较于哈尔小波等张量积小波只能捕捉3个方向的高频雨纹，DUFN能获得更全面的雨纹信息；其次，在各尺度上串联两个由Transformer Block（TB）构成的U-Net，将浅层解码器的语义特征传递到深层阶段，并更彻底地去除雨纹；同时，使用尺度引导编码器通过浅层各尺度信息引导编码阶段，并利用基于CBAM（Convolutional Block Attention Module）的门控融合模块（GFM）使融合过程更专注于有雨区域。实验结果表明，相较于先进方法SPDNet（Structure-Preserving Deraining Network），在Rain200H、Rain200L、Rain1200和Rain12这4个合成数据集上，DUFN的结构相似度（SSIM）平均提高了0.009 7，在Rain200H、Rain200L和Rain12这3个合成数据集上，DUFN的峰值信噪比（PSNR）平均提高了0.657 dB；在真实世界数据集SPA-Data上，相较于先进方法ECNetLL（Embedding Consistency Network+Layered Long short-term memory），DUFN的PSNR和SSIM分别提高了0.976 dB和0.003 1。验证了DUFN可以通过增强捕捉高频信息的能力提升去雨性能。

关键词: 图像去雨, 不可分提升小波, 多尺度, Transformer, 尺度引导