Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (9): 2858-2864.DOI: 10.11772/j.issn.1001-9081.2021081379

• Multimedia computing and computer simulation • Previous Articles    

Residual attention deraining network based on convolutional long short-term memory

Zanxia QIANG, Xianfu BAO()   

  1. School of Computer Science,Zhongyuan University of Technology,Zhengzhou Henan 450007,China
  • Received:2021-08-02 Revised:2021-11-08 Accepted:2021-11-25 Online:2022-01-07 Published:2022-09-10
  • Contact: Xianfu BAO
  • About author:QIANG Zanxia, born in 1972, Ph. D., associate professor. Her research interests include pattern recognition, artificial intelligence, computer vision.
  • Supported by:
    National Natural Science Foundation of China(61772020)


强赞霞, 鲍先富()   

  1. 中原工学院 计算机学院,郑州 450007
  • 通讯作者: 鲍先富
  • 作者简介:强赞霞(1972—),女,河南项城人,副教授,博士,CCF会员,主要研究方向:模式识别、人工智能、计算机视觉;
  • 基金资助:


Unmanned driving vehicles driving in rainy environment face the following problems: the images collected by the car on-board camera contain rain streak noise, which reduces the target detection accuracy and difficulty in identifying key targets of the unmanned driving system. In order to solve these problems, a residual attention deraining network based on convolutional long short-term memory was proposed. Firstly, the Convolutional Long Short-Term Memory (CLSTM) units were proposed to learn the distribution of different scales of rain streaks. Then, the residual channel attention mechanism was used to extract the rain streaks. Finally, the extracted rain streak information was subtracted from the rain image to obtain the restored background image. To determine the optimal network structure, the ablation experiments of each network module were carried out, and the structure with best rain removal effect was selected as the deraining network. Through the continuous optimization of network parameters, the proposed algorithm was tested on Rain100H, Rain100L and Real100 datasets, the results illustrate that the Peak Signal-to-Noise Ratio (PSNR) of the proposed algorithm reaches 29.1 dB, 33.1 dB and 32.4 dB respectively, and the Structural SIMilarity (SSIM) of the algorithm reaches 0.89, 0.94 and 0.93 respectively. Experimental results show that through the additional supervision of the Generative Adversarial Network (GAN) discriminator, the proposed algorithm achieves an visible rain streak removal effect and enhances the environmental perception ability of unmanned driving system under complex rainfall condition.

Key words: deraining, Generative Adversarial Network (GAN), Convolutional Long Short-Term Memory (CLSTM) network, residual channel attention, multi-scale feature fusion


无人驾驶汽车在雨天环境中行驶,由于车载相机采集的图片包含雨纹噪声,导致无人驾驶系统的目标检测精度降低,关键目标识别困难。为解决这些问题,提出了一种基于卷积长短期记忆的残差注意力去雨网络。首先提出卷积长短期记忆(CLSTM)单元对不同尺度的雨纹分布进行学习,然后使用残差通道注意力机制对雨纹进行提取,最后将雨图与雨纹提取信息相减得到修复后的背景图。为确定最优的网络结构,对各网络模块进行消融实验,然后选择去雨效果最优的结构作为去雨网络。通过对网络参数的不断优化,所提算法在数据集Rain100H、Rain100L、Real200上进行测试,结果显示该算法的峰值信噪比(PSNR)分别达到29.1 dB、33.1 dB、32.4 dB,结构相似性(SSIM)分别达到0.89、0.94和0.93。实验结果表明,通过生成对抗网络(GAN)判别器对雨纹去除效果的额外监督,所提算法取得了明显的雨纹去除效果,增强了无人驾驶系统在复杂降雨条件下的环境感知能力。

关键词: 去雨, 生成对抗网络, 卷积长短期记忆网络, 残差通道注意力, 多尺度特征融合

CLC Number: