Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (8): 2578-2585.DOI: 10.11772/j.issn.1001-9081.2021061072

• Multimedia computing and computer simulation • Previous Articles    

De-raining algorithm based on joint attention mechanism for single image

Chengxia XU1, Qing YAN1, Teng LI1,2, Kaichao MIAO3()   

  1. 1.School of Electrical Engineering and Automation,Anhui University,Hefei Anhui 230601,China
    2.Key Laboratory of Intelligent Computing & Signal Processing,Ministry of Education (Anhui University),Hefei Anhui 230601,China
    3.Anhui Public Meteorological Service Center,Anhui Meteorology Service,Hefei Anhui 230091,China
  • Received:2021-06-24 Revised:2022-01-01 Accepted:2022-01-20 Online:2022-03-02 Published:2022-08-10
  • Contact: Kaichao MIAO
  • About author:XU Chengxia, born in 1995, M. S. candidate. Her research interests include image denoising based on deep learning.
    YAN Qing, born in 1978, Ph. D., associate professor. Her research interests include sparse subspace clustering, image processing.
    LI Teng, born in 1980, Ph. D., professor. His research interests include speech recognition, image processing.
    MIAO Kaichao, born in 1973, M. S., senior engineer (research fellow). His research interests include professional meteorological service.

基于联合注意力机制的单幅图像去雨算法

徐成霞1, 阎庆1, 李腾1,2, 苗开超3()   

  1. 1.安徽大学 电气工程与自动化学院, 合肥 230601
    2.计算智能与信号处理教育部重点实验室(安徽大学), 合肥 230601
    3.安徽省气象局 安徽省公共气象服务中心, 合肥 230091
  • 通讯作者: 苗开超
  • 作者简介:徐成霞(1995—),女,安徽合肥人,硕士研究生,主要研究方向:基于深度学习的图像去噪;
    阎庆(1978—),女,安徽合肥人,副教授,博士,主要研究方向:稀疏子空间聚类、图像处理;
    李腾(1980—),男,安徽合肥人,教授,博士,CCF会员,主要研究方向:语音识别、图像处理;
    苗开超(1973—),男,安徽合肥人,正研级高级工程师,硕士,主要研究方向:专业气象服务。

Abstract:

It is challenging for the existing single image de-raining algorithms to fully explore the interaction of attention mechanisms in different dimensions. Therefore, an algorithm based on joint attention mechanism was proposed to realize single image de-raining. The algorithm contains a channel attention mechanism and a spatial attention mechanism. Specifically, in the channel attention mechanism, the distribution of rain streak features in each channel was detected and the importance of each feature channel was differentiated. In the spatial attention mechanism, aiming at the spatial relationship of rain streak distribution within channels, the context information was accumulated in a local to global manner to realize efficient and accurate de-raining. Additionally, a deep residual shrinkage network with a soft threshold nonlinear transformation sub-network embedded in the residual module was used to zero out redundant information via a soft threshold function, thereby improving the ability of the CNN in retaining image details in noise. Experiments were carried out on open rainfall data sets and self constructed rainfall data sets. Compared with spatial attention, the joint attention rain removal algorithm improved Peak Signal-to-Noise Ratio (PSNR) by 4.5% and the Structural SIMilarity (SSIM) by 0.3%. Experimental results show that the proposed algorithm can effectively perform single image de-raining and image detail preserving. At the same time, this algorithm outperforms the comparison algorithms in terms of visual effect and quantitative metrics.

Key words: image de-raining, Convolutional Neural Network (CNN), attention mechanism, deep residual shrinkage network, soft threshold

摘要:

现有的单幅图像去雨算法难以充分发掘不同维度注意力机制的相互作用,因此提出一种基于联合注意力机制的单幅图像去雨算法。该算法包含通道注意力机制和空间注意力机制:通道注意力机制检测各通道雨线特征的分布,并差异化各个特征通道的重要程度;空间注意力机制则针对通道内雨线分布的空间关系,以局部到全局的方式积累上下文信息,从而高效准确地去雨。此外,引入深度残差收缩网络,以利用残差模块中嵌入的软阈值非线性变换子网络来通过软阈值函数将冗余信息置零,从而提升CNN在噪声中保留图像细节的能力。在公开降雨数据集与自构建的降雨数据集上进行实验,相较于单一空间注意力算法,联合注意力去雨算法的峰值信噪比(PSNR)提升4.5%,结构相似性(SSIM)提升0.3%。实验结果表明,所提算法可以有效地进行单幅图像去雨和图像细节的信息保留,在目视效果和定量指标上均优于对比算法。

关键词: 图像去雨, 卷积神经网络, 注意力机制, 深度残差收缩网络, 软阈值

CLC Number: