Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (12): 3618-3623.DOI: 10.11772/j.issn.1001-9081.2020050681

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Single image super-resolution method based on non-local channel attention mechanism

YE Yang1, CAI Qiong2, DU Xiaobiao3   

  1. 1. School of Computer and Information Engineering, The College of Post and Telecommunication of WIT, Wuhan Hubei 430073, China;
    2. School of Computer Science and Engineering, Wuhan Institute of Technology, Wuhan Hubei 430205, China;
    3. School of Electronic Information, Zhuhai College of Jilin University, Zhuhai Guangdong 519000, China
  • Received:2020-05-22 Revised:2020-07-27 Online:2020-12-10 Published:2020-08-14

基于非局部通道注意力机制的单图像超分辨率方法

叶杨1, 蔡琼2, 杜晓标3   

  1. 1. 武汉工程大学邮电与信息工程学院 计算机与信息工程学院, 武汉 430073;
    2. 武汉工程大学 计算机科学与工程学院, 武汉 430205;
    3. 吉林大学珠海学院 电子信息系, 广东 珠海 519000
  • 通讯作者: 叶杨(1984-),女,湖北武汉人,讲师,硕士,CCF会员,主要研究方向:深度学习、神经网络、超分辨率、图像重建。yeyang0325@qq.com
  • 作者简介:蔡琼(1961-),女,湖北武汉人,副教授,硕士,主要研究方向:数据挖掘、人工智能、机器学习;杜晓标(1999-),男,广东梅州人,主要研究方向:深度学习、超分辨率

Abstract: Single image super-resolution is an ill-posed problem, which aims to reconstruct the texture pattern with the given blurry and low-resolution image. Recently, Convolution Neural Network (CNN) was introduced into the field of super-resolution. Although excellent performance was obtained by current studies through designing the structure and the connection way of CNN, the use of edge data for training more powerful model was ignored. Therefore, a method based on edge data enhancement, that is, Non-local Channel Attention (NCA) method for single image super-resolution was proposed. The proposed method can make full use of the training data and improve performance by non-local channel attention. Not only the guideline to design the network was provided by the proposed method, but also the interpretation of super-resolution task was able to be performed by using the proposed method. The NCA Network (NCAN) model was composed of main branch and edge enhancement branch. The main branch self-attention was made for reconstructing the super-resolution images by taking the low-resolution images as input of the model and predicting the edge data. Experimental results show that, compared with the Second-order Attention Network (SAN) model, NCAN has the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) improved by 0.21 dB and 0.009 respectively on the benchmark dataset BSD100 at the magnification factor of 3; compared with the deep Residual Channel Attention Network (RCAN) model, NCAN has the PSNR and SSIM significantly improved on benchmark datasets of Set5 and Set14 at the magnification factor of 3 and 4. NCAN outperforms the state-of-the-art models on comparable parameters.

Key words: super-resolution, Convolution Neural Network (CNN), deep learning, image reconstruction, image restoration

摘要: 单图像超分辨率是一个不适定的问题,是指在给定模糊和低分辨率图像的情况下重建纹理图案。卷积神经网络(CNN)最近被引入超分辨率领域中,尽管当前研究通过设计CNN的结构和连接方式获得了出色的性能,但是忽略了可以使用边缘数据来训练更强大的模型,因此提出了一种基于边缘数据增强的方法,即单图像超分辨率的非局部通道注意力(NCA)方法。该方法可以充分利用训练数据并通过非局部通道注意力提高性能。所提方法不仅为设计网络提供了引导,而且也可以对超分辨率任务进行解释。非局部通道注意力网络(NCAN)模型由主分支和边缘增强分支组成,通过往模型里输入低分辨率图像并预测边缘数据,使主分支自注意力重建超分辨率图像。实验结果表明,在BSD100基准数据集上与二阶注意力网络(SAN)模型相比,NCAN在3倍放大因子下的峰值信噪比(PSNR)和结构相似度(SSIM)分别提升了0.21 dB和0.009;在Set5、Set14等其他基准数据集上与深度残差通道注意力网络(RCAN)模型相比,NCAN在3倍和4倍放大因子下的PSNR和SSIM都取得了较为明显的提升。NCAN在可比参数方面性能超过了最新模型。

关键词: 超分辨率, 卷积神经网络, 深度学习, 图像重建, 图像恢复

CLC Number: