Single image super-resolution method based on non-local channel attention mechanism

doi:10.11772/j.issn.1001-9081.2020050681

Abstract

Abstract: Single image super-resolution is an ill-posed problem, which aims to reconstruct the texture pattern with the given blurry and low-resolution image. Recently, Convolution Neural Network (CNN) was introduced into the field of super-resolution. Although excellent performance was obtained by current studies through designing the structure and the connection way of CNN, the use of edge data for training more powerful model was ignored. Therefore, a method based on edge data enhancement, that is, Non-local Channel Attention (NCA) method for single image super-resolution was proposed. The proposed method can make full use of the training data and improve performance by non-local channel attention. Not only the guideline to design the network was provided by the proposed method, but also the interpretation of super-resolution task was able to be performed by using the proposed method. The NCA Network (NCAN) model was composed of main branch and edge enhancement branch. The main branch self-attention was made for reconstructing the super-resolution images by taking the low-resolution images as input of the model and predicting the edge data. Experimental results show that, compared with the Second-order Attention Network (SAN) model, NCAN has the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) improved by 0.21 dB and 0.009 respectively on the benchmark dataset BSD100 at the magnification factor of 3; compared with the deep Residual Channel Attention Network (RCAN) model, NCAN has the PSNR and SSIM significantly improved on benchmark datasets of Set5 and Set14 at the magnification factor of 3 and 4. NCAN outperforms the state-of-the-art models on comparable parameters.

Key words: super-resolution, Convolution Neural Network (CNN), deep learning, image reconstruction, image restoration

摘要： 单图像超分辨率是一个不适定的问题，是指在给定模糊和低分辨率图像的情况下重建纹理图案。卷积神经网络（CNN）最近被引入超分辨率领域中，尽管当前研究通过设计CNN的结构和连接方式获得了出色的性能，但是忽略了可以使用边缘数据来训练更强大的模型，因此提出了一种基于边缘数据增强的方法，即单图像超分辨率的非局部通道注意力（NCA）方法。该方法可以充分利用训练数据并通过非局部通道注意力提高性能。所提方法不仅为设计网络提供了引导，而且也可以对超分辨率任务进行解释。非局部通道注意力网络（NCAN）模型由主分支和边缘增强分支组成，通过往模型里输入低分辨率图像并预测边缘数据，使主分支自注意力重建超分辨率图像。实验结果表明，在BSD100基准数据集上与二阶注意力网络（SAN）模型相比，NCAN在3倍放大因子下的峰值信噪比（PSNR）和结构相似度（SSIM）分别提升了0.21 dB和0.009；在Set5、Set14等其他基准数据集上与深度残差通道注意力网络（RCAN）模型相比，NCAN在3倍和4倍放大因子下的PSNR和SSIM都取得了较为明显的提升。NCAN在可比参数方面性能超过了最新模型。

关键词: 超分辨率, 卷积神经网络, 深度学习, 图像重建, 图像恢复

CLC Number:

TP391.41

YE Yang, CAI Qiong, DU Xiaobiao. Single image super-resolution method based on non-local channel attention mechanism[J]. Journal of Computer Applications, 2020, 40(12): 3618-3623.

叶杨, 蔡琼, 杜晓标. 基于非局部通道注意力机制的单图像超分辨率方法[J]. 计算机应用, 2020, 40(12): 3618-3623.

References

[1] KENNEDY J A,ISRAEL O,FRENKEL A,et al. Super-resolution in PET imaging[J]. IEEE Transactions on Medical Imaging, 2006,25(2):137-147.
[2] LI B,CHANG H,SHAN S,et al. Low-resolution face recognition via coupled locality preserving mappings[J]. IEEE Signal Processing Letters,2010,17(1):20-23.
[3] DONG C,LOY C C,HE K,et al. Image super-resolution using deep convolutionalnetworks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[4] DONG C,LOY C C,TANG X. Accelerating the super-resolution convolutional neuralnetwork[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9906. Cham:Springer,2016:391-407.
[5] KIM J, LEE J K, LEE K M. Deeply-recursive convolutionalnetwork for image super-resolution[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1637-1645.
[6] LAI W,HUANG J,AHUJA N,et al. Deep Laplacian pyramidnetworks for fast and accurate super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5835-5843.
[7] LIM B,SON S,KIM H,et al. Enhanced deep residualnetworks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1134-1140.
[8] TAI Y,YANG J,LIU X. Image super-resolution via deep recursive residualnetwork[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:2790-2798.
[9] ZHANG Y,LI K,LI K,et al. Image super-resolution using very deep residual channel attentionnetworks[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11211. Cham:Springer,2018:294-310.
[10] YANG J,WRIGHT J,HUANG T S,et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[11] CHANG H,YEUNG D Y,XIONG Y. Super-resolution through neighbor embedding[C]//Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2004:275-282.
[12] FREEDMAN G,FATTAL R. Image and video upscaling from local self-examples[J]. ACM Transactions on Graphics,2011,30(2):Article No. 12.
[13] HUANG J,SINGH A,AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:5197-5206.
[14] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutionalnetworks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1646-1654.
[15] LI J,FANG F,MEI K,et al. Multi-scale residualnetwork for image super-resolution[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11212. Cham:Springer, 2018:527-548.
[16] ITTI L,KOCH C,NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[17] HU J,SHEN L,SUN G. Squeeze-and-excitationnetworks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:7132-7141.
[18] WANG X,GIRSHICK R,GUPTA A,et al. Non-local neuralnetworks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:7794-7803.
[19] FU J,LIU J,TIAN H,et al. Dual attentionnetwork for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:3141-3149.
[20] ZHANG Y,LI K,LI K,et al. Residual non-local attentionnetworks for image restoration[EB/OL].[2020-03-01]. https://arxiv.org/pdf/1903.10082.pdf.
[21] DAI T,CAI J,ZHANG Y,et al. Second-order attentionnetwork for single image super-resolution[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:11057-11066.
[22] TIMOFTE R,AGUSTSSON E,VAN GOOL L,et al. NTIRE 2017 challenge on single image super-resolution:methods and results[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1110-1121.
[23] ZHANG Y,TIAN Y,KONG Y,et al. Residual densenetwork for image super-resolution[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:2472-2481
[24] TAI Y,YANG J,LIU X,et al. MemNet:a persistent memorynetwork for image restoration[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:4549-4557.