基于非局部通道注意力机制的单图像超分辨率方法

doi:10.11772/j.issn.1001-9081.2020050681

计算机应用 ›› 2020, Vol. 40 ›› Issue (12): 3618-3623.DOI: 10.11772/j.issn.1001-9081.2020050681

• 虚拟现实与多媒体计算 • 上一篇下一篇

基于非局部通道注意力机制的单图像超分辨率方法

叶杨¹, 蔡琼², 杜晓标³

1. 武汉工程大学邮电与信息工程学院计算机与信息工程学院, 武汉 430073;
2. 武汉工程大学计算机科学与工程学院, 武汉 430205;
3. 吉林大学珠海学院电子信息系, 广东珠海 519000

收稿日期:2020-05-22 修回日期:2020-07-27 出版日期:2020-12-10 发布日期:2020-08-14
通讯作者: 叶杨(1984-),女,湖北武汉人,讲师,硕士,CCF会员,主要研究方向:深度学习、神经网络、超分辨率、图像重建。yeyang0325@qq.com
作者简介:蔡琼(1961-),女,湖北武汉人,副教授,硕士,主要研究方向:数据挖掘、人工智能、机器学习;杜晓标(1999-),男,广东梅州人,主要研究方向:深度学习、超分辨率

Single image super-resolution method based on non-local channel attention mechanism

YE Yang¹, CAI Qiong², DU Xiaobiao³

1. School of Computer and Information Engineering, The College of Post and Telecommunication of WIT, Wuhan Hubei 430073, China;
2. School of Computer Science and Engineering, Wuhan Institute of Technology, Wuhan Hubei 430205, China;
3. School of Electronic Information, Zhuhai College of Jilin University, Zhuhai Guangdong 519000, China

Received:2020-05-22 Revised:2020-07-27 Online:2020-12-10 Published:2020-08-14

摘要/Abstract

摘要： 单图像超分辨率是一个不适定的问题，是指在给定模糊和低分辨率图像的情况下重建纹理图案。卷积神经网络（CNN）最近被引入超分辨率领域中，尽管当前研究通过设计CNN的结构和连接方式获得了出色的性能，但是忽略了可以使用边缘数据来训练更强大的模型，因此提出了一种基于边缘数据增强的方法，即单图像超分辨率的非局部通道注意力（NCA）方法。该方法可以充分利用训练数据并通过非局部通道注意力提高性能。所提方法不仅为设计网络提供了引导，而且也可以对超分辨率任务进行解释。非局部通道注意力网络（NCAN）模型由主分支和边缘增强分支组成，通过往模型里输入低分辨率图像并预测边缘数据，使主分支自注意力重建超分辨率图像。实验结果表明，在BSD100基准数据集上与二阶注意力网络（SAN）模型相比，NCAN在3倍放大因子下的峰值信噪比（PSNR）和结构相似度（SSIM）分别提升了0.21 dB和0.009；在Set5、Set14等其他基准数据集上与深度残差通道注意力网络（RCAN）模型相比，NCAN在3倍和4倍放大因子下的PSNR和SSIM都取得了较为明显的提升。NCAN在可比参数方面性能超过了最新模型。

关键词: 超分辨率, 卷积神经网络, 深度学习, 图像重建, 图像恢复

Abstract: Single image super-resolution is an ill-posed problem, which aims to reconstruct the texture pattern with the given blurry and low-resolution image. Recently, Convolution Neural Network (CNN) was introduced into the field of super-resolution. Although excellent performance was obtained by current studies through designing the structure and the connection way of CNN, the use of edge data for training more powerful model was ignored. Therefore, a method based on edge data enhancement, that is, Non-local Channel Attention (NCA) method for single image super-resolution was proposed. The proposed method can make full use of the training data and improve performance by non-local channel attention. Not only the guideline to design the network was provided by the proposed method, but also the interpretation of super-resolution task was able to be performed by using the proposed method. The NCA Network (NCAN) model was composed of main branch and edge enhancement branch. The main branch self-attention was made for reconstructing the super-resolution images by taking the low-resolution images as input of the model and predicting the edge data. Experimental results show that, compared with the Second-order Attention Network (SAN) model, NCAN has the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) improved by 0.21 dB and 0.009 respectively on the benchmark dataset BSD100 at the magnification factor of 3; compared with the deep Residual Channel Attention Network (RCAN) model, NCAN has the PSNR and SSIM significantly improved on benchmark datasets of Set5 and Set14 at the magnification factor of 3 and 4. NCAN outperforms the state-of-the-art models on comparable parameters.

Key words: super-resolution, Convolution Neural Network (CNN), deep learning, image reconstruction, image restoration

中图分类号:

TP391.41

叶杨, 蔡琼, 杜晓标. 基于非局部通道注意力机制的单图像超分辨率方法[J]. 计算机应用, 2020, 40(12): 3618-3623.

YE Yang, CAI Qiong, DU Xiaobiao. Single image super-resolution method based on non-local channel attention mechanism[J]. Journal of Computer Applications, 2020, 40(12): 3618-3623.

参考文献

[1] KENNEDY J A,ISRAEL O,FRENKEL A,et al. Super-resolution in PET imaging[J]. IEEE Transactions on Medical Imaging, 2006,25(2):137-147.
[2] LI B,CHANG H,SHAN S,et al. Low-resolution face recognition via coupled locality preserving mappings[J]. IEEE Signal Processing Letters,2010,17(1):20-23.
[3] DONG C,LOY C C,HE K,et al. Image super-resolution using deep convolutionalnetworks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[4] DONG C,LOY C C,TANG X. Accelerating the super-resolution convolutional neuralnetwork[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9906. Cham:Springer,2016:391-407.
[5] KIM J, LEE J K, LEE K M. Deeply-recursive convolutionalnetwork for image super-resolution[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1637-1645.
[6] LAI W,HUANG J,AHUJA N,et al. Deep Laplacian pyramidnetworks for fast and accurate super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5835-5843.
[7] LIM B,SON S,KIM H,et al. Enhanced deep residualnetworks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1134-1140.
[8] TAI Y,YANG J,LIU X. Image super-resolution via deep recursive residualnetwork[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:2790-2798.
[9] ZHANG Y,LI K,LI K,et al. Image super-resolution using very deep residual channel attentionnetworks[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11211. Cham:Springer,2018:294-310.
[10] YANG J,WRIGHT J,HUANG T S,et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing,2010,19(11):2861-2873.
[11] CHANG H,YEUNG D Y,XIONG Y. Super-resolution through neighbor embedding[C]//Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2004:275-282.
[12] FREEDMAN G,FATTAL R. Image and video upscaling from local self-examples[J]. ACM Transactions on Graphics,2011,30(2):Article No. 12.
[13] HUANG J,SINGH A,AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:5197-5206.
[14] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutionalnetworks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1646-1654.
[15] LI J,FANG F,MEI K,et al. Multi-scale residualnetwork for image super-resolution[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11212. Cham:Springer, 2018:527-548.
[16] ITTI L,KOCH C,NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[17] HU J,SHEN L,SUN G. Squeeze-and-excitationnetworks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:7132-7141.
[18] WANG X,GIRSHICK R,GUPTA A,et al. Non-local neuralnetworks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2018:7794-7803.
[19] FU J,LIU J,TIAN H,et al. Dual attentionnetwork for scene segmentation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:3141-3149.
[20] ZHANG Y,LI K,LI K,et al. Residual non-local attentionnetworks for image restoration[EB/OL].[2020-03-01]. https://arxiv.org/pdf/1903.10082.pdf.
[21] DAI T,CAI J,ZHANG Y,et al. Second-order attentionnetwork for single image super-resolution[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:11057-11066.
[22] TIMOFTE R,AGUSTSSON E,VAN GOOL L,et al. NTIRE 2017 challenge on single image super-resolution:methods and results[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1110-1121.
[23] ZHANG Y,TIAN Y,KONG Y,et al. Residual densenetwork for image super-resolution[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:2472-2481
[24] TAI Y,YANG J,LIU X,et al. MemNet:a persistent memorynetwork for image restoration[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:4549-4557.

基于非局部通道注意力机制的单图像超分辨率方法

Single image super-resolution method based on non-local channel attention mechanism

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[2]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[3]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[4]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[5]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[6]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[7]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[8]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[9]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[10]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[11]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[12]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[13]	曾祥银, 郑伯川, 刘丹. 基于深度卷积神经网络和聚类的左右轨道线检测[J]. 计算机应用, 2021, 41(8): 2324-2329.
[14]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[15]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.