基于深度渐进式反投影注意力网络的图像超分辨率重建

doi:10.11772/j.issn.1001-9081.2019122155

计算机应用 ›› 2020, Vol. 40 ›› Issue (7): 2077-2083.DOI: 10.11772/j.issn.1001-9081.2019122155

• 虚拟现实与多媒体计算 • 上一篇下一篇

基于深度渐进式反投影注意力网络的图像超分辨率重建

胡高鹏¹, 陈子鎏¹, 王晓明^1,2, 张开放¹

1. 西华大学计算机与软件工程学院, 成都 610039;
2. 西华大学机器人研究中心, 成都 610039

收稿日期:2019-12-24 修回日期:2020-03-10 发布日期:2020-05-13 出版日期:2020-07-10
通讯作者: 王晓明
作者简介:胡高鹏(1994-),男,河南商丘人,硕士研究生,主要研究方向:模式识别、图像处理;陈子鎏(1996-),男,广东韶关人,硕士研究生,主要研究方向:模式识别、图像处理;王晓明(1977-),男,四川简阳人,副教授,博士,主要研究方向:模式识别、机器学习、图像处理、计算机视觉;张开放(1994-),男,河南周口人,硕士研究生,主要研究方向:模式识别、图像处理。
基金资助:
西华大学研究生创新基金资助项目（ycjj2019095）。

Image super-resolution reconstruction based on deep progressive back-projection attention network

HU Gaopeng¹, CHEN Ziliu¹, WANG Xiaoming^1,2, ZHANG Kaifang¹

1. School of Computer and Software Engineering, Xihua University, Chengdu Sichuan 610039, China;
2. Robotics Research Center, Xihua University, Chengdu Sichuan 610039, China

Received:2019-12-24 Revised:2020-03-10 Online:2020-05-13 Published:2020-07-10

摘要/Abstract

摘要： 针对多数单帧图像超分辨率（SISR）方法在重建预测图像时存在高频信息丢失和上采样过程中会引入噪声以及特征图各通道之间的相互依赖关系难以确定等问题，提出了深度渐进式反投影注意力网络。首先使用渐进式上采样方法将低分辨率（LR）图像逐步缩放至给定的倍率，缓解上采样过程中造成的高频信息丢失等问题；然后在渐进式上采样的每个阶段融合迭代反投影思想，学习高分辨率（HR）和LR特征图之间的映射关系并减少上采样过程中引入的噪声；最后使用注意力机制为渐进式反投影网络不同阶段产生的特征图动态分配注意力资源，使网络模型学习到各特征图之间的相互依赖关系。实验结果表明，所提出的方法相比主流的超分辨率方法，峰值信噪比（PSNR）最高可增加3.16 dB，结构相似性最高可提升0.218 4。

关键词: 超分辨率, 渐进式上采样, 反投影网络, 注意力机制, 深度学习

Abstract: Focused on the problems of Single Image Super-Resolution (SISR) reconstruction methods, such as the loss of high frequency information during the process of image reconstruction, the introduction of noise during the process of upsampling and the difficulty of determining the interdependence relationships between the channels of the feature map, a deep progressive back-projection attention network was proposed. Firstly, a progressive upsampling method was used to gradually scale the Low Resolution (LR) image to a given magnification in order to alleviate problems such as high-frequency information loss caused by upsampling. Then, at each stage of progressive upsampling, iterative back-projection idea was merged to learn mapping relationship between High Resolution (HR) and LR feature maps and reduce the introduced noise in the upsampling process. Finally, the attention mechanism was used to dynamically allocate attention resources to the feature maps generated at different stages of the progressive back-projection network, so that the interdependence relationships between the feature maps were learned by the network model. Experimental results show that the proposed method can increase the Peak Signal-to-Noise Ratio (PSNR) by up to 3.16 dB and the structural similarity by up to 0.218 4.

Key words: super-resolution, progressive upsampling, back-projection network, attention mechanism, deep learning

中图分类号:

TP391.41

胡高鹏, 陈子鎏, 王晓明, 张开放. 基于深度渐进式反投影注意力网络的图像超分辨率重建[J]. 计算机应用, 2020, 40(7): 2077-2083.

HU Gaopeng, CHEN Ziliu, WANG Xiaoming, ZHANG Kaifang. Image super-resolution reconstruction based on deep progressive back-projection attention network[J]. Journal of Computer Applications, 2020, 40(7): 2077-2083.

参考文献

[1] DONG C,LOY C C,HE K,et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[2] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1646-1654.
[3] KIM J,KWON LEE J,MU LEE K. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the 2016 IEEE International Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1637-1645.
[4] SHI W,CABALLERO J,HUSZÁR F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1874-1883.
[5] LAI W S,HUANG J B,AHUJA N,et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5835-5843.
[6] LIM B,SON S,KIM H,et al. Enhanced deep residual networks for single image super-resolution[C]//Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1132-1140.
[7] LEDIG C,THEIS L,HUSZÁR F,et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:105-114.
[8] HARIS M,SHAKHNAROVICH G,UKITA N. Deep back-projection networks for super-resolution[C]//Proceeding of the 2018 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:1664-1673.
[9] ZHANG Y,LI K,LI K,et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the 15th European Conference on Computer Vision,LNCS 11211. Cham:Springer,2018:294-310.
[10] 邵文泽, 韦志辉. 基于各向异性MRF建模的多帧图像变分超分辨率重建[J]. 电子学报,2009,37(6):1256-1263.(SHAO W Z,WEI Z H. Multi-frame super-resolution reconstruction based on anisotropic Markov random field modeling[J]. Acta Electronica Sinica,2009,37(6):1256-1263.)
[11] 李展, 张庆丰, 孟小华, 等. 多分辨率图像序列的超分辨率重建[J]. 自动化学报,2012,38(11):1804-1814.(LI Z,ZHANG Q F,MENG X H,et al. Super-resolution reconstruction of multi-resolution image sequences[J]. Acta Automatica Sinica,2012,38(11):1804-1814.)
[12] 任福全, 邱天爽, 韩军, 等. 基于二阶广义全变差的多帧图像超分辨率重建[J]. 电子学报,2015,43(7):1275-1280.(REN F Q,QIU T S,HAN J,et al. Multiframe image super resolution based on second order total generalized variation[J]. Acta Electronica Sinica,2015,43(7):1275-1280.)
[13] 严宏海, 卜方玲, 徐新. 基于结构张量的视频超分辨率算法[J]. 计算机应用,2016,36(7):1944-1948,1987.(YAN H H, PU F L,XU X. Video super-resolution algorithm based on structure tensor[J]. Journal of Computer Applications,2016,36(7):1944-1948,1987.)
[14] 吉晓红, 熊淑华, 何小海, 等. 多阶导数自适应视频超分辨率重建[J]. 计算机应用,2016,36(4):1092-1095,1150.(JI X H, XIONG S H,HE X H,et al. Adaptive video super-resolution reconstruction algorithm based on multi-order derivative[J]. Journal of Computer Applications,2016,36(4):1092-1095,1150.)
[15] 孙琰玥, 何小海, 宋海英, 等. 一种用于视频超分辨率重建的块匹配图像配准方法[J]. 自动化学报,2011,37(1):37-43. (SUN Y Y,HE X H,SONG H Y,et al. A block-matching image registration algorithm for video super-resolution reconstruction[J]. Acta Automatica Sinica,2011,37(1):37-43.)
[16] TIMOFTE R,AGUSTSSON E,VAN GOOL L,et al. NTIRE 2017 challenge on single image super-resolution:methods and results[C]//Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE, 2017:1110-1121.
[17] HE K,ZHANG X,REN S,et al. Delving deep into rectifiers:surpassing human-level performance on ImageNet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:1026-1034.
[18] KINGMA D P,BA J. Adam:a method for stochastic optimization[EB/OL].[2018-12-22]. https://arxiv.org/pdf/1412.6980.pdf.
[19] YE Y,SHAN J,BRUZZONE L,et a1. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55(5):2941-2958.
[20] TAI Y,YANG J,LIU X,et al. MemNet:a persistent memory network for image restoration[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:4539-4547.
[21] DONG C,LOY C C,TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision,LNCS 9906. Cham:Springer,2016:391-407.
[22] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceeding of the 2012 British Machine Vision Conference. Durham:BMVA Press,2012:No. 135.
[23] ZEYDE R,ELAD M,PROTTER M. On single image scale-up using sparse-representations[C]//Proceeding of the 7th International Conference on Curves and Surfaces,LNCS 6920. Berlin:Springer,2010:711-730.
[24] MARTIN D,FOWLKES C,TAL D,et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceeding of the 8th IEEE International Conference on Computer Vision. Piscataway:IEEE,2001:416-423.
[25] HUANG J B,SINGH A,AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the 2015 Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:5197-5206.
[26] WANG Z,LIU D,YANG J,et al. Deep networks for image superresolution with sparse prior[C]//Proceeding of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:370-378.
[27] ZHANG K,ZUO W,ZHANG L. Learning a single convolutional super-resolution network for multiple degradations[C]//Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:3262-3271.
[28] ZHANG Y,TIAN Y,KONG Y,et al. Residual dense network for image super-resolution[C]//Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:2472-2481.

基于深度渐进式反投影注意力网络的图像超分辨率重建

Image super-resolution reconstruction based on deep progressive back-projection attention network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	秦璟, 秦志光, 李发礼, 彭悦恒. 基于概率稀疏自注意力神经网络的重性抑郁疾患诊断[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2970-2974.
[2]	王熙源, 张战成, 徐少康, 张宝成, 罗晓清, 胡伏原. 面向手术导航3D/2D配准的无监督跨域迁移网络[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2911-2918.
[3]	李力铤, 华蓓, 贺若舟, 徐况. 基于解耦注意力机制的多变量时序预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2732-2738.
[4]	李顺勇, 李师毅, 胥瑞, 赵兴旺. 基于自注意力融合的不完整多视图聚类算法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2696-2703.
[5]	黄云川, 江永全, 黄骏涛, 杨燕. 基于元图同构网络的分子毒性预测[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2964-2969.
[6]	潘烨新, 杨哲. 基于多级特征双向融合的小目标检测优化模型[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2871-2877.
[7]	赵志强, 马培红, 黑新宏. 基于双重注意力机制的人群计数方法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2886-2892.
[8]	薛凯鹏, 徐涛, 廖春节. 融合自监督和多层交叉注意力的多模态情感分析网络[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2387-2392.
[9]	汪雨晴, 朱广丽, 段文杰, 李书羽, 周若彤. 基于交互注意力机制的心理咨询文本情感分类模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2393-2399.
[10]	高鹏淇, 黄鹤鸣, 樊永红. 融合坐标与多头注意力机制的交互语音情感识别[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2400-2406.
[11]	刘禹含, 吉根林, 张红苹. 基于骨架图与混合注意力的视频行人异常检测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2551-2557.
[12]	李钟华, 白云起, 王雪津, 黄雷雷, 林初俊, 廖诗宇. 基于图像增强的低照度人脸检测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2588-2594.
[13]	莫尚斌, 王文君, 董凌, 高盛祥, 余正涛. 基于多路信息聚合协同解码的单通道语音增强[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2611-2617.
[14]	顾焰杰, 张英俊, 刘晓倩, 周围, 孙威. 基于时空多图融合的交通流量预测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2618-2625.
[15]	石乾宏, 杨燕, 江永全, 欧阳小草, 范武波, 陈强, 姜涛, 李媛. 面向空气质量预测的多粒度突变拟合网络[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2643-2650.