Image super-resolution reconstruction based on deep progressive back-projection attention network
HU Gaopeng1, CHEN Ziliu1, WANG Xiaoming1,2, ZHANG Kaifang1
1. School of Computer and Software Engineering, Xihua University, Chengdu Sichuan 610039, China; 2. Robotics Research Center, Xihua University, Chengdu Sichuan 610039, China
Abstract：Focused on the problems of Single Image Super-Resolution (SISR) reconstruction methods, such as the loss of high frequency information during the process of image reconstruction, the introduction of noise during the process of upsampling and the difficulty of determining the interdependence relationships between the channels of the feature map, a deep progressive back-projection attention network was proposed. Firstly, a progressive upsampling method was used to gradually scale the Low Resolution (LR) image to a given magnification in order to alleviate problems such as high-frequency information loss caused by upsampling. Then, at each stage of progressive upsampling, iterative back-projection idea was merged to learn mapping relationship between High Resolution (HR) and LR feature maps and reduce the introduced noise in the upsampling process. Finally, the attention mechanism was used to dynamically allocate attention resources to the feature maps generated at different stages of the progressive back-projection network, so that the interdependence relationships between the feature maps were learned by the network model. Experimental results show that the proposed method can increase the Peak Signal-to-Noise Ratio (PSNR) by up to 3.16 dB and the structural similarity by up to 0.218 4.
[1] DONG C,LOY C C,HE K,et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307. [2] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1646-1654. [3] KIM J,KWON LEE J,MU LEE K. Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the 2016 IEEE International Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1637-1645. [4] SHI W,CABALLERO J,HUSZÁR F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1874-1883. [5] LAI W S,HUANG J B,AHUJA N,et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5835-5843. [6] LIM B,SON S,KIM H,et al. Enhanced deep residual networks for single image super-resolution[C]//Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1132-1140. [7] LEDIG C,THEIS L,HUSZÁR F,et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:105-114. [8] HARIS M,SHAKHNAROVICH G,UKITA N. Deep back-projection networks for super-resolution[C]//Proceeding of the 2018 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:1664-1673. [9] ZHANG Y,LI K,LI K,et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the 15th European Conference on Computer Vision,LNCS 11211. Cham:Springer,2018:294-310. [10] 邵文泽, 韦志辉. 基于各向异性MRF建模的多帧图像变分超分辨率重建[J]. 电子学报,2009,37(6):1256-1263.(SHAO W Z,WEI Z H. Multi-frame super-resolution reconstruction based on anisotropic Markov random field modeling[J]. Acta Electronica Sinica,2009,37(6):1256-1263.) [11] 李展, 张庆丰, 孟小华, 等. 多分辨率图像序列的超分辨率重建[J]. 自动化学报,2012,38(11):1804-1814.(LI Z,ZHANG Q F,MENG X H,et al. Super-resolution reconstruction of multi-resolution image sequences[J]. Acta Automatica Sinica,2012,38(11):1804-1814.) [12] 任福全, 邱天爽, 韩军, 等. 基于二阶广义全变差的多帧图像超分辨率重建[J]. 电子学报,2015,43(7):1275-1280.(REN F Q,QIU T S,HAN J,et al. Multiframe image super resolution based on second order total generalized variation[J]. Acta Electronica Sinica,2015,43(7):1275-1280.) [13] 严宏海, 卜方玲, 徐新. 基于结构张量的视频超分辨率算法[J]. 计算机应用,2016,36(7):1944-1948,1987.(YAN H H, PU F L,XU X. Video super-resolution algorithm based on structure tensor[J]. Journal of Computer Applications,2016,36(7):1944-1948,1987.) [14] 吉晓红, 熊淑华, 何小海, 等. 多阶导数自适应视频超分辨率重建[J]. 计算机应用,2016,36(4):1092-1095,1150.(JI X H, XIONG S H,HE X H,et al. Adaptive video super-resolution reconstruction algorithm based on multi-order derivative[J]. Journal of Computer Applications,2016,36(4):1092-1095,1150.) [15] 孙琰玥, 何小海, 宋海英, 等. 一种用于视频超分辨率重建的块匹配图像配准方法[J]. 自动化学报,2011,37(1):37-43. (SUN Y Y,HE X H,SONG H Y,et al. A block-matching image registration algorithm for video super-resolution reconstruction[J]. Acta Automatica Sinica,2011,37(1):37-43.) [16] TIMOFTE R,AGUSTSSON E,VAN GOOL L,et al. NTIRE 2017 challenge on single image super-resolution:methods and results[C]//Proceeding of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE, 2017:1110-1121. [17] HE K,ZHANG X,REN S,et al. Delving deep into rectifiers:surpassing human-level performance on ImageNet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:1026-1034. [18] KINGMA D P,BA J. Adam:a method for stochastic optimization[EB/OL].[2018-12-22]. https://arxiv.org/pdf/1412.6980.pdf. [19] YE Y,SHAN J,BRUZZONE L,et a1. Robust registration of multimodal remote sensing images based on structural similarity[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55(5):2941-2958. [20] TAI Y,YANG J,LIU X,et al. MemNet:a persistent memory network for image restoration[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:4539-4547. [21] DONG C,LOY C C,TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision,LNCS 9906. Cham:Springer,2016:391-407. [22] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceeding of the 2012 British Machine Vision Conference. Durham:BMVA Press,2012:No. 135. [23] ZEYDE R,ELAD M,PROTTER M. On single image scale-up using sparse-representations[C]//Proceeding of the 7th International Conference on Curves and Surfaces,LNCS 6920. Berlin:Springer,2010:711-730. [24] MARTIN D,FOWLKES C,TAL D,et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceeding of the 8th IEEE International Conference on Computer Vision. Piscataway:IEEE,2001:416-423. [25] HUANG J B,SINGH A,AHUJA N. Single image super-resolution from transformed self-exemplars[C]//Proceedings of the 2015 Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:5197-5206. [26] WANG Z,LIU D,YANG J,et al. Deep networks for image superresolution with sparse prior[C]//Proceeding of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:370-378. [27] ZHANG K,ZUO W,ZHANG L. Learning a single convolutional super-resolution network for multiple degradations[C]//Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:3262-3271. [28] ZHANG Y,TIAN Y,KONG Y,et al. Residual dense network for image super-resolution[C]//Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:2472-2481.
2020, Vol. 40 
