Abstract：For the problem that the existing deep-learning based super-resolution reconstruction methods mainly study on the reconstruction problem of amplifying integer times, not on the cases of amplifying arbitrary times (e.g. non-integer times), a super-resolution reconstruction method with arbitrary magnification based on spatial meta-learning was proposed. Firstly, the coordinate projection was used to find the correspondence between the coordinates of high-resolution image and low-resolution image. Secondly, based on the meta-learning network, considering the spatial information of feature map, the extracted spatial features and coordinate positions were combined as the input of weighted prediction network. Finally, the convolution kernels predicted by the weighted prediction network were combined with the feature map in order to amplify the size of feature map effectively and obtain the high-resolution image with arbitrary magnification. The proposed spatial meta-learning module was able to be combined with other deep networks to obtain super-resolution reconstruction methods with arbitrary magnification. The provided super-resolution reconstruction method with arbitrary magnification (non-integer magnification) was able to solve the reconstruction problem with a fixed size but non-integer scale in the real life. Experimental results show that, when the space complexity (network parameters) is equivalent, the time complexity (computational cost) of the proposed method is 25%-50% of that of the other reconstruction methods, the Peak Signal-to-Noise Ratio (PSNR) of the proposed method is 0.01-5 dB higher than that of the others, and the Structural Similarity (SSIM) of the proposed method is 0.03-0.11 higher than that of the others.
[1] SHI W,CABALLERO J,LEDIG C,et al. Cardiac image superresolution with global correspondence using multi-atlas PatchMatch[C]//Proceedings of the 201316th International Conference on Medical Image Computing and Computer-Assisted Intervention, LNCS 8151. Berlin:Springer,2013:9-16. [2] YILDIRIM D,GÜNGÖR O. A novel image fusion method using IKONOS satellite images[J]. Journal of Geodesy and Geoinformation,2012,1(1):75-83. [3] 苏衡, 周杰, 张志浩. 超分辨率图像重建方法综述[J]. 自动化学报, 2013, 39(8):1202-1213.(SU H,ZHOU J,ZHANG Z H. Survey of super-resolution image reconstruction methods[J]. Acta Automatica Sinica,2013,39(8):1202-1213.) [4] DONG C,LOY C C,TANG X. Accelerating the super-resolution convolutional neuralnetwork[C]//Proceedings of the 201614th European Conference on Computer Vision,LNCS 9906. Cham:Springer,2016:391-407. [5] SHI W,CABALLERO J,HUSZÁR F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neuralnetwork[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1874-1883. [6] DONG C,LOY C C,HE K,et al. Learning a deep convolutionalnetwork for image super-resolution[C]//Proceedings of the 201413th European Conference on Computer Vision, LNCS 8692. Cham:Springer,2014:184-199. [7] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutionalnetworks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1646-1654. [8] KIM J, LEE J K, LEE K M. Deeply-recursive convolutionalnetwork for image super-resolution[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1637-1645. [9] TAI Y,YANG J,LIU X. Image super-resolution via deep recursive residualnetwork[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:2790-2798. [10] 黄陶冶, 赵建伟. 周正华. 双层可变形卷积网络的超分辨率图像重建[J]. 计算机应用, 2019, 39(S2):68-74.(HUANG T Y, ZHAO J W,ZHOU Z H. Super-resolution reconstruction using bilayer deformable convolutionalnetworks[J]. Journal of Computer Applications,2019,39(S2):68-74.) [11] LAI W,HUANG J,AHUJA N,et al. Deep Laplacian pyramidnetworks for fast and accurate super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5835-5843. [12] HARIS M, SHAKHNAROVICH G, UKITA N. Deep backprojectionnetworks for super-resolution[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:1664-1673. [13] QIU Y,WANG R,TAO D,et al. Embedded block residualnetwork:a recursive restoration model for single-image superresolution[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE,2019:4179-4188. [14] LI J,FANG F,MEI K,et al. Multi-scale residualnetwork for image super-resolution[C]//Proceedings of the 201815th European Conference on Computer Vision,LNCS 11212. Cham:Springer,2018:527-542. [15] ZHANG Y,LI K,LI K,et al. Image super-resolution using very deep residual channel attentionnetworks[C]//Proceedings of the 201815th European Conference on Computer Vision, LNCS 11211. Cham:Springer,2018:294-310. [16] ZHANG Y,TIAN Y,KONG Y,et al. Residual densenetwork for image super-resolution[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:2472-2481. [17] HU X,MU H,ZHANG X,et al. Meta-SR:a magnificationarbitrarynetwork for super-resolution[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:1575-1584. [18] YANG J,WRIGHT J,HUANG T S,et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing,2010,19(11):2861-2873. [19] MARTIN D,FOWLKES C,TAL D,et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Proceedings of the 20018th IEEE International Conference on Computer Vision. Piscataway:IEEE,2001:416-423. [20] BEVILACQUA M,ROUMY A,GUILLEMOT C,et al. Lowcomplexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the 201223rd British Machine Vision Conference. Durham:BMVA Press, 2012:No. 135. [21] ZEYDE R,ELAD M,PROTTER M. On single image scale-up using sparse-representations[C]//Proceedings of the 20107th International Conference on Curves and Surfaces,LNCS 6920. Berlin:Springer,2010:711-730. [22] ARBELÁEZ P, MAIRE M, FOWLKES C, et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2011, 33(5):898-916. [23] MATSUI Y,ITO K,ARAMAKI Y,et al. Sketch-based manga retrieval using manga109 dataset[J]. Multimedia Tools and Applications,2017,76(20):21811-21838. [24] KINGMA D P, BA J L. Adam:a method for stochastics optimization[EB/OL].[2020-02-07]. https://arxiv.org/pdf/1412.6980.pdf.
2020, Vol. 40 
