基于轻量自动残差缩放网络的图像超分辨率重建

doi:10.11772/j.issn.1001-9081.2019112014

摘要
图/表
参考文献(39)
相关文章 (15)

全文: PDF (1461 KB) HTML
输出: BibTeX | EndNote (RIS)

摘要

近年来，由于深度卷积神经网络的出色性能，深度学习已成为图像超分辨率领域的研究热点，已经有许多具有很深结构的大型模型被提出。而在实际应用中，普通个人计算机或智能终端的硬件显然不适合大规模深度神经网络模型。提出了一种针对单幅图像超分辨率且具有自动残差缩放功能的轻量级网络(ARSN)，与许多基于深度学习的方法相比，它的层和参数更少。此外，该网络中有特殊的残差块和跳跃连接用来进行残差缩放以及全局和局部残差学习。根据测试数据集结果，该网络在重建质量和运行速度上都达到了非常优异的性能。所提出的网络在性能、速度和硬件消耗方面均取得了良好的效果，具有较高的实用价值。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	代强
	程曦
	王永梅
	牛子未
	刘飞

关键词 ：深度学习, 超分辨率, 残差缩放, 跳跃连接, 残差网络

Abstract：

Recently, deep learning has been a hot research topic in the field of image super-resolution due to the excellent performance of deep convolutional neural networks. Many large-scale models with very deep structures have been proposed. However, in practical applications, the hardware of ordinary personal computers or smart terminals are obviously not suitable for large-scale deep neural network models. A light-weight Network with Automatic Residual Scaling (ARSN) for single image super-resolution was proposed, which has fewer layers and parameters compared with many other deep learning based methods. In addition, the specified residual blocks and skip connections in this network were utilized for residual scaling, global and local residual learning. The results on test datasets show that this model achieves state-of-the-art performance on both reconstruction quality and running speed. The proposed network achieves good results in terms of performance, speed and hardware consumption， and has high practical value.

Key words： deep learning super-resolution residual scaling skip connection residual network

收稿日期: 2019-11-28

中图分类号:

TP391.41

基金资助:

安徽省教育厅质量工程项目（2016ckjh080）；教育部产学研协同育人项目（201702126125）；大学生创新创业训练项目（201910364073）；江淮中部粮食作物生产智能化作业与全程信息项目（11004836）。

通讯作者: 王永梅（1974—） E-mail: wym@ahau.edu.cn

作者简介: 代强（1997—），男，安徽合肥人，主要研究方向：深度学习、计算机视觉；程曦（1996—），男，江苏南京人，博士研究生，CCF会员，主要研究方向：深度学习、计算机视觉、数字图像处理；王永梅（1974—），女，安徽合肥人，副教授，硕士，主要研究方向：智能信息处理、农业物联网；牛子未（2000—），男，安徽淮北人，主要研究方向：深度学习、数字图像处理；刘飞（1989—），男，安徽合肥人，助理研究员，硕士，主要研究方向：法学理论、农产品质量安全。

引用本文:

代强, 程曦, 王永梅, 牛子未, 刘飞. 基于轻量自动残差缩放网络的图像超分辨率重建[J]. 计算机应用, 2020, 40(5): 1446-1452. DAI Qiang, CHENG Xi, WANG Yongmei, NIU Ziwei, LIU Fei. Light-weight automatic residual scaling network for image super-resolution reconstruction. Journal of Computer Applications, 2020, 40(5): 1446-1452.

链接本文:

http://www.joca.cn/CN/10.11772/j.issn.1001-9081.2019112014 或 http://www.joca.cn/CN/Y2020/V40/I5/1446

1 TAYLOR J G , CUTSURIDIS V . Saliency, attention, active visual search, and picture scanning[J]. Cognitive Computation, 2011, 3(1): 1-3.
2 ABOUDIB A , GRIPON V , COPPIN G . A biologically inspired framework for visual information processing and an application on modeling bottom-up visual attention[J]. Cognitive Computation, 2016, 8(6): 1007-1026.
3 KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[C]// Proceedings of the 25th International Conference on Neural Information Processing Systems. New York: Curran Associates Inc., 2012: 1097-1105.
4 SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[EB/OL]. [2019-04-05].https://arxiv.org/pdf/1409.1556.pdf.
5 SRIVASTAVA R K , GREFF K , SCHMIDHUBER J . Highway networks[EB/OL]. [2019-04-05].http://de.arxiv.org/pdf/1505.00387.
6 HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
7 REN S , HE K , GIRSHICK R , et al . Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
8 DAI J , LI Y , HE K , et al . R-FCN: object detection via region-based fully convolutional networks[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: Curran Associates Inc., 2016: 379-387.
9 SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
10 彭晏飞,孙鲁 . 基于图像分割的语义标注方法[J]. 计算机应用, 2012,32(6): 1548-1551. (PENG Y F, SUN L. Semantic annotation based on image segmentation[J]. Journal of Computer Applications, 2012, 32(6): 1548-1551.)
11 谢志峰,叶冠桦,闫淑萁,等 . 基于生成对抗网络的HDR图像风格迁移技术[J]. 上海大学学报(自然科学版), 2018, 24(4):524-534. XIE Z F , YE G H , YAN S Q , et al . HDR image style transfer technique based on generative adversarial networks[J]. Journal of Shanghai University (Natural Science), 2018, 24(4): 524-534.
12 LI Y , FANG C , YANG J , et al . Universal style transfer via feature transforms[EB/OL]. [2019-04-05].https://arxiv.org/pdf/1705.08086.pdf.
13 YANG J , WRIGHT J , HUANG T S , et al . Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.
14 YANG J , WANG Z , LIN Z , et al . Coupled dictionary training for image super-resolution[J]. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478.
15 SALVADOR J , PéEREZ-PELLITERO E . Naive Bayes super-resolution forest[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 325-333.
16 TIMOFTE R , DE SMET V , GOOL L VAN . A+: adjusted anchored neighborhood regression for fast super-resolution[C]// Proceedings of the 2014 Asian Conference on Computer Vision, LNCS 9006. Cham: Springer, 2014: 111-126.
17 TIMOFTE R , ROTHE R , GOOL L VAN . Seven ways to improve example-based single image super resolution[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1865-1873.
18 HUANG J B , SINGH A , AHUJA N . Single image super-resolution from transformed self-exemplars[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 5197-5206.
19 DONG C , LOY C C, HE K , et al . Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.
20 WANG Z , LIU D , YANG J , et al . Deep networks for image super-resolution with sparse prior[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 370-378.
21 LIU D , WANG Z , WEN B , et al . Robust single image super-resolution via deep networks with sparse prior[J]. IEEE Transactions on Image Processing, 2016, 25(7): 3194-3207.
22 SHI W , CABALLERO J , HUSZAR F , et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1874-1883.
23 LAI W S , HUANG J B , AHUJA N , et al . Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017:5835-5843.
24 REN P , SUN W , LUO C , et al . Clustering-oriented multiple convolutional neural networks for single image super-resolution[J]. Cognitive Computation, 2018, 10(1):165-178.
25 TAI Y , YANG J , LIU X . Image super-resolution via deep recursive residual network[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2790-2798.
26 TONG T , LI G , LIU X , et al . Image super-resolution using dense skip connections[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017:4809-4817.
27 DONG C , LOY C C, TANG X . Accelerating the super-resolution convolutional neural network[C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9906. Cham: Springer, 2016: 391-407.
28 KIM J , LEE J K, LEE K M . Accurate image super-resolution using very deep convolutional networks[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:1646-1654.
29 KIM J , LEE J K, LEE K M . Deeply-recursive convolutional network for image super-resolution[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1637-1645.
30 TAI Y , YANG J , LIU X , et al . MemNet: a persistent memory network for image restoration[C]// Proceedings of the 2017 IEEE Conference on Computer Vision. Piscataway: IEEE, 2017: 4549-4557.
31 HE K , ZHANG X , REN S , et al . Delving deep into rectifiers: surpassing human-level performance on ImageNet classification[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 1026-1034.
32 SZEGEDY C , IOFFE S , VANHOUCKE V , et al. Inception-v 4, inception-ResNet and the impact of residual connections on learning[C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence Palo Alto, CA: AAAI Press, 2017: 4278-4284.
33 LIM B, SON S, KIM H , et al . Enhanced deep residual networks for single image super-resolution[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2017:1132-1140.
34 MARTIN D , FOWLKES C , TAL D, et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]// Proceedings of the 8th IEEE International Conference on Computer Vision. Piscataway: IEEE, 2001: 416-423.
35 BEVILACQUA M , ROUMY A , GUILLEMOT C , et al . Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]// Proceedings of the 2012 British Machine Vision Conference. Durham: BMVA, 2012: No.135.
36 ZEYDE R , ELAD M , PROTTER M . On single image scale-up using sparse-representations[C]// Proceedings of the 2010 International conference on curves and surfaces. Berlin: Springer, 2010: 711-730.
37 YAMANAKA J , KUWASHIMA S , KURITA T . Fast and accurate image super resolution by deep CNN with skip connection and network in network[EB/OL]. [2019-04-05].https://arxiv.org/ftp/arxiv/papers/1707/1707.05425.pdf.
38 MAO X , SHEN C , YANG Y . Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems.Cambridge, MA: MIT Press, 2016: 2810-2818.
39 RAMACHANDRAN P , ZOPH B , LE Q V . Swish: a self-gated activation function[EB/OL]. [2019-04-05].https://arxiv.org/pdf/1710.05941.pdf.