Light-weight automatic residual scaling network for image super-resolution reconstruction

doi:10.11772/j.issn.1001-9081.2019112014

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (5): 1446-1452.DOI: 10.11772/j.issn.1001-9081.2019112014

• Virtual reality and multimedia computing • Previous Articles Next Articles

Light-weight automatic residual scaling network for image super-resolution reconstruction

DAI Qiang¹, CHENG Xi², WANG Yongmei¹, NIU Ziwei¹, LIU Fei¹

1.School of Information and Computer, Anhui Agricultural University, HefeiAnhui 230036, China
2.School of Computer Science and Engineering, Nanjing University of Science and Technology, NanjingJiangsu 210094, China

Received:2019-11-28 Revised:2020-01-06 Online:2020-05-10 Published:2020-05-15
Contact: WANG Yongmei, born in 1974, M.S., associate professor. Her research interests include intelligent information processing, agricultural internet of things.
About author:DAI Qiang, born in 1997. His research interests include deep learning, computer vision.CHENG Xi, born in 1996, Ph. D. candidate. His research interests include deep learning, computer vision, digital image processing.WANG Yongmei, born in 1974, M.S., associate professor. Her research interests include intelligent information processing, agricultural internet of things.NIU Ziwei, born in 2000. His research interests include deep learning, digital image processing.LIU Fei, born in 1989, M.S., research assistant. His research interests include legal theory, agricultural product quality and safety.
Supported by:
This work is partially supported by the Quality Engineering Project of Anhui Provincial Department of Education (2016ckjh080), the Industry University Research Collaborative Education Project of Ministry of Education (201702126125), the National Undergraduate Training Program for Innovation and Entrepreneurship (201910364073), the Intelligent Operation and Whole Process Information Project of Grain Production in the Middle of Jianghuai Region (11004836).

基于轻量自动残差缩放网络的图像超分辨率重建

代强¹, 程曦², 王永梅¹, 牛子未¹, 刘飞¹

1.安徽农业大学信息与计算机学院，合肥 230036
2.南京理工大学计算机科学与工程学院，南京 210094

通讯作者: 王永梅（1974—）
作者简介:代强（1997—），男，安徽合肥人，主要研究方向：深度学习、计算机视觉；程曦（1996—），男，江苏南京人，博士研究生，CCF会员，主要研究方向：深度学习、计算机视觉、数字图像处理；王永梅（1974—），女，安徽合肥人，副教授，硕士，主要研究方向：智能信息处理、农业物联网；牛子未（2000—），男，安徽淮北人，主要研究方向：深度学习、数字图像处理；刘飞（1989—），男，安徽合肥人，助理研究员，硕士，主要研究方向：法学理论、农产品质量安全。
基金资助:
安徽省教育厅质量工程项目（2016ckjh080）；教育部产学研协同育人项目（201702126125）；大学生创新创业训练项目（201910364073）；江淮中部粮食作物生产智能化作业与全程信息项目（11004836）。

Abstract

Abstract:

Recently, deep learning has been a hot research topic in the field of image super-resolution due to the excellent performance of deep convolutional neural networks. Many large-scale models with very deep structures have been proposed. However, in practical applications, the hardware of ordinary personal computers or smart terminals are obviously not suitable for large-scale deep neural network models. A light-weight Network with Automatic Residual Scaling (ARSN) for single image super-resolution was proposed, which has fewer layers and parameters compared with many other deep learning based methods. In addition, the specified residual blocks and skip connections in this network were utilized for residual scaling, global and local residual learning. The results on test datasets show that this model achieves state-of-the-art performance on both reconstruction quality and running speed. The proposed network achieves good results in terms of performance, speed and hardware consumption， and has high practical value.

Key words: deep learning, super-resolution, residual scaling, skip connection, residual network

摘要：

近年来，由于深度卷积神经网络的出色性能，深度学习已成为图像超分辨率领域的研究热点，已经有许多具有很深结构的大型模型被提出。而在实际应用中，普通个人计算机或智能终端的硬件显然不适合大规模深度神经网络模型。提出了一种针对单幅图像超分辨率且具有自动残差缩放功能的轻量级网络(ARSN)，与许多基于深度学习的方法相比，它的层和参数更少。此外，该网络中有特殊的残差块和跳跃连接用来进行残差缩放以及全局和局部残差学习。根据测试数据集结果，该网络在重建质量和运行速度上都达到了非常优异的性能。所提出的网络在性能、速度和硬件消耗方面均取得了良好的效果，具有较高的实用价值。

关键词: 深度学习, 超分辨率, 残差缩放, 跳跃连接, 残差网络

CLC Number:

TP391.41

DAI Qiang, CHENG Xi, WANG Yongmei, NIU Ziwei, LIU Fei. Light-weight automatic residual scaling network for image super-resolution reconstruction[J]. Journal of Computer Applications, 2020, 40(5): 1446-1452.

代强, 程曦, 王永梅, 牛子未, 刘飞. 基于轻量自动残差缩放网络的图像超分辨率重建[J]. 计算机应用, 2020, 40(5): 1446-1452.

References

1 TAYLOR J G , CUTSURIDIS V . Saliency, attention, active visual search, and picture scanning[J]. Cognitive Computation, 2011, 3(1): 1-3.
2 ABOUDIB A , GRIPON V , COPPIN G . A biologically inspired framework for visual information processing and an application on modeling bottom-up visual attention[J]. Cognitive Computation, 2016, 8(6): 1007-1026.
3 KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[C]// Proceedings of the 25th International Conference on Neural Information Processing Systems. New York: Curran Associates Inc., 2012: 1097-1105.
4 SIMONYAN K , ZISSERMAN A . Very deep convolutional networks for large-scale image recognition[EB/OL]. [2019-04-05].https://arxiv.org/pdf/1409.1556.pdf.
5 SRIVASTAVA R K , GREFF K , SCHMIDHUBER J . Highway networks[EB/OL]. [2019-04-05].http://de.arxiv.org/pdf/1505.00387.
6 HE K , ZHANG X , REN S , et al . Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
7 REN S , HE K , GIRSHICK R , et al . Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149.
8 DAI J , LI Y , HE K , et al . R-FCN: object detection via region-based fully convolutional networks[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: Curran Associates Inc., 2016: 379-387.
9 SHELHAMER E , LONG J , DARRELL T . Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
10 彭晏飞,孙鲁 . 基于图像分割的语义标注方法[J]. 计算机应用, 2012,32(6): 1548-1551. (PENG Y F, SUN L. Semantic annotation based on image segmentation[J]. Journal of Computer Applications, 2012, 32(6): 1548-1551.)
11 谢志峰,叶冠桦,闫淑萁,等 . 基于生成对抗网络的HDR图像风格迁移技术[J]. 上海大学学报(自然科学版), 2018, 24(4):524-534. XIE Z F , YE G H , YAN S Q , et al . HDR image style transfer technique based on generative adversarial networks[J]. Journal of Shanghai University (Natural Science), 2018, 24(4): 524-534.
12 LI Y , FANG C , YANG J , et al . Universal style transfer via feature transforms[EB/OL]. [2019-04-05].https://arxiv.org/pdf/1705.08086.pdf.
13 YANG J , WRIGHT J , HUANG T S , et al . Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11): 2861-2873.
14 YANG J , WANG Z , LIN Z , et al . Coupled dictionary training for image super-resolution[J]. IEEE Transactions on Image Processing, 2012, 21(8): 3467-3478.
15 SALVADOR J , PéEREZ-PELLITERO E . Naive Bayes super-resolution forest[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 325-333.
16 TIMOFTE R , DE SMET V , GOOL L VAN . A+: adjusted anchored neighborhood regression for fast super-resolution[C]// Proceedings of the 2014 Asian Conference on Computer Vision, LNCS 9006. Cham: Springer, 2014: 111-126.
17 TIMOFTE R , ROTHE R , GOOL L VAN . Seven ways to improve example-based single image super resolution[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1865-1873.
18 HUANG J B , SINGH A , AHUJA N . Single image super-resolution from transformed self-exemplars[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 5197-5206.
19 DONG C , LOY C C, HE K , et al . Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307.
20 WANG Z , LIU D , YANG J , et al . Deep networks for image super-resolution with sparse prior[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 370-378.
21 LIU D , WANG Z , WEN B , et al . Robust single image super-resolution via deep networks with sparse prior[J]. IEEE Transactions on Image Processing, 2016, 25(7): 3194-3207.
22 SHI W , CABALLERO J , HUSZAR F , et al . Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1874-1883.
23 LAI W S , HUANG J B , AHUJA N , et al . Deep Laplacian pyramid networks for fast and accurate super-resolution[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017:5835-5843.
24 REN P , SUN W , LUO C , et al . Clustering-oriented multiple convolutional neural networks for single image super-resolution[J]. Cognitive Computation, 2018, 10(1):165-178.
25 TAI Y , YANG J , LIU X . Image super-resolution via deep recursive residual network[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 2790-2798.
26 TONG T , LI G , LIU X , et al . Image super-resolution using dense skip connections[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017:4809-4817.
27 DONG C , LOY C C, TANG X . Accelerating the super-resolution convolutional neural network[C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9906. Cham: Springer, 2016: 391-407.
28 KIM J , LEE J K, LEE K M . Accurate image super-resolution using very deep convolutional networks[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:1646-1654.
29 KIM J , LEE J K, LEE K M . Deeply-recursive convolutional network for image super-resolution[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1637-1645.
30 TAI Y , YANG J , LIU X , et al . MemNet: a persistent memory network for image restoration[C]// Proceedings of the 2017 IEEE Conference on Computer Vision. Piscataway: IEEE, 2017: 4549-4557.
31 HE K , ZHANG X , REN S , et al . Delving deep into rectifiers: surpassing human-level performance on ImageNet classification[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 1026-1034.
32 SZEGEDY C , IOFFE S , VANHOUCKE V , et al. Inception-v 4, inception-ResNet and the impact of residual connections on learning[C]// Proceedings of the 31st AAAI Conference on Artificial Intelligence Palo Alto, CA: AAAI Press, 2017: 4278-4284.
33 LIM B, SON S, KIM H , et al . Enhanced deep residual networks for single image super-resolution[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2017:1132-1140.
34 MARTIN D , FOWLKES C , TAL D, et al . A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]// Proceedings of the 8th IEEE International Conference on Computer Vision. Piscataway: IEEE, 2001: 416-423.
35 BEVILACQUA M , ROUMY A , GUILLEMOT C , et al . Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]// Proceedings of the 2012 British Machine Vision Conference. Durham: BMVA, 2012: No.135.
36 ZEYDE R , ELAD M , PROTTER M . On single image scale-up using sparse-representations[C]// Proceedings of the 2010 International conference on curves and surfaces. Berlin: Springer, 2010: 711-730.
37 YAMANAKA J , KUWASHIMA S , KURITA T . Fast and accurate image super resolution by deep CNN with skip connection and network in network[EB/OL]. [2019-04-05].https://arxiv.org/ftp/arxiv/papers/1707/1707.05425.pdf.
38 MAO X , SHEN C , YANG Y . Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections[C]// Proceedings of the 30th International Conference on Neural Information Processing Systems.Cambridge, MA: MIT Press, 2016: 2810-2818.
39 RAMACHANDRAN P , ZOPH B , LE Q V . Swish: a self-gated activation function[EB/OL]. [2019-04-05].https://arxiv.org/pdf/1710.05941.pdf.

[1]	CHEN Chengrui, SUN Ning, HE Shibiao, LIAO Yong. Deep learning-based joint channel estimation and equalization algorithm for C-V2X communications [J]. Journal of Computer Applications, 2021, 41(9): 2687-2693.
[2]	XIE Defeng, JI Jianmin. Syntax-enhanced semantic parsing with syntax-aware representation [J]. Journal of Computer Applications, 2021, 41(9): 2489-2495.
[3]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[4]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[5]	ZHENG Zhiqiang, HU Xin, WENG Zhi, WANG Yuhe, CHENG Xi. Cattle eye image feature extraction method based on improved DenseNet [J]. Journal of Computer Applications, 2021, 41(9): 2780-2784.
[6]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[7]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[8]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[9]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[10]	HE Zhenghai, XIAN Yantuan, WANG Meng, YU Zhengtao. Case reading comprehension method combining syntactic guidance and character attention mechanism [J]. Journal of Computer Applications, 2021, 41(8): 2427-2431.
[11]	LI Yafang, LIANG Ye, FENG Weiwei, ZU Baokai, KANG Yujian. Deep network embedding method based on community optimization [J]. Journal of Computer Applications, 2021, 41(7): 1956-1963.
[12]	WANG Yue, JIANG Yiming, LAN Julong. Intrusion detection based on improved triplet network and K-nearest neighbor algorithm [J]. Journal of Computer Applications, 2021, 41(7): 1996-2002.
[13]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[14]	HOU Xiaohan, JIN Guodong, TAN Lining, XUE Yuanliang. Synthetic aperture radar ship detection method based on self-adaptive and optimal features [J]. Journal of Computer Applications, 2021, 41(7): 2150-2155.
[15]	DU Yan, LYU Liangfu, JIAO Yichen. Fuzzy prototype network based on fuzzy reasoning [J]. Journal of Computer Applications, 2021, 41(7): 1885-1890.

Light-weight automatic residual scaling network for image super-resolution reconstruction

基于轻量自动残差缩放网络的图像超分辨率重建

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics