基于残差神经网络的图像超分辨率改进算法

doi:10.11772/j.issn.1001-9081.2017061461

计算机应用 ›› 2018, Vol. 38 ›› Issue (1): 246-254.DOI: 10.11772/j.issn.1001-9081.2017061461

• 虚拟现实与多媒体计算 • 上一篇下一篇

基于残差神经网络的图像超分辨率改进算法

王一宁, 秦品乐, 李传朋, 崔雨豪

中北大学大数据学院, 太原 030051

收稿日期:2017-06-14 修回日期:2017-08-10 出版日期:2018-01-10 发布日期:2018-01-22
通讯作者: 秦品乐
作者简介:王一宁(1992-),女,山西长治人,硕士研究生,主要研究方向:深度学习、机器视觉、数字图像处理;秦品乐(1978-),男,山西长治人,副教授,博士,主要研究方向:大数据、机器视觉、三维重建;李传朋(1991-),男,山东济南人,硕士研究生,主要研究方向:深度学习、机器视觉、数字图像处理;崔雨豪(1996-),男,浙江杭州人,主要研究方向:深度学习、数字图像处理。
基金资助:
山西省自然科学基金资助项目（2015011045）。

Improved algorithm of image super resolution based on residual neural network

WANG Yining, QIN Pinle, LI Chuanpeng, CUI Yuhao

School of Data Science, North University of China, Taiyuan Shanxi 030051, China

Received:2017-06-14 Revised:2017-08-10 Online:2018-01-10 Published:2018-01-22
Supported by:
This work is partially supported by the Natural Science Foundation of Shanxi Province (2015011045).

摘要/Abstract

摘要： 为更有效地提升图像的超分辨率（SR）效果，提出了一种多阶段级联残差卷积神经网络模型。首先，该模型采用了两阶段超分辨率图像重建方法先重建2倍超分辨率图像，再重建4倍超分辨率图像；其次，第一阶段与第二阶段皆使用残差层和跳层结构预测出高分辨率空间的纹理信息，由反卷积层分别重建出2倍与4倍大小的超分辨率图像；最后，以两阶段的结果分别构建多任务损失函数，利用第一阶段的损失指导第二阶段的损失，从而提高网络的训练速度，加强网络学习中的监督指导。实验结果表明，与bilinear算法、bicubic算法、基于卷积神经网络的图像超分辨率（SRCNN）算法和加速的超分辨率卷积神经网络（FSRCNN）算法相比，所提模型能更好地重建出图像的细节和纹理，避免了经过迭代之后造成的图像过度平滑，获得更高的峰值信噪比（PSNR）和平均结构相似度（MSSIM）。

关键词: 超分辨率, 深度学习, 残差块, 跳层, 反卷积, 多任务损失

Abstract: To efficiently improve the effects of image Super Resolution (SR), a multi-stage cascade residual convolution neural network model was proposed. Firstly, two-stage SR image reconstruction method was used to reconstruct the 2-times SR image and then reconstruct the 4-times SR image; secondly, residual layer and jump layer were used to predict the texture information of the high resolution space in the first and second stages, and deconvolution layer was used to reconstruct 2-times and 4-times SR images. Finally, two multi-task loss functions were constructed respectively by the results of two stages. And the loss of the first stage guided that of the second one, which accelerated the network training and enhanced the supervision and guidance of the network learning. The experimental results show that compared with bilinear algorithm, bicubic algorithm, Super Resolution using Convolutional Neural Network (SRCNN) algorithm and Fast Super Resolution Convolutional Neural Network (FSRCNN) algorithm, the proposed model can better construct the details and texture of images, which avoids the image over smoothing after iterating, and achieves higher Peak Signal-to-Noise Ratio (PSNR) and Mean Structural SIMilarity (MSSIM).

Key words: Super Resolution (SR), Deep Learning (DL), residual block, jump layer, deconvolution, multi-task loss

中图分类号:

TP391.413

王一宁, 秦品乐, 李传朋, 崔雨豪. 基于残差神经网络的图像超分辨率改进算法[J]. 计算机应用, 2018, 38(1): 246-254.

WANG Yining, QIN Pinle, LI Chuanpeng, CUI Yuhao. Improved algorithm of image super resolution based on residual neural network[J]. Journal of Computer Applications, 2018, 38(1): 246-254.

参考文献

[1] PELED S, YESHURUN Y. Superresolution in MRI:application to human white matter fiber tract visualization by diffusion tensor imaging[J]. Magnetic Resonance in Medicine Official Journal of the Society of Magnetic Resonance in Medicine, 2001, 45(1):29.
[2] SHI W, CABALLERO J, LEDIG C, et al. Cardiac image super-resolution with global correspondence using multi-atlas patchmatch[C]//MICCAI 2013:Proceedings of the 2013 International Conference on Medical Image Computing and Computer-Assisted Intervention. Berlin:Springer, 2013:9-16.
[3] THORNTON M W, ATKINSON P M, HOLLAND D A. Sub-pixel mapping of rural land cover objects from fine spatial resolution satellite sensor imagery using super-resolution pixel-swapping[J]. International Journal of Remote Sensing, 2006, 27(3):473-491.
[4] GUNTURK B K, BATUR A U, ALTUNBASAK Y, et al. Eigenface-domain super-resolution for face recognition[J]. IEEE Transactions on Image Processing, 2003, 12(5):597-606.
[5] ZHANG L, ZHANG H, SHEN H, et al. A super-resolution reconstruction algorithm for surveillance images[J]. Signal Processing, 2010, 90(3):848-859.
[6] 苏衡,周杰,张志浩.超分辨率图像重建方法综述[J].自动化学报,2013,39(8):1202-1213.(SU H, ZHOU J, ZHANG Z H. A summary of super-resolution image reconstruction methods[J]. Acta Automatica Sinica, 2013, 39(8):1202-1213.)
[7] KEYS R. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics Speech & Signal Processing, 1981, 29(6):1153-1160.
[8] 袁琪,荆树旭.改进的序列图像超分辨率重建方法[J].计算机应用,2009,29(12):3310-3313.(YUAN Q, JIN S X. Improved sequence image super-resolution reconstruction method[J]. Journal of Computer Applications, 2009, 29(12):3310-3313.)
[9] TIMOFTE R, DE V, GOOL L V. Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2013:1920-1927.
[10] TIMOFTE R, SMET V D, GOOL L V. A+:adjusted anchored neighborhood regression for fast super-resolution[C]//Proceedings of the 2014 Asian Conference on Computer Vision. Berlin:Springer, 2014:111-126.
[11] YANG C Y, YANG M H. Fast direct super-resolution by simple functions[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2013:561-568.
[12] YANG J, WRIGHT J, HUANG T, et al. Image super-resolution as sparse representation of raw image patches[C]//CVPR 2008:Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2008:1-8.
[13] YANG J, WRIGHT J, HUANG T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873.
[14] WANG J, ZHU S, GONG Y. Resolution enhancement based on learning the sparse association of image patches[J]. Pattern Recognition Letters, 2010, 31(1):1-10.
[15] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 2012 International Conference on Neural Information Processing Systems. West Chester, OH:Curran Associates Inc., 2012:1097-1105.
[16] DONG C, CHEN C L, HE K, et al. Learning a deep convolutional network for image super-resolution[C]//ECCV 2014:Proceedings of the 2014 European Conference on Computer Vision. Berlin:Springer, 2014:184-199.
[17] DONG C, CHEN C L, TANG X. Accelerating the super-resolution convolutional neural network[C]//ECCV 2016:Proceedings of the 2016 European Conference on Computer Vision. Berlin:Springer, 2016:391-407.
[18] SUN J, SHUM H Y. Image super-resolution using gradient profile prior:US9064476[P]. 2015-06-23.
[19] NAIR V, HINTON G E. Rectified linear units improve restricted boltzmann machines[C]//Proceedings of the 2010 International Conference on International Conference on Machine Learning. Madison, WI:Omnipress, 2010:807-814.
[20] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//CVPR 2016:Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2016:770-778.
[21] JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the 2016 European Conference on Computer Vision. Berlin:Springer, 2016:694-711.
[22] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2017-03-21]. https://arxiv.org/abs/1409.1556.
[23] REN S, HE K, GIRSHICK R, et al. Object detection networks on convolutional feature maps[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(7):1476-1481.
[24] SRIVASTAVA R K, GREFF K, SCHMIDHUBER J. Training very deep networks[EB/OL].[2017-03-20]. https://arxiv.org/abs/1507.06228.
[25] SAXE A M, MCCLELLAND J L, GANGULI S. Exact solutions to the nonlinear dynamics of learning in deep linear neural networks[EB/OL].[2017-03-12]. http://arxiv.org/abs/1312.6120.
[26] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2017-03-10]. https://arxiv.org/abs/1409.1556.
[27] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[C]//Proceedings of the 2012 British Machine Vision Conference.[S.l.]:BMVC Press, 2012:135-1-135-10.
[28] ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations[C]//Proceedings of the 2010 International Conference on Curves and Surfaces. Berlin:Springer, 2010:711-730.
[29] JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]//Proceedings of the 2016 European Conference on Computer Vision. Berlin:Springer, 2016:694-711.

基于残差神经网络的图像超分辨率改进算法

Improved algorithm of image super resolution based on residual neural network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[2]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[3]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[4]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[5]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[6]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[7]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[8]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[9]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[10]	李亚芳, 梁烨, 冯韦玮, 祖宝开, 康玉健. 基于社区优化的深度网络嵌入方法[J]. 计算机应用, 2021, 41(7): 1956-1963.
[11]	杜炎, 吕良福, 焦一辰. 基于模糊推理的模糊原型网络[J]. 计算机应用, 2021, 41(7): 1885-1890.
[12]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.
[13]	侯笑晗, 金国栋, 谭力宁, 薛远亮. 基于自适应和最优特征的合成孔径雷达舰船检测方法[J]. 计算机应用, 2021, 41(7): 2150-2155.
[14]	王月, 江逸茗, 兰巨龙. 基于改进三元组网络和K近邻算法的入侵检测[J]. 计算机应用, 2021, 41(7): 1996-2002.
[15]	史杨潇, 章军, 陈鹏, 王兵. 基于轻量级网络的钢铁表面缺陷分类[J]. 计算机应用, 2021, 41(6): 1836-1841.