结合结构自相似性和卷积网络的单幅图像超分辨率

doi:10.11772/j.issn.1001-9081.2017081920

计算机应用 ›› 2018, Vol. 38 ›› Issue (3): 854-858.DOI: 10.11772/j.issn.1001-9081.2017081920

• 虚拟现实与多媒体计算 • 上一篇下一篇

结合结构自相似性和卷积网络的单幅图像超分辨率

向文, 张灵, 陈云华, 姬秋敏

广东工业大学计算机学院, 广州 511400

收稿日期:2017-08-07 修回日期:2017-10-12 出版日期:2018-03-10 发布日期:2018-03-07
通讯作者: 向文
作者简介:向文(1993-),男,湖南怀化人,硕士研究生,主要研究方向:图像处理、模式识别;张灵(1968-),女,广西南宁人,教授,博士,主要研究方向:数据挖掘、计算机视觉、无线传感;陈云华(1977-),女,湖北仙桃人,讲师,博士,主要研究方向:计算机视觉、模式识别、深度学习;姬秋敏(1992-),女,山东济宁人,硕士研究生,主要研究方向:图像处理、模式识别。
基金资助:
广东省自然科学基金博士启动项目（2014A030310169）；广东省自然科学基金面上项目（2016A030313703，2016A030313713）；广东省科技计划项目（2016B030305002）；广东省交通运输厅科技项目（科技-2016-02-030）。

Single image super resolution combining with structural self-similarity and convolution networks

XIANG Wen, ZHANG Ling, CHEN Yunhua, JI Qiumin

School of Computers, Guangdong University of Technology, Guangzhou Guangdong 511400, China

Received:2017-08-07 Revised:2017-10-12 Online:2018-03-10 Published:2018-03-07
Supported by:
This work is partially supported by the Guangdong Natural Science Foundation Doctoral Program (2014A030310169), the Natural Science Foundation Project of Guangdong Province (2016A030313703, 2016A030313713), the Science and Technology Project of Guangdong Province (2016B030305002), the Science and Technology Project of Guangdong Department of Transportation (Sci. and Tech.-2016-02-030).

摘要/Abstract

摘要： 针对单幅图像超分辨率（SR）复原病态逆问题，在重建过程边缘细节丢失导致的模糊，提出一种结合结构自相似和卷积网络的单幅图像超分辨率算法。首先，通过将尺度分解获得待重构图片样本的自身结构相似性，结合外部数据库样本结合作为训练样本，可以解决样本过于分散的问题；其次，将样本输入卷积神经网络（CNN）进行训练学习，得到单幅图像超分辨率的先验知识；然后，利用非局部约束项自适应选择最优字典重建图像；最后，利用迭代反投影算法对图像超分辨率效果进一步提升。实验结果表明，与双三次插值（Bicubic）方法、K-SVD算法和基于卷积神经网络的图像超分辨率（SRCNN）方法等优秀算法相比，所提算法可以得到边缘更为清晰的超分辨率重建效果。

关键词: 超分辨率, 结构自相似性, 深度卷积网络, 正则化, 块匹配

Abstract: Aiming at the ill-posed inverse problem of single-image Super Resolution (SR) restoration, a single image super resolution algorithm combining with structural self-similarity and convolution networks was proposed. Firstly, the self-structure similarity of samples to be reconstructed was obtained by scaling decomposition. Combined with external database samples as training samples, the problem of over-dispersion of samples could be solved. Secondly, the sample was input into a Convolution Neural Network (CNN) for training and learning, and the prior knowledge of the super resolution of the single image was obtained. Then, the optimal dictionary was used to reconstruct the image by using a nonlocal constraint. Finally, an iterative backprojection algorithm was used to further improve the image super resolution effect. The experimental results show that compared with the excellent algorithms such as Bicubic, K-SVD (Singular Value Decomposition of k iterations) algorithm and Super-Resolution Convolution Neural Network (SRCNN) algorithm, the proposed algorithm can get super-resolution reconstruction with clearer edges.

Key words: Super Resolution (SR), structural self-similarity, deep convolution network, regularization, block matching

中图分类号:

TP391.4

向文, 张灵, 陈云华, 姬秋敏. 结合结构自相似性和卷积网络的单幅图像超分辨率[J]. 计算机应用, 2018, 38(3): 854-858.

XIANG Wen, ZHANG Ling, CHEN Yunhua, JI Qiumin. Single image super resolution combining with structural self-similarity and convolution networks[J]. Journal of Computer Applications, 2018, 38(3): 854-858.

参考文献

[1] LI Z, HE H, WANG R, et al. Single image super-resolution via directional group sparsity and directional features[J]. IEEE Transactions on Image Processing, 2015, 24(9):2874-2888.
[2] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[3] YANG J, WRIGHT J, HUANG T S, et al. Image super-resolution via spare representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873.
[4] TIMOFTE R, de SMET V, van GOOL L. Anchored neighborhood regression for fast example-based super-resolution[C]//ICCV'13:Proceedings of the 2013 IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2013:1920-1927.
[5] TIMOFTE R, de SMET V, GOOL L. A+:adjusted anchored neighborhood regression for fast super resolution[C]//ACCV 2014:Proceedings of the 2014 Asian Conference on Computer Vision, LNCS 9006. Berlin:Springer-Verlag, 2014:111-126.
[6] YANG C Y, YANG M H. Fast direct super-resolution by simple functions[C]//ICCV'13:Proceedings of the 2013 IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2013:561-568.
[7] DAI D, TIMOFTE R, van GOOL L. Jointly optimized regressors for image super-resolution[J]. Computer Graphics Forum, 2015, 34(2):95-104.
[8] FREEDMAN G, FATTAL R. Image and video upscaling from local self-examples[J]. ACM Translations on Graphics, 2011, 30(2):Article No. 12.
[9] PROTTER M, ELAD M, TAKEDA H, et al. Generalizing the nonlocal-means to super-resolution reconstruction[J]. IEEE Transactions on Image Processing, 2009, 18(1):36-51.
[10] MAIRAL J, BACH F, PONCE J, et al. Non-local sparse models for image restoration[C]//ICCV'09:Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2009:2272-2279.
[11] DONG C, LOY C C, HE K, et al. Learning a deep convolutional network for image super resolution[C]//ECCV 2014:Proceedings of the 2014 European Conference on Computer Vision, LNCS 8692. Berlin:Springer-Verlag, 2014:184-199.
[12] GKASNER D, BAGON S, IRANI M. Super-resolution from a single image[C]//ICCV'09:Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2009:349-356.
[13] CANDOCIA F M, PRINCIPE J C. Super-resolution of images based on local correlations[J]. IEEE Transactions on Neural Networks, 1999, 10(2):372-380.
[14] DONG W, ZHANG L, SHI G, et al. Nonlocally centralized sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2013, 22(4):1620-1630.
[15] YOU X, XUE W, LEI J, et al.Single image super-resolution with non-local balanced low-rank matrix restoration[C]//ICPR 2016:Proceedings of the 201623rd International Conference on Pattern Recognition. Washington, DC:IEEE Computer Society, 2016:1255-1260.
[16] XU J, ZHANG L, ZUO W, et al. Patch group based nonlocal self-similarity prior learning for image denoising[C]//ICCV'15:Proceedings of the 2015 IEEE International Conference on Computer Vision. Washington, DC:IEEE Computer Society, 2015:244-252.
[17] TEKALP A M, OZKAN M K, SEZAN M I. High-resolution image reconstruction from lower-resolution image sequences and space-varying image restoration[C]//ICASSP'92:Proceedings of the 1992 IEEE International Conference on Acoustics, Speech and Signal Processing. Washington, DC:IEEE Computer Society, 1992:169-172.
[18] LU Y, IMANURA M. Pyramid-based super-resolution of the undersampled and subpixel shifted image sequence[J]. International Journal on Systems Technology, 2002, 12(6):254-263.
[19] AHARON M, ELAD M, BRUCKSTEIN A. K-SVD:an algorithm for designing overcomplete dictionaries for sparse representation[J]. IEEE Transactions on Signal Processing, 2006, 54(11):4311-4322.

结合结构自相似性和卷积网络的单幅图像超分辨率

Single image super resolution combining with structural self-similarity and convolution networks

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	牛康力, 谌雨章, 沈君凤, 曾张帆, 潘永才, 王绎冲. 基于深度学习的双通道夜视图像复原方法[J]. 计算机应用, 2021, 41(6): 1775-1784.
[2]	林筠超, 万源. 基于图结构优化的自适应多度量非监督特征选择方法[J]. 计算机应用, 2021, 41(5): 1282-1289.
[3]	梁敏, 王昊榕, 张瑶, 李杰. 基于加速残差网络的图像超分辨率重建方法[J]. 计算机应用, 2021, 41(5): 1438-1444.
[4]	卞鹏程, 郑忠龙, 李明禄, 何依然, 王天翔, 张大伟, 陈丽媛. 基于注意力融合网络的视频超分辨率重建[J]. 计算机应用, 2021, 41(4): 1012-1019.
[5]	欧莉莉, 邵峰晶, 孙仁诚, 隋毅. 基于半监督方法的脑梗死图像识别[J]. 计算机应用, 2021, 41(4): 1221-1226.
[6]	段友祥, 张含笑, 孙歧峰, 孙友凯. 基于拉普拉斯金字塔生成对抗网络的图像超分辨率重建算法[J]. 计算机应用, 2021, 41(4): 1020-1026.
[7]	王永金, 左羽, 吴恋, 崔忠伟, 赵晨洁. 基于注意力机制的图像超分辨率重建[J]. 计算机应用, 2021, 41(3): 845-850.
[8]	贺杰安, 吴晓红, 何小海, 胡建蓉, 卿粼波. 结合图像增强和卷积神经网络的小麦不完善粒识别[J]. 计算机应用, 2021, 41(3): 911-916.
[9]	李萌, 秦品乐, 曾建潮, 李俊伯. 基于磁共振影像层间插值的超分辨率及多视角融合[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3362-3367.
[10]	高世伟, 张长柱, 王祝萍. 基于可分离金字塔的轻量级实时语义分割算法[J]. 计算机应用, 2021, 41(10): 2937-2944.
[11]	林静, 黄玉清, 李磊民. 基于球形矩匹配与特征判别的图像超分辨率重建[J]. 计算机应用, 2020, 40(8): 2345-2350.
[12]	汪志远, 降爱莲, 奥斯曼·穆罕默德. 基于正则互表示的无监督特征选择方法[J]. 计算机应用, 2020, 40(7): 1896-1900.
[13]	胡雪影, 郭海儒, 朱蓉. 基于混合深度卷积网络的图像超分辨率重建[J]. 计算机应用, 2020, 40(7): 2069-2076.
[14]	胡高鹏, 陈子鎏, 王晓明, 张开放. 基于深度渐进式反投影注意力网络的图像超分辨率重建[J]. 计算机应用, 2020, 40(7): 2077-2083.
[15]	代强, 程曦, 王永梅, 牛子未, 刘飞. 基于轻量自动残差缩放网络的图像超分辨率重建[J]. 计算机应用, 2020, 40(5): 1446-1452.