基于极深卷积神经网络的人脸超分辨率重建算法

doi:10.11772/j.issn.1001-9081.2017092378

计算机应用 ›› 2018, Vol. 38 ›› Issue (4): 1141-1145.DOI: 10.11772/j.issn.1001-9081.2017092378

• 虚拟现实与多媒体计算 • 上一篇下一篇

基于极深卷积神经网络的人脸超分辨率重建算法

孙毅堂, 宋慧慧, 张开华, 严飞

江苏省大数据分析技术重点实验室(南京信息工程大学), 南京 210044

收稿日期:2017-10-09 修回日期:2017-11-16 出版日期:2018-04-10 发布日期:2018-04-09
通讯作者: 宋慧慧
作者简介:孙毅堂(1992-),男,江苏苏州人,硕士研究生,主要研究方向:图像超分辨率重建;宋慧慧(1986-),女,山东聊城人,教授,博士,主要研究方向:遥感图像处理;张开华(1983-),男,山东日照人,教授,博士,CCF会员,主要研究方法:图像分割、目标跟踪;严飞(1983-),男,江苏南京人,讲师,博士,主要研究方向:图像处理。
基金资助:
国家自然科学基金资助项目（41501377，61605083）；江苏省自然科学基金资助项目（BK20150906，BK20170040）。

Face super-resolution via very deep convolutional neural network

SUN Yitang, SONG Huihui, ZHANG Kaihua, YAN Fei

Jiangsu Key Laboratory of Big Data Analysis Technology(Najing University of Information Science and Technology), Nanjing Jiangsu 210044, China

Received:2017-10-09 Revised:2017-11-16 Online:2018-04-10 Published:2018-04-09
Supported by:
This work is partially supported by the National Natural Science Foundation of China (41501377, 61605083), the Natural Science Foundation of Jiangsu Province (BK20150906, BK20170040).

摘要/Abstract

摘要： 针对多种放大倍数的人脸超分辨率重建问题，提出一种基于极深卷积神经网络的人脸超分辨率重建方法，并通过实验发现增加网络深度能够有效提升人脸重建的精度。首先，设计一个包含20个卷积层的网络从低分辨率图片和高分辨率图片之间学习一种端到端的映射关系，并通过在网络结构中将多个小的滤波器进行多次串联以扩大提取纹理信息的范围。其次，引入了残差学习的方法来解决随着深度的提升细节信息丢失的问题。另外，将不同放大因子的低分辨率人脸图片融合到一个训练集中训练，使得该卷积网络能够解决不同放大因子的人脸超分辨率重建问题。在CASPEAL测试集上的结果显示，该极深卷积神经网络的方法比基于双三次插值的人脸重建方法在峰值信噪比（PSNR）和结构相似度上有2.7 dB和2%的提升，和SRCNN的方法比较也有较大的提升，在精度和视觉改善方面都有较大提升。这显示了更深的网络结构能够在重建中取得更好的结果。

关键词: 超分辨率重建, 卷积神经网络, 机器学习, 深度学习, 残差学习

Abstract: For multiple scale factors of face super-resolution, a face super-resolution method based on very deep convolutional neural network was proposed; and through experiments, it was found that the increase of network depth can effectively improve the accuracy of face reconstruction. Firstly, a network that consists of 20 convolution layers were designed to learn an end-to-end mapping between the low-resolution images and the high-resolution images, and many small filters were cascaded to extract more textural information. Secondly, a residual-learning method was introduced to solve the problem of detail information loss caused by increasing depth. In addition, the low-resolution face images with multiple scale factors were merged to one training set to enable the network to achieve the face super resolution with multiple scale factors. The results on the CASPEAL test dataset show that the proposed method based on this very deep convolutional neural network has 2.7 dB increasement in Peak Signal-to-Noise Ratio (PSNR), and 2% increasement in structural similarity compared to the Bicubic based face reconstruction method. Compared with the SRCNN method, there is also a greater improvement. as well as a greater improvement in accuracy and visual improvement. It means that deeper network structures can achieve better results in reconstruction.

Key words: super-resolution reconstruction, convolutional neural network, machine learning, deep learning, residual learning

中图分类号:

TP391.41

孙毅堂, 宋慧慧, 张开华, 严飞. 基于极深卷积神经网络的人脸超分辨率重建算法[J]. 计算机应用, 2018, 38(4): 1141-1145.

SUN Yitang, SONG Huihui, ZHANG Kaihua, YAN Fei. Face super-resolution via very deep convolutional neural network[J]. Journal of Computer Applications, 2018, 38(4): 1141-1145.

参考文献

[1] 苏衡, 周杰, 张志浩. 超分辨率图像重建方法综述[J]. 自动化学报, 2013, 39(8):1202-1213.(SU H, ZHOU J, ZHANG Z H. Survey of super-resolution image reconstruction methods[J]. Acta Automatica Sinica, 2013, 39(8):1202-1213.)
[2] CHANG H, YEUNG D-Y, XIONG Y. Super-resolution through neighbor embedding[C]//CVPR 2004:Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2004:275-282.
[3] BEVILACQUA M, ROUMY A, GUILLEMOT C, et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding[EB/OL].[2017-05-10]. http://eprints.imtlucca.it/2412/1/Bevilacqua_2012.pdf.
[4] YANG J, WRIGHT J, HUANG T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873.
[5] SCHULTER S, LEISTNER C, BISCHOF H. Fast and accurate image upscaling with super-resolution forests[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2015:3791-3799.
[6] DONG C, LOY C C, HE K, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2):295-307.
[7] WANG Z, LIU D, YANG J, et al. Deep networks for image super-resolution with sparse prior[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:370-378.
[8] CUI Z, CHANG H, SHANG S, et al. Deep network cascade for image super-resolution[C]//ECCV 2014:Proceedings of the 13th European Conference on Computer Vision. Berlin:Springer, 2014:49-64.
[9] NIE H, LU Y, IKRAM J. Face hallucination via convolution neural network[C]//Proceedings of the 2016 IEEE 28th International Conference on Tools with Artificial Intelligence. Piscataway, NJ:IEEE, 2017:485-489.
[10] KIM C, CHOI K, RA J B. Improvement on learning-based super-resolution by adopting residual information and patch reliability[C]//Proceedings of the 200916th IEEE International Conference on Image Processing. Piscataway, NJ:IEEE, 2010:1197-1200.
[11] TIMOFITE R, DE V, GOOL L V. Anchored neighborhood regression for fast example-based super-resolution[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2014:1920-1927.
[12] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2017-05-10]. https://arxiv.org/abs/1409.1556.
[13] TIMOFTE R, SMET V D, GOOL L V. A+:adjusted anchored neighborhood regression for fast super-resolution[C]//ACCV 2014:Proceedings of the 12th Asian Conference on Computer Vision. Berlin:Springer, 2014:111-126.
[14] TIMOFTE R, ROTHE R, GOOL L V. Seven ways to improve example-based single image super resolution[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2016:1865-1873.
[15] 练秋生, 张钧芹, 陈书贞. 基于两级字典与分频带字典的图像超分辨率算法[J]. 自动化学报, 2013, 39(8):1310-1320.(LIAN Q S, ZHANG J Q, CHEN S Z. Single image super-resolution algorithm based on two-stage and multi-frequency-band dictionaries[J]. Acta Automatica Sinica, 2013, 39(8):1310-1320.)
[16] BENGIO Y, SIMARD P, FRASCION P. Learning long-term dependencies with gradient descent is difficult[J]. IEEE Transactions on Neural Networks, 1994, 5(2):157-66.
[17] HE K, ZHANG X, REN S, et al. Delving deep into rectifiers:surpassing human-level performance on ImageNet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:1026-1034.
[18] 傅天宇, 金柳颀, 雷震, 等. 基于关键点逐层重建的人脸图像超分辨率方法[J]. 信号处理, 2016, 32(7):834-841.(FU T Y, JIN L Q, LEI Z, et al. Face super-resolution method based on key points layer by layer[J]. Journal of Signal Processing, 2016, 32(7):834-841.)

基于极深卷积神经网络的人脸超分辨率重建算法

Face super-resolution via very deep convolutional neural network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[2]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[3]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[4]	郭棉, 张锦友. 移动边缘计算环境中面向机器学习的计算迁移策略[J]. 计算机应用, 2021, 41(9): 2639-2645.
[5]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[6]	卞凌志, 王直杰. 基于增强多维多粒度级联森林的信用评分模型[J]. 计算机应用, 2021, 41(9): 2539-2544.
[7]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[8]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[9]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[10]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[11]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[12]	毛铭泽, 曹芮浩, 闫春钢. 基于权值多样性的半监督分类算法[J]. 计算机应用, 2021, 41(9): 2473-2480.
[13]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[14]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[15]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.