基于拉普拉斯金字塔生成对抗网络的图像超分辨率重建算法

doi:10.11772/j.issn.1001-9081.2020081299

计算机应用 ›› 2021, Vol. 41 ›› Issue (4): 1020-1026.DOI: 10.11772/j.issn.1001-9081.2020081299

所属专题： CCF第35届中国计算机应用大会（CCF NCCA 2020）

• CCF第35届中国计算机应用大会（CCF NCCA 2020） • 上一篇下一篇

基于拉普拉斯金字塔生成对抗网络的图像超分辨率重建算法

段友祥¹, 张含笑¹, 孙歧峰¹, 孙友凯²

1. 中国石油大学(华东)计算机科学与技术学院, 山东青岛 266580;
2. 中国石化胜利油田分公司物探研究院, 山东东营 257000

收稿日期:2020-08-25 修回日期:2020-09-15 出版日期:2021-04-10 发布日期:2020-11-05
通讯作者: 张含笑
作者简介:段友祥（1964—），男，山东东营人，教授，博士，CCF会员，主要研究方向：网络与服务计算、计算机技术在油气领域的应用；张含笑（1995—），女，山东潍坊人，硕士研究生，主要研究方向：人工智能；孙歧峰（1976—），男，山东东营人，讲师，博士，主要研究方向：计算机技术在油气领域的应用；孙友凯（1974—），男，山东陵城人，高级工程师，硕士，主要研究方向：云计算。
基金资助:
中央高校基本科研业务费专项资金资助项目（20CX05017A）；中石油重大科技项目（ZD2019-183-006）。

Image super-resolution reconstruction algorithm based on Laplacian pyramid generative adversarial network

DUAN Youxiang¹, ZHANG Hanxiao¹, SUN Qifeng¹, SUN Youkai²

1. College of Computer Science and Technology, China University of Petroleum, Qingdao Shandong 266580, China;
2. Geophysical Research Institute of Sinopec Shengli Oilfield Company Limited, Dongying Shandong 257000, China

Received:2020-08-25 Revised:2020-09-15 Online:2021-04-10 Published:2020-11-05
Supported by:
This work is partially supported by the Fundamental Research Funds for the Central Universities (20CX05017A), the Major Scientific and Technological Project of CNPC (ZD2019-183-006).

摘要/Abstract

摘要： 针对目前的图像超分辨率重建算法中存在的大尺度因子的重建效果较差、不同尺度的图像重建均需要单独训练等问题，提出一种基于拉普拉斯金字塔生成对抗网络（GAN）的图像超分辨率重建算法。算法中的生成器使用金字塔结构实现多尺度的图像重建，从而以渐进上采样的方式降低了大尺度因子的学习难度，并在层与层之间使用密集连接加强特征传播，从而有效避免了梯度弥散问题。算法中使用马尔可夫判别器将输入数据映射为结果矩阵，并在训练的过程中引导生成器关注图像的局部特征，从而丰富了重建图像的细节。实验结果表明：在Set5等基准数据集上分别进行放大2倍、4倍、8倍的图像重建时，所提算法的平均峰值信噪比（PSNR）分别达到了33.97 dB、29.15 dB、25.43 dB，平均结构相似性（SSIM）分别达到了0.924、0.840、0.667，相比用于超分辨率重建的卷积神经网络（SRCNN）、深度拉普拉斯金字塔超分辨率重建网络（LapSRN）、用于超分辨率重建的生成对抗式网络（SRGAN）等其他算法有较大提升，且其重建的图像在主观视觉上保留了更多生动的纹理和小颗粒细节。

关键词: 超分辨率重建, 大尺度因子, 密集连接, 拉普拉斯金字塔, 生成对抗网络

Abstract: Concerning the problems of poor reconstructing performance with large-scale factors and requirement of separate training in image reconstruction with different scales in current image super-resolution reconstruction algorithms, an image super-resolution reconstruction algorithm based on Laplacian pyramid Generative Adversarial Network(GAN) was proposed. The pyramid structure generator of the proposed algorithm was used to realize the multi-scale image reconstruction, so as to reduce the difficulty in learning large-scale factors by progressive up-sampling, and dense connection was used between layers to enhance feature propagation, which effectively avoided the vanishing gradient problem. In the algorithm, Markovian discriminator was used to map the input data into the result matrix, and the generator was guided to pay attention to the local features of the image in the process of training, which enriched the details of the reconstructed images. Experimental results show that, when performing 2-times, 4-times and 8-times image reconstruction on Set5 and other benchmark datasets, the average Peak Signal-to-Noise Ratio(PSNR) of the proposed algorithm reaches 33.97 dB, 29.15 dB, 25.43 dB respectively, and the average Structural SIMilarity(SSIM) of the algorithm reaches 0.924, 0.840, 0.667 respectively, outperforming to those of other algorithms such as Super Resolution Convolutional Neural Network(SRCNN), fast and accurate image Super-Resolution with deep Laplacian pyramid Network(LapSRN) and Super-Resolution GAN(SRGAN), and the images reconstructed by the proposed algorithm retain more vivid textures and fine-grained details in subjective vision.

Key words: super-resolution reconstruction, large-scale factor, dense connection, Laplacian pyramid, Generative Adversarial Network (GAN)

中图分类号:

TP391

段友祥, 张含笑, 孙歧峰, 孙友凯. 基于拉普拉斯金字塔生成对抗网络的图像超分辨率重建算法[J]. 计算机应用, 2021, 41(4): 1020-1026.

DUAN Youxiang, ZHANG Hanxiao, SUN Qifeng, SUN Youkai. Image super-resolution reconstruction algorithm based on Laplacian pyramid generative adversarial network[J]. Journal of Computer Applications, 2021, 41(4): 1020-1026.

参考文献

[1] ZHANG S,LIANG G,PAN S,et al. A fast medical image super resolution method based on deep learning network[J]. IEEE Access,2019,7:12319-12327.
[2] YANG X,WU W,LIU K,et al. Long-distance object recognition with image super resolution:a comparative study[J]. IEEE Access,2018,6:13429-13438.
[3] KEYS R. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics, Speech, and Signal Processing,1981,29(6):1153-1160.
[4] 符祥, 郭宝龙. 图像插值技术综述[J]. 计算机工程与设计, 2009,30(1):141-144,193.(FU X,GUO B L. Overview of image interpolation technology[J]. Computer Engineering and Design,2009,30(1):141-144,193.)
[5] KARIMI E,KANGARLOO K,JAVADI S. A survey on superresolution methods for image reconstruction[J]. International Journal of Computer Applications,2014,90(3):32-39.
[6] LU Z,WU C,CHEN D,et al. Overview on image super resolution reconstruction[C]//Proceedings of the 26th Chinese Control and Decision Conference. Piscataway:IEEE,2014:2009-2014.
[7] 黄陶冶, 赵建伟, 周正华. 双层可变形卷积网络的超分辨率图像重建[J]. 计算机应用,2019,39(S2):68-74.(HUANG T Y, ZHAO J W,ZHOU Z H. Super-resolution image reconstruction using two-layer deformable convolution networks[J]. Journal of Computer Applications,2019,39(S2):68-74.)
[8] WANG Z,LIU D,YANG J,et al. Deep networks for image superresolution with sparse prior[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2015:370-378.
[9] 孙旭, 李晓光, 李嘉锋, 等. 基于深度学习的图像超分辨率复原研究进展[J]. 自动化学报,2017,43(5):697-709.(SUN X,LI X G,LI J F,et al. Review on deep learning based image superresolution restoration algorithms[J]. Acta Automatica Sinica, 2017,43(5):697-709.)
[10] DONG C,LOY C C,HE K,et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[11] 南方哲, 钱育蓉, 行艳妮, 等. 基于深度学习的单图像超分辨率重建研究综述[J]. 计算机应用研究,2020,37(2):321-326. (NAN F Z,QIAN Y R,XING Y N,et al. Survey of single image super resolution based on deep learning[J]. Application Research of Computers,2020,37(2):321-326.)
[12] LAI W S,HUANG J B,AHUJA N,et al. Deep Laplacian pyramid networks for fast and accurate super-resolution[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5835-5843.
[13] HA V K,REN J,XU X,et al. Deep learning based single image super-resolution:a survey[C]//Proceedings of the 9th International Conference on Brain Inspired Cognitive Systems, LNCS 10989. Cham:Springer,2018:106-119.
[14] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2014:2672-2680.
[15] LEDIG C,THEIS L,HUSZÁR F,et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:105-114.
[16] WANG X,YU K,WU S,et al. ESRGAN:enhanced superresolution generative adversarial networks[C]//Proceedings of the 15th European Conference on Computer Vision,LNCS 11133. Cham:Springer,2018:63-79.
[17] WANG Y, PERAZZI F, MCWILLIAMS B, et al. A fully progressive approach to single-image super-resolution[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE, 2018:977-986.
[18] LIM B,SON S,KIM H,et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1132-1140.
[19] ISOLA P,ZHU J Y,ZHOU T,et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:5967-5976.
[20] LAN Z,LIN M,LI X,et al. Beyond Gaussian pyramid:multiskip feature stacking for action recognition[C]//Proceedings of the 28th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:204-212.
[21] BURT P,ADELSON E. The Laplacian pyramid as a compact image code[J]. IEEE Transactions on Communications,1983,31(4):532-540.
[22] PARIS S,HASINOFF S W,KAUTZ J. Local Laplacian filters:edge-aware image processing with a Laplacian pyramid[J]. Communications of the ACM,2015,58(3):81-91.
[23] DENTON E,CHINTALA S,SZLAM A,et al. Deep generative image models using a Laplacian pyramid of adversarial networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2015:1486-1494.
[24] TONG T,LI G,LIU X,et al. Image super-resolution using dense skip connections[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:4809-4817.
[25] SHI W,CABALLERO J,HUSZÁR F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 29th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1874-1883.
[26] BIAN Y A,LI X,LIU Y,et al. Parallel coordinate descent newton method for efficient L1-regularized loss minimization[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019,30(11):3233-3245.
[27] BÜHLMANN P,YU B. Boosting with the L2 loss:regression and classification[J]. Journal of the American Statistical Association, 2003,98(462):324-339.
[28] NICOLSON A,PALIWAL K K. Deep learning for minimum meansquare error approaches to speech enhancement[J]. Speech Communication,2019,111:44-55.

基于拉普拉斯金字塔生成对抗网络的图像超分辨率重建算法

Image super-resolution reconstruction algorithm based on Laplacian pyramid generative adversarial network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	管其杰, 张挺, 李德亚, 周绍景, 杜奕. 基于多分辨率生成对抗网络的空间数据不确定性重建方法[J]. 计算机应用, 2021, 41(8): 2306-2311.
[2]	孙潇, 徐金东. 基于级联生成对抗网络的遥感图像去雾方法[J]. 计算机应用, 2021, 41(8): 2440-2444.
[3]	汤桂花, 孙磊, 毛秀青, 戴乐育, 胡永进. 基于深度对齐网络的生成对抗网络伪造人脸检测[J]. 计算机应用, 2021, 41(7): 1922-1927.
[4]	牛康力, 谌雨章, 沈君凤, 曾张帆, 潘永才, 王绎冲. 基于深度学习的双通道夜视图像复原方法[J]. 计算机应用, 2021, 41(6): 1775-1784.
[5]	王先武, 张挺, 吉欣, 杜奕. 基于带梯度惩罚深度卷积生成对抗网络的页岩三维数字岩心重构方法[J]. 计算机应用, 2021, 41(6): 1805-1811.
[6]	李衍志, 范勇, 高琳. 基于形态流的石油钻井水流异常检测[J]. 计算机应用, 2021, 41(6): 1842-1848.
[7]	井贝贝, 郭嘉, 王丽清, 陈静, 丁洪伟. 结合降噪卷积神经网络和条件生成对抗网络的图像双重盲降噪算法[J]. 计算机应用, 2021, 41(6): 1767-1774.
[8]	孙鹤立, 孙玉柱, 张晓云. 基于生成对抗网络的事件描述生成[J]. 计算机应用, 2021, 41(5): 1256-1261.
[9]	梁敏, 王昊榕, 张瑶, 李杰. 基于加速残差网络的图像超分辨率重建方法[J]. 计算机应用, 2021, 41(5): 1438-1444.
[10]	郭茂祖, 杨倩楠, 赵玲玲. 基于条件Wassertein生成对抗网络的图像生成[J]. 计算机应用, 2021, 41(5): 1432-1437.
[11]	欧莉莉, 邵峰晶, 孙仁诚, 隋毅. 基于半监督方法的脑梗死图像识别[J]. 计算机应用, 2021, 41(4): 1221-1226.
[12]	王永金, 左羽, 吴恋, 崔忠伟, 赵晨洁. 基于注意力机制的图像超分辨率重建[J]. 计算机应用, 2021, 41(3): 845-850.
[13]	李虹霞, 秦品乐, 闫寒梅, 曾建潮, 鲍骞月, 柴锐. 基于面部特征图对称的人脸正面化生成对抗网络算法[J]. 计算机应用, 2021, 41(3): 714-720.
[14]	张亚, 金鑫, 江倩, 李昕洁, 董云云, 姚绍文. 基于自动编码器的深度伪造图像检测方法[J]. 计算机应用, 2021, 41(10): 2985-2990.
[15]	邱耀儒, 孙为军, 黄永慧, 唐瑜祺, 张浩川, 吴俊鹏. 基于生成对抗网络联合时空模型的行人重识别方法[J]. 计算机应用, 2020, 40(9): 2493-2498.