《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (5): 1570-1576.DOI: 10.11772/j.issn.1001-9081.2021050742
所属专题: 多媒体计算与计算机仿真
收稿日期:
2021-05-10
修回日期:
2021-09-09
接受日期:
2021-10-14
发布日期:
2022-03-08
出版日期:
2022-05-10
通讯作者:
徐岩
作者简介:
王汇丰(1995—),男,甘肃天水人,硕士研究生,CCF会员,主要研究方向:计算机视觉、图像处理基金资助:
Huifeng WANG, Yan XU(), Yiming WEI, Huizhen WANG
Received:
2021-05-10
Revised:
2021-09-09
Accepted:
2021-10-14
Online:
2022-03-08
Published:
2022-05-10
Contact:
Yan XU
About author:
WANG Huifeng, born in 1995,M. S. candidate. His researchinterests include computer vision,image processing.Supported by:
摘要:
现有的图像超分辨率重建算法可以改善图像整体视觉效果或者提升重建图像的客观评价值,然而对图像感知效果和客观评价值的均衡提升效果不佳,且重建图像缺乏高频信息,导致纹理模糊。针对上述问题,提出了一种基于并联卷积与残差网络的图像超分辨率重建算法。首先,以并联结构为整体框架,在并联结构上采用不同卷积组合来丰富特征信息,并加入跳跃连接来进一步丰富特征信息并融合输出,从而提取更多的高频信息。其次,引入自适应残差网络以补充信息并优化网络性能。最后,采用感知损失来提升恢复后图像的整体质量。实验结果表明,相较于超分辨率卷积神经网络(SRCNN)、深度超分辨率重建网络(VDSR)和超分辨率生成对抗网络(SRGAN)等算法,所提算法在重建图像上有更好的表现,其放大效果图的细节纹理更清晰。在客观评价上,所提算法在4倍重建时的峰值信噪比(PSNR)和结构相似性(SSIM)相较于SRGAN分别平均提升了0.25 dB和0.019。
中图分类号:
王汇丰, 徐岩, 魏一铭, 王会真. 基于并联卷积与残差网络的图像超分辨率重建[J]. 计算机应用, 2022, 42(5): 1570-1576.
Huifeng WANG, Yan XU, Yiming WEI, Huizhen WANG. Image super-resolution reconstruction based on parallel convolution and residual network[J]. Journal of Computer Applications, 2022, 42(5): 1570-1576.
Layer_name | Kernel_size | Input_channel | Output_channel |
---|---|---|---|
FEM1 | (3,3) | 3 | 32 |
(3,3) | 32 | 32 | |
(3,3) | 32 | 32 | |
(3,3) | 32 | 32 | |
FEM2 | (3,3) | 3 | 32 |
(5,5) | 32 | 32 | |
(3,3) | 32 | 32 | |
(1,1) | 3 | 3 | |
ARM1 | (3,3) | 3 | 3 |
ARM2 | (3,3) | 3 | 3 |
Conv1 | (3,3) | 131 | 3 |
Conv2 | (3,3) | 99 | 3 |
Conv2 | (3,3) | 3 | 3 |
Conv3 | (3,3) | 3 | 3 |
Conv4 | (3,3) | 6 | 3 |
Upconv | (3,3) | 3 | 3 |
表1 网络结构和卷积核参数
Tab. 1 Network structure and convolution kernel parameters
Layer_name | Kernel_size | Input_channel | Output_channel |
---|---|---|---|
FEM1 | (3,3) | 3 | 32 |
(3,3) | 32 | 32 | |
(3,3) | 32 | 32 | |
(3,3) | 32 | 32 | |
FEM2 | (3,3) | 3 | 32 |
(5,5) | 32 | 32 | |
(3,3) | 32 | 32 | |
(1,1) | 3 | 3 | |
ARM1 | (3,3) | 3 | 3 |
ARM2 | (3,3) | 3 | 3 |
Conv1 | (3,3) | 131 | 3 |
Conv2 | (3,3) | 99 | 3 |
Conv2 | (3,3) | 3 | 3 |
Conv3 | (3,3) | 3 | 3 |
Conv4 | (3,3) | 6 | 3 |
Upconv | (3,3) | 3 | 3 |
算法 | Set5 | Set14 | BSD100 | Urban100 | ||||
---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
Bicubic | 33.59 | 0.938 | 30.21 | 0.866 | 29.54 | 0.841 | 26.85 | 0.837 |
SRCNN | 36.56 | 0.945 | 32.42 | 0.906 | 31.34 | 0.887 | 29.45 | 0.894 |
VDSR | 37.51 | 0.951 | 32.99 | 0.913 | 31.92 | 0.898 | 30.75 | 0.916 |
SRGAN | 37.91 | 0.936 | 33.24 | 0.901 | 32.03 | 0.904 | 31.42 | 0.925 |
文献[ | 36.77 | 0.958 | 32.52 | 0.907 | 31.76 | 0.901 | 30.64 | 0.919 |
本文算法 | 37.85 | 0.959 | 33.36 | 0.910 | 32.85 | 0.901 | 31.95 | 0.928 |
表2 不同算法在2倍上的PSNR、SSIM对比
Tab. 2 Comparison of PSNR and SSIM of different algorithms on ×2
算法 | Set5 | Set14 | BSD100 | Urban100 | ||||
---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
Bicubic | 33.59 | 0.938 | 30.21 | 0.866 | 29.54 | 0.841 | 26.85 | 0.837 |
SRCNN | 36.56 | 0.945 | 32.42 | 0.906 | 31.34 | 0.887 | 29.45 | 0.894 |
VDSR | 37.51 | 0.951 | 32.99 | 0.913 | 31.92 | 0.898 | 30.75 | 0.916 |
SRGAN | 37.91 | 0.936 | 33.24 | 0.901 | 32.03 | 0.904 | 31.42 | 0.925 |
文献[ | 36.77 | 0.958 | 32.52 | 0.907 | 31.76 | 0.901 | 30.64 | 0.919 |
本文算法 | 37.85 | 0.959 | 33.36 | 0.910 | 32.85 | 0.901 | 31.95 | 0.928 |
算法 | Set5 | Set14 | BSD100 | Urban100 | ||||
---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
Bicubic | 30.40 | 0.868 | 27.47 | 0.778 | 27.20 | 0.739 | 24.46 | 0.741 |
SRCNN | 32.67 | 0.909 | 29.24 | 0.820 | 28.38 | 0.787 | 26.23 | 0.799 |
VDSR | 33.65 | 0.921 | 29.77 | 0.831 | 28.81 | 0.796 | 27.13 | 0.827 |
SRGAN | 33.95 | 0.902 | 29.85 | 0.827 | 28.94 | 0.807 | 27.82 | 0.836 |
文献[ | 32.89 | 0.913 | 29.41 | 0.828 | 28.74 | 0.801 | 27.03 | 0.822 |
本文算法 | 33.98 | 0.931 | 30.30 | 0.842 | 29.34 | 0.821 | 28.20 | 0.864 |
表3 不同算法在3倍上的PSNR、SSIM对比
Tab. 3 Comparison of PSNR and SSIM of different algorithms on ×3
算法 | Set5 | Set14 | BSD100 | Urban100 | ||||
---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
Bicubic | 30.40 | 0.868 | 27.47 | 0.778 | 27.20 | 0.739 | 24.46 | 0.741 |
SRCNN | 32.67 | 0.909 | 29.24 | 0.820 | 28.38 | 0.787 | 26.23 | 0.799 |
VDSR | 33.65 | 0.921 | 29.77 | 0.831 | 28.81 | 0.796 | 27.13 | 0.827 |
SRGAN | 33.95 | 0.902 | 29.85 | 0.827 | 28.94 | 0.807 | 27.82 | 0.836 |
文献[ | 32.89 | 0.913 | 29.41 | 0.828 | 28.74 | 0.801 | 27.03 | 0.822 |
本文算法 | 33.98 | 0.931 | 30.30 | 0.842 | 29.34 | 0.821 | 28.20 | 0.864 |
算法 | Set5 | Set14 | BSD100 | Urban100 | ||||
---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
Bicubic | 28.42 | 0.810 | 26.10 | 0.704 | 25.96 | 0.669 | 23.15 | 0.659 |
SRCNN | 30.49 | 0.862 | 27.61 | 0.754 | 26.91 | 0.712 | 24.53 | 0.713 |
VDSR | 31.34 | 0.881 | 28.01 | 0.769 | 27.29 | 0.726 | 25.18 | 0.753 |
SRGAN | 31.90 | 0.881 | 29.32 | 0.780 | 27.55 | 0.734 | 26.07 | 0.781 |
文献[ | 30.51 | 0.887 | 29.30 | 0.771 | 27.15 | 0.721 | 25.23 | 0.756 |
本文算法 | 31.92 | 0.893 | 29.95 | 0.782 | 27.63 | 0.736 | 26.34 | 0.798 |
表4 不同算法在4倍上的PSNR、SSIM对比
Tab. 4 Comparison of PSNR and SSIM of different algorithms on ×4
算法 | Set5 | Set14 | BSD100 | Urban100 | ||||
---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
Bicubic | 28.42 | 0.810 | 26.10 | 0.704 | 25.96 | 0.669 | 23.15 | 0.659 |
SRCNN | 30.49 | 0.862 | 27.61 | 0.754 | 26.91 | 0.712 | 24.53 | 0.713 |
VDSR | 31.34 | 0.881 | 28.01 | 0.769 | 27.29 | 0.726 | 25.18 | 0.753 |
SRGAN | 31.90 | 0.881 | 29.32 | 0.780 | 27.55 | 0.734 | 26.07 | 0.781 |
文献[ | 30.51 | 0.887 | 29.30 | 0.771 | 27.15 | 0.721 | 25.23 | 0.756 |
本文算法 | 31.92 | 0.893 | 29.95 | 0.782 | 27.63 | 0.736 | 26.34 | 0.798 |
算法 | img048 | img077 | img087 | |||
---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
OM | 31.101 | 0.645 | 30.331 | 0.567 | 31.710 | 0.694 |
OM+ARM1 | 31.586 | 0.702 | 30.751 | 0.699 | 32.235 | 0.725 |
OM+ARM1+ARM2 | 31.801 | 0.795 | 31.631 | 0.787 | 32.710 | 0.805 |
本文算法 | 32.011 | 0.886 | 31.964 | 0.887 | 33.383 | 0.874 |
表5 消融实验中不同算法的PSNR、SSIM对比
Tab. 5 Comparison of PSNR and SSIM of different algorithms in ablation experiment
算法 | img048 | img077 | img087 | |||
---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | |
OM | 31.101 | 0.645 | 30.331 | 0.567 | 31.710 | 0.694 |
OM+ARM1 | 31.586 | 0.702 | 30.751 | 0.699 | 32.235 | 0.725 |
OM+ARM1+ARM2 | 31.801 | 0.795 | 31.631 | 0.787 | 32.710 | 0.805 |
本文算法 | 32.011 | 0.886 | 31.964 | 0.887 | 33.383 | 0.874 |
1 | SHI W Z, CABALLERO J, LEDIG C, et al. Cardiac image super-resolution with global correspondence using multi-atlas PatchMatch [C]// Proceedings of the 2013 International Conference on Medical Image Computing and Computer-Assisted Intervention, LNCS 8151. Berlin: Springer, 2013: 9-16. |
2 | HU M G, WANG J F, GE Y. Super-resolution reconstruction of remote sensing images using multifractal analysis [J]. Sensors, 2009, 9(11): 8669-8683. 10.3390/s91108669 |
3 | ZHU S Y, ZENG B, LIU G H, et al. Image interpolation based on non-local geometric similarities [C]// Proceedings of the 2015 IEEE International Conference on Multimedia and Expo. Piscataway: IEEE, 2015: 1-6. 10.1109/icme.2015.7177417 |
4 | SCHULTZ R R, STEVENSON R L. Extraction of high-resolution frames from video sequences [J]. IEEE Transactions on Image Processing, 1996, 5(6): 996-1011. 10.1109/83.503915 |
5 | PELEG T, ELAD M. A statistical prediction model based on sparse representations for single image super-resolution [J]. IEEE Transactions on Image Processing, 2014, 23(6): 2569-2582. 10.1109/tip.2014.2305844 |
6 | DONG C, LOY C C, HE K M, et al. Learning a deep convolutional network for image super-resolution [C]// Proceedings of the 2014 European Conference on Computer Vision, LNCS 8692. Cham: Springer, 2014: 184-199. |
7 | KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1646-1654. 10.1109/cvpr.2016.182 |
8 | LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2017: 1132-1140. 10.1109/cvprw.2017.151 |
9 | ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2472-2481. 10.1109/cvpr.2018.00262 |
10 | 梁敏,王昊榕,张瑶,等.基于加速残差网络的图像超分辨率重建方法[J].计算机应用,2021,41(5):1438-1444. 10.11772/j.issn.1001-9081.2020091520 |
LIANG M, WANG H R, ZHANG Y, et al. Image super-resolution reconstruction method based on accelerated residual network [J]. Journal of Computer Applications, 2021, 41(5): 1438-1444. 10.11772/j.issn.1001-9081.2020091520 | |
11 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778. 10.1109/cvpr.2016.90 |
12 | 唐艳秋,潘泓,朱亚平,等.图像超分辨率重建研究综述[J].电子学报,2020,48(7):1407-1420. 10.3969/j.issn.0372-2112.2020.07.022 |
TANG Y Q, PAN H, ZHU Y P, et al. A survey of image super-resolution reconstruction [J]. Acta Electronica Sinica, 2020, 48(7): 1407-1420. 10.3969/j.issn.0372-2112.2020.07.022 | |
13 | ABUDU RAHIMAN V, GEORGE S N. Single image super resolution using neighbor embedding and statistical prediction model [J]. Computers and Electrical Engineering, 2017, 62(1): 281-292. 10.1016/j.compeleceng.2016.12.018 |
14 | ZEYDE R, ELAD M, PROTTER M. On single image scale-up using sparse-representations [C]// Proceedings of the 2010 International Conference on Curves and Surfaces, LNCS 6920. Berlin: Springer, 2010: 711-730. |
15 | MARTIN D, FOWLKES C, TAL D, et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics [C]// Proceedings of the 2001 8th IEEE International Conference on Computer Vision. Piscataway: IEEE, 2001: 416-423. 10.1109/iccv.2001.937655 |
16 | HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 5197-5206. 10.1109/cvpr.2015.7299156 |
17 | PEREYRA M, SCHNITER P, CHOUZENOUX É, et al. A survey of stochastic simulation and optimization methods in signal processing [J]. IEEE Journal of Selected Topics in Signal Processing, 2016, 10(2): 224-241. 10.1109/jstsp.2015.2496908 |
18 | YUAN Y, LIU S Y, ZHANG J W, et al. Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2018: 814-823. 10.1109/cvprw.2018.00113 |
19 | LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 105-114. 10.1109/cvpr.2017.19 |
20 | 南方哲,钱育蓉,行艳妮,等.基于深度学习的单图像超分辨率重建研究综述[J].计算机应用研究,2020,37(2):321-326. 10.19734/j.issn.1001-3695.2018.10.0610 |
NAN F Z, QIAN Y R, XING Y N, et al. Survey of single image super resolution based on deep learning [J]. Application Research of Computers, 2020, 37(2): 321-326. 10.19734/j.issn.1001-3695.2018.10.0610 | |
21 | 刘月峰,杨涵晰,蔡爽,等.基于改进卷积神经网络的单幅图像超分辨率重建方法[J].计算机应用,2019,39(5):1440-1447. 10.11772/j.issn.1001-9081.2018091887 |
LIU Y F, YANG H X, CAI S, et al. Single image super-resolution reconstruction method based on improved convolutional neural network [J]. Journal of Computer Applications, 2019, 39(5): 1440-1447. 10.11772/j.issn.1001-9081.2018091887 | |
22 | WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity [J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612. 10.1016/j.jvcir.2019.102655 |
[1] | 刘源泂, 何茂征, 黄益斌, 钱程. 基于ResNet50和改进注意力机制的船舶识别模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1935-1941. |
[2] | 郭琳, 刘坤虎, 马晨阳, 来佑雪, 徐映芬. 基于感受野扩展残差注意力网络的图像超分辨率重建[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1579-1587. |
[3] | 付顺旺, 陈茜, 李智, 王国美, 卢妤. 用于篡改图像检测和定位的双通道渐进式特征过滤网络[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1303-1309. |
[4] | 周景贤, 李希娜. 基于改进卷积神经网络和射频指纹的无人机检测与识别[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 876-882. |
[5] | 黄学雨, 贺怀宇, 林慧敏, 陈金水. 基于特征聚合的铜合金金相图分类识别方法[J]. 《计算机应用》唯一官方网站, 2023, 43(8): 2593-2601. |
[6] | 张慧斌, 冯丽萍, 郝耀军, 王一宁. 基于注意力机制和迁移学习的古壁画朝代识别[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1826-1832. |
[7] | 申利华, 李波. 基于特征金字塔网络和密集网络的肺部CT图像超分辨率重建[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1612-1619. |
[8] | 宋其洪, 刘建勋, 扈海泽, 张祥平. 基于协同融合网络的代码搜索模型[J]. 《计算机应用》唯一官方网站, 2023, 43(12): 3896-3902. |
[9] | 张志昂, 廖光忠. 基于U-Net的多尺度特征增强视网膜血管分割算法[J]. 《计算机应用》唯一官方网站, 2023, 43(10): 3275-3281. |
[10] | 廖列法, 李志明, 张赛赛. 基于深度残差网络的迭代量化哈希图像检索方法[J]. 《计算机应用》唯一官方网站, 2022, 42(9): 2845-2852. |
[11] | 贺怀清, 闫建青, 惠康华. 基于深度残差网络的轻量级人脸识别方法[J]. 《计算机应用》唯一官方网站, 2022, 42(7): 2030-2036. |
[12] | 张杨, 郝江波. 基于注意力机制和残差网络的恶意代码检测方法[J]. 《计算机应用》唯一官方网站, 2022, 42(6): 1708-1715. |
[13] | 董明宇, 严迪群. 基于ResNet的音频场景声替换造假的检测算法[J]. 《计算机应用》唯一官方网站, 2022, 42(6): 1724-1728. |
[14] | 张晔, 刘蓉, 刘明, 陈明. 基于多通道注意力机制的图像超分辨率重建网络[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1563-1569. |
[15] | 孙鹏翔, 毕利, 王俊杰. 基于改进深度残差网络的光伏板积灰程度识别[J]. 《计算机应用》唯一官方网站, 2022, 42(12): 3733-3739. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||