《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (7): 2280-2287.DOI: 10.11772/j.issn.1001-9081.2022060877
• 多媒体计算与计算机仿真 • 上一篇
收稿日期:
2022-06-16
修回日期:
2022-09-06
接受日期:
2022-09-08
发布日期:
2022-10-18
出版日期:
2023-07-10
通讯作者:
梁敏
作者简介:
梁敏(1979—),女,山西忻州人,副教授,博士,CCF会员,主要研究方向:图像处理、模式识别;基金资助:
Min LIANG(), Jiayi LIU, Jie LI
Received:
2022-06-16
Revised:
2022-09-06
Accepted:
2022-09-08
Online:
2022-10-18
Published:
2023-07-10
Contact:
Min LIANG
About author:
LIANG Min, born in 1979, Ph. D., associate professor. Her research interests include image processing, pattern recognition.Supported by:
摘要:
针对图像超分辨重建过程中原始高清图片与低质量图像之间缺乏依赖关系、深度网络中特征图信息不分主次重构导致的图像高频信息高精度重构困难的问题,提出一种融合迭代反馈与注意力机制的单幅图像超分辨重建方法。首先使用频率分解模块分别提取图像中的高、低频信息,并将二者分别处理,使网络重点关注提取出的高频细节部分,增强方法在图像细节上的复原能力;其次通过通道注意力机制将重建的重点放在有效特征所在的特征通道上,增强网络提取特征图信息的能力;然后采用迭代反馈的思想,在反复重建和比对过程中增加图像的还原程度;最后通过重建模块生成输出图像。在Set5、Set14、BSD100、Urban100和Manga109基准数据集上的2倍、4倍和8倍放大实验中,与主流超分辨率方法相比,所提方法表现出更优越的性能。在Manga109数据集的8倍放大实验中,相较于传统插值方法和基于卷积神经网络的图像超分辨率算法(SRCNN),所提方法的峰值信噪比(PSNR)均值分别提升了约3.01 dB和2.32 dB。实验结果表明:所提方法能够降低重建过程中出现的误差,并有效重建出更精细的高分辨率图像。
中图分类号:
梁敏, 刘佳艺, 李杰. 融合迭代反馈与注意力机制的图像超分辨重建方法[J]. 计算机应用, 2023, 43(7): 2280-2287.
Min LIANG, Jiayi LIU, Jie LI. Image super-resolution reconstruction method based on iterative feedback and attention mechanism[J]. Journal of Computer Applications, 2023, 43(7): 2280-2287.
频率分解模块 | 注意力模块 | PSNR/dB |
---|---|---|
× | × | 26.80 |
× | √ | 26.89 |
√ | × | 26.85 |
√ | √ | 26.90 |
表1 不同模型结构在测试集上的实验结果(α=8)
Tab. 1 Experimental results of different model structures on test set (α=8)
频率分解模块 | 注意力模块 | PSNR/dB |
---|---|---|
× | × | 26.80 |
× | √ | 26.89 |
√ | × | 26.85 |
√ | √ | 26.90 |
λ值 | PSNR/dB | λ值 | PSNR/dB |
---|---|---|---|
0 | 26.30 | 1 | 25.12 |
0.01 | 27.08 | 10 | 23.41 |
0.1 | 27.03 |
表2 不同λ值在General100数据集上的实验结果(α=4)
Tab. 2 Experimental results for different λ values on General100 dataset (α=4)
λ值 | PSNR/dB | λ值 | PSNR/dB |
---|---|---|---|
0 | 26.30 | 1 | 25.12 |
0.01 | 27.08 | 10 | 23.41 |
0.1 | 27.03 |
算法 | 放大 倍数 | Set5 | Set14 | BSD100 | Urban100 | Manga109 | |||||
---|---|---|---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | ||
Bicubic | 2× | 33.65 | 0.930 | 30.24 | 0.869 | 29.56 | 0.844 | 26.88 | 0.841 | 30.84 | 0.935 |
SRCNN[ | 36.66 | 0.954 | 32.45 | 0.906 | 31.36 | 0.888 | 29.52 | 0.895 | 35.72 | 0.968 | |
FSRCNN[ | 37.00 | 0.956 | 32.63 | 0.909 | 31.50 | 0.891 | 29.88 | 0.902 | — | — | |
VDSR[ | 37.74 | 0.959 | 32.97 | 0.913 | 31.90 | 0.896 | 30.77 | 0.914 | 37.16 | 0.974 | |
LapSRN[ | 37.52 | 0.929 | 33.08 | 0.913 | 31.80 | 0.895 | 31.05 | 0.910 | 37.53 | 0.974 | |
ADSR[ | 37.36 | 0.958 | 32.86 | 0.911 | 31.78 | 0.894 | 30.44 | 0.910 | — | — | |
DBPN[ | 37.60 | 0.959 | 33.18 | 0.914 | 31.94 | 0.897 | 31.20 | 0.918 | 37.57 | 0.974 | |
IFANet(本文方法) | 37.70 | 0.959 | 33.24 | 0.914 | 31.99 | 0.898 | 31.15 | 0.912 | 37.85 | 0.975 | |
Bicubic | 4× | 28.42 | 0.810 | 26.10 | 0.702 | 25.96 | 0.667 | 23.15 | 0.657 | 24.92 | 0.789 |
SRCNN[ | 30.49 | 0.862 | 27.61 | 0.751 | 26.91 | 0.710 | 24.53 | 0.722 | 27.66 | 0.858 | |
FSRCNN[ | 30.71 | 0.865 | 27.70 | 0.756 | 26.97 | 0.714 | 24.61 | 0.727 | 27.89 | 0.859 | |
VDSR[ | 31.53 | 0.883 | 28.03 | 0.767 | 27.29 | 0.725 | 25.18 | 0.752 | 28.82 | 0.886 | |
SRGAN[ | 29.46 | 0.838 | 26.60 | 0.718 | 25.74 | 0.666 | 24.50 | 0.736 | 27.79 | 0.856 | |
LapSRN[ | 31.54 | 0.885 | 28.09 | 0.770 | 27.31 | 0.727 | 25.21 | 0.756 | 29.09 | 0.890 | |
ADSR[ | 31.19 | 0.881 | 27.88 | 0.763 | 27.20 | 0.721 | 25.00 | 0.744 | — | — | |
DBPN[ | 31.76 | 0.887 | 28.39 | 0.778 | 27.48 | 0.733 | 25.71 | 0.772 | 30.22 | 0.902 | |
IFANet (本文方法) | 32.09 | 0.890 | 28.30 | 0.776 | 27.75 | 0.738 | 25.90 | 0.786 | 30.65 | 0.912 | |
Bicubic | 8× | 24.39 | 0.657 | 23.19 | 0.568 | 23.67 | 0.547 | 20.74 | 0.515 | 21.68 | 0.649 |
SRCNN[ | 25.33 | 0.689 | 23.85 | 0.593 | 24.13 | 0.565 | 21.29 | 0.543 | 22.37 | 0.682 | |
FSRCNN[ | 25.41 | 0.682 | 23.93 | 0.592 | 24.21 | 0.567 | 21.32 | 0.537 | 22.39 | 0.672 | |
VDSR[ | 25.72 | 0.711 | 24.21 | 0.609 | 24.37 | 0.576 | 21.54 | 0.560 | 22.83 | 0.707 | |
SRGAN[ | 23.04 | 0.626 | 21.57 | 0.495 | 21.78 | 0.442 | 19.64 | 0.468 | 20.42 | 0.625 | |
LapSRN[ | 26.15 | 0.737 | 24.35 | 0.620 | 24.54 | 0.585 | 21.81 | 0.580 | 23.39 | 0.734 | |
ADSR[ | 25.60 | 0.710 | 24.18 | 0.600 | 24.31 | 0.572 | 21.40 | 0.552 | 22.75 | 0.698 | |
DBPN[ | 26.43 | 0.748 | 24.39 | 0.623 | 24.60 | 0.589 | 22.01 | 0.592 | 23.97 | 0.756 | |
IFANet (本文方法) | 26.90 | 0.770 | 24.70 | 0.635 | 24.70 | 0.590 | 22.25 | 0.605 | 24.69 | 0.779 |
表3 测试集上不同算法在不同放大因子下的PSNR及SSIM均值
Tab. 3 Mean values of PSNR/SSIM of different algorithms on test sets under different amplification factors
算法 | 放大 倍数 | Set5 | Set14 | BSD100 | Urban100 | Manga109 | |||||
---|---|---|---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | ||
Bicubic | 2× | 33.65 | 0.930 | 30.24 | 0.869 | 29.56 | 0.844 | 26.88 | 0.841 | 30.84 | 0.935 |
SRCNN[ | 36.66 | 0.954 | 32.45 | 0.906 | 31.36 | 0.888 | 29.52 | 0.895 | 35.72 | 0.968 | |
FSRCNN[ | 37.00 | 0.956 | 32.63 | 0.909 | 31.50 | 0.891 | 29.88 | 0.902 | — | — | |
VDSR[ | 37.74 | 0.959 | 32.97 | 0.913 | 31.90 | 0.896 | 30.77 | 0.914 | 37.16 | 0.974 | |
LapSRN[ | 37.52 | 0.929 | 33.08 | 0.913 | 31.80 | 0.895 | 31.05 | 0.910 | 37.53 | 0.974 | |
ADSR[ | 37.36 | 0.958 | 32.86 | 0.911 | 31.78 | 0.894 | 30.44 | 0.910 | — | — | |
DBPN[ | 37.60 | 0.959 | 33.18 | 0.914 | 31.94 | 0.897 | 31.20 | 0.918 | 37.57 | 0.974 | |
IFANet(本文方法) | 37.70 | 0.959 | 33.24 | 0.914 | 31.99 | 0.898 | 31.15 | 0.912 | 37.85 | 0.975 | |
Bicubic | 4× | 28.42 | 0.810 | 26.10 | 0.702 | 25.96 | 0.667 | 23.15 | 0.657 | 24.92 | 0.789 |
SRCNN[ | 30.49 | 0.862 | 27.61 | 0.751 | 26.91 | 0.710 | 24.53 | 0.722 | 27.66 | 0.858 | |
FSRCNN[ | 30.71 | 0.865 | 27.70 | 0.756 | 26.97 | 0.714 | 24.61 | 0.727 | 27.89 | 0.859 | |
VDSR[ | 31.53 | 0.883 | 28.03 | 0.767 | 27.29 | 0.725 | 25.18 | 0.752 | 28.82 | 0.886 | |
SRGAN[ | 29.46 | 0.838 | 26.60 | 0.718 | 25.74 | 0.666 | 24.50 | 0.736 | 27.79 | 0.856 | |
LapSRN[ | 31.54 | 0.885 | 28.09 | 0.770 | 27.31 | 0.727 | 25.21 | 0.756 | 29.09 | 0.890 | |
ADSR[ | 31.19 | 0.881 | 27.88 | 0.763 | 27.20 | 0.721 | 25.00 | 0.744 | — | — | |
DBPN[ | 31.76 | 0.887 | 28.39 | 0.778 | 27.48 | 0.733 | 25.71 | 0.772 | 30.22 | 0.902 | |
IFANet (本文方法) | 32.09 | 0.890 | 28.30 | 0.776 | 27.75 | 0.738 | 25.90 | 0.786 | 30.65 | 0.912 | |
Bicubic | 8× | 24.39 | 0.657 | 23.19 | 0.568 | 23.67 | 0.547 | 20.74 | 0.515 | 21.68 | 0.649 |
SRCNN[ | 25.33 | 0.689 | 23.85 | 0.593 | 24.13 | 0.565 | 21.29 | 0.543 | 22.37 | 0.682 | |
FSRCNN[ | 25.41 | 0.682 | 23.93 | 0.592 | 24.21 | 0.567 | 21.32 | 0.537 | 22.39 | 0.672 | |
VDSR[ | 25.72 | 0.711 | 24.21 | 0.609 | 24.37 | 0.576 | 21.54 | 0.560 | 22.83 | 0.707 | |
SRGAN[ | 23.04 | 0.626 | 21.57 | 0.495 | 21.78 | 0.442 | 19.64 | 0.468 | 20.42 | 0.625 | |
LapSRN[ | 26.15 | 0.737 | 24.35 | 0.620 | 24.54 | 0.585 | 21.81 | 0.580 | 23.39 | 0.734 | |
ADSR[ | 25.60 | 0.710 | 24.18 | 0.600 | 24.31 | 0.572 | 21.40 | 0.552 | 22.75 | 0.698 | |
DBPN[ | 26.43 | 0.748 | 24.39 | 0.623 | 24.60 | 0.589 | 22.01 | 0.592 | 23.97 | 0.756 | |
IFANet (本文方法) | 26.90 | 0.770 | 24.70 | 0.635 | 24.70 | 0.590 | 22.25 | 0.605 | 24.69 | 0.779 |
1 | DONG C, LOY C C, HE K M, et al. Image super-resolution using deep convolutional networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(2): 295-307. 10.1109/tpami.2015.2439281 |
2 | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10) [2022-06-01]. . |
3 | DONG C, LOY C C, TANG X O. Accelerating the super-resolution convolutional neural network [C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9906. Cham: Springer, 2016: 391-407. |
4 | KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1646-1654. 10.1109/cvpr.2016.182 |
5 | LAI W S, HUANG J B, AHUJA N, et al. Deep Laplacian pyramid networks for fast and accurate super-resolution [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 5835-5843. 10.1109/cvpr.2017.618 |
6 | HARIS M, SHAKHNAROVICH G, UKITA N. Deep back-projection networks for super-resolution [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 1664-1673. 10.1109/cvpr.2018.00179 |
7 | HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141. 10.1109/cvpr.2018.00745 |
8 | WANG F, JIANG M Q, QIAN C, et al. Residual attention network for image classification [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6450-6458. 10.1109/cvpr.2017.683 |
9 | WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7794-7803. 10.1109/cvpr.2018.00813 |
10 | LIU D, WEN B H, FAN Y C, et al. Non-local recurrent network for image restoration [C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2018: 1680-1689. |
11 | ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks [C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018: 294-310. |
12 | GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets [C]// Proceedings of the 27th International Conference on Neural Information Processing Systems — Volume 2. Cambridge: MIT Press, 2014: 2672-2680. |
13 | JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution [C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9906. Cham: Springer, 2016: 694-711. |
14 | LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 105-114. 10.1109/cvpr.2017.19 |
15 | SHI W Z, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1874-1883. 10.1109/cvpr.2016.207 |
16 | HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778. 10.1109/cvpr.2016.90 |
17 | CHANG J R, CHEN Y S. Pyramid stereo matching network [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 5410-5418. 10.1109/cvpr.2018.00567 |
18 | ANWAR S, KHAN S, BARNES N. A deep journey into super-resolution: a survey[J]. ACM Computing Surveys, 2020, 53(3): No.60. 10.1145/3390462 |
19 | ANWAR S, BARNES N. Densely residual Laplacian super-resolution[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(3): 1192-1204. 10.1109/tpami.2020.3021088 |
20 | BELL S, UPCHURCH P, SNAVELY N, et al. Material recognition in the wild with the materials in context database [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 3479-3487. 10.1109/cvpr.2015.7298970 |
21 | 梁敏,王昊榕,张瑶,等.基于加速残差网络的图像超分辨率重建方法[J].计算机应用, 2021, 41(5): 1438-1444. |
LIANG M, WANG H R, ZHANG Y, et al. Image super-resolution reconstruction method based on accelerated residual network[J]. Journal of Computer Applications, 2021, 41(5): 1438-1444. |
[1] | 拓雨欣, 薛涛. 融合指针网络与关系嵌入的三元组联合抽取模型[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2116-2124. |
[2] | 岑黎彬, 李靖东, 林淳波, 王晓玲. 基于深度自回归模型的近似查询处理方法[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2034-2039. |
[3] | 魏远, 林彦, 郭晟楠, 林友芳, 万怀宇. 融合出发地与目的地时空相关性的城市区域间出租车需求预测[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2100-2106. |
[4] | 李忠雨, 孙浩东, 李娇. 轻量化篮球裁判手势识别算法[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2173-2181. |
[5] | 叶坤佩, 熊熙, 丁哲. 基于领域融合和时间权重的招工推荐模型[J]. 《计算机应用》唯一官方网站, 2023, 43(7): 2133-2139. |
[6] | 郑智雄, 刘建华, 孙水华, 徐戈, 林鸿辉. 融合多窗口局部信息的方面级情感分析模型[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1796-1802. |
[7] | 王辉, 李建红. 基于Transformer的三维模型小样本识别方法[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1750-1758. |
[8] | 张慧斌, 冯丽萍, 郝耀军, 王一宁. 基于注意力机制和迁移学习的古壁画朝代识别[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1826-1832. |
[9] | 秦静, 马雪倩, 高福杰, 季长清, 汪祖民. 基于步态分析的帕金森病辅助诊断方法综述[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1687-1695. |
[10] | 张奕, 王真梅. 图自动编码器上二阶段融合实现的环状RNA-疾病关联预测[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1979-1986. |
[11] | 陈一驰, 陈斌. 计算机视觉中的终身学习综述[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1785-1795. |
[12] | 方可, 刘蓉, 魏驰宇, 张心月, 刘杨. 复杂场景下的行人跌倒检测算法[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1811-1817. |
[13] | 鲁斌, 柳杰林. 基于特征增强的三维点云语义分割[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1818-1825. |
[14] | 靳鑫, 刘仰川, 朱叶晨, 张子健, 高欣. 基于残差编解码-生成对抗网络的正弦图修复的稀疏角度锥束CT图像重建[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1950-1957. |
[15] | 董润婷, 吴利, 王晓英, 曹腾飞, 黄建强, 管琴, 吴洁瑕. 深度学习在天气预报领域的应用分析及研究进展综述[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1958-1968. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||