《计算机应用》唯一官方网站 ›› 2022, Vol. 42 ›› Issue (5): 1563-1569.DOI: 10.11772/j.issn.1001-9081.2021030498
收稿日期:
2021-04-02
修回日期:
2021-06-28
接受日期:
2021-07-01
发布日期:
2022-06-11
出版日期:
2022-05-10
通讯作者:
刘明
作者简介:
张晔(1997—),女,河北石家庄人,硕士研究生,主要研究方向:模式识别、智能信息处理基金资助:
Ye ZHANG1, Rong LIU1, Ming LIU2(), Ming CHEN1
Received:
2021-04-02
Revised:
2021-06-28
Accepted:
2021-07-01
Online:
2022-06-11
Published:
2022-05-10
Contact:
Ming LIU
About author:
ZHANG Ye, born in 1997,M. S. candidate. Her research interestsinclude pattern recognition,intelligent information processing.Supported by:
摘要:
针对现有的图像超分辨率重建方法存在生成图像纹理扭曲、细节模糊等问题,提出了一种基于多通道注意力机制的图像超分辨率重建网络。首先,该网络中的纹理提取模块通过设计多通道注意力机制并结合一维卷积实现跨通道的信息交互,以关注重要特征信息;然后,该网络中的纹理恢复模块引入密集残差块来尽可能恢复部分高频纹理细节,从而提升模型性能并产生优质重建图像。所提网络不仅能够有效提升图像的视觉效果,而且在基准数据集CUFED5上的结果表明所提网络与经典的基于卷积神经网络的超分辨率重建(SRCNN)方法相比,峰值信噪比(PSNR)和结构相似度(SSIM)分别提升了1.76 dB和0.062。实验结果表明,所提网络可提高纹理迁移的准确性,并有效提升生成图像的质量。
中图分类号:
张晔, 刘蓉, 刘明, 陈明. 基于多通道注意力机制的图像超分辨率重建网络[J]. 计算机应用, 2022, 42(5): 1563-1569.
Ye ZHANG, Rong LIU, Ming LIU, Ming CHEN. Image super-resolution reconstruction network based on multi-channel attention mechanism[J]. Journal of Computer Applications, 2022, 42(5): 1563-1569.
方法 | 算法 | CUFED5 | Sun80 | Urban100 | Manga109 | ||||
---|---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | ||
SISR | SRCNN | 25.33 | 0.745 | 28.26 | 0.781 | 24.41 | 0.738 | 27.12 | 0.850 |
MDSR | 25.93 | 0.777 | 28.52 | 0.792 | 25.51 | 0.783 | 28.93 | 0.891 | |
RDN | 25.95 | 0.769 | 29.63 | 0.806 | 25.38 | 0.768 | 29.24 | 0.894 | |
RCAN | 26.06 | 0.769 | 29.86 | 0.810 | 25.42 | 0.768 | 29.38 | 0.895 | |
SRGAN | 24.40 | 0.702 | 26.76 | 0.725 | 24.07 | 0.729 | 25.12 | 0.802 | |
ENet | 24.24 | 0.695 | 26.24 | 0.702 | 23.63 | 0.711 | 25.25 | 0.802 | |
ESRGAN | 21.90 | 0.633 | 24.18 | 0.651 | 20.91 | 0.620 | 23.53 | 0.797 | |
RSRGAN | 22.31 | 0.635 | 25.60 | 0.667 | 21.47 | 0.624 | 25.04 | 0.803 | |
RefSR | CrossNet | 25.48 | 0.764 | 28.52 | 0.793 | 25.11 | 0.764 | 23.36 | 0.741 |
SRNTT_rec | 26.24 | 0.784 | 28.54 | 0.793 | 25.50 | 0.783 | 28.95 | 0.885 | |
SRNTT | 25.61 | 0.764 | 27.59 | 0.756 | 25.09 | 0.774 | 27.54 | 0.862 | |
TTSR_rec | 27.09** | 0.804** | 30.02* | 0.814* | 25.87** | 0.784** | 30.09** | 0.907** | |
TTSR | 25.53 | 0.765 | 28.59 | 0.774 | 24.62 | 0.747 | 28.70 | 0.886 | |
SRCA_rec | 27.09* | 0.807* | 29.93** | 0.813** | 25.93* | 0.786* | 30.25* | 0.909* | |
SRCA | 25.87 | 0.771 | 28.75 | 0.777 | 25.04 | 0.757 | 29.33 | 0.891 |
表1 在四个不同数据集上不同算法的PSNR/SSIM比较
Tab. 1 PSNR/SSIM comparison of different algorithms on four different datasets
方法 | 算法 | CUFED5 | Sun80 | Urban100 | Manga109 | ||||
---|---|---|---|---|---|---|---|---|---|
PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | PSNR/dB | SSIM | ||
SISR | SRCNN | 25.33 | 0.745 | 28.26 | 0.781 | 24.41 | 0.738 | 27.12 | 0.850 |
MDSR | 25.93 | 0.777 | 28.52 | 0.792 | 25.51 | 0.783 | 28.93 | 0.891 | |
RDN | 25.95 | 0.769 | 29.63 | 0.806 | 25.38 | 0.768 | 29.24 | 0.894 | |
RCAN | 26.06 | 0.769 | 29.86 | 0.810 | 25.42 | 0.768 | 29.38 | 0.895 | |
SRGAN | 24.40 | 0.702 | 26.76 | 0.725 | 24.07 | 0.729 | 25.12 | 0.802 | |
ENet | 24.24 | 0.695 | 26.24 | 0.702 | 23.63 | 0.711 | 25.25 | 0.802 | |
ESRGAN | 21.90 | 0.633 | 24.18 | 0.651 | 20.91 | 0.620 | 23.53 | 0.797 | |
RSRGAN | 22.31 | 0.635 | 25.60 | 0.667 | 21.47 | 0.624 | 25.04 | 0.803 | |
RefSR | CrossNet | 25.48 | 0.764 | 28.52 | 0.793 | 25.11 | 0.764 | 23.36 | 0.741 |
SRNTT_rec | 26.24 | 0.784 | 28.54 | 0.793 | 25.50 | 0.783 | 28.95 | 0.885 | |
SRNTT | 25.61 | 0.764 | 27.59 | 0.756 | 25.09 | 0.774 | 27.54 | 0.862 | |
TTSR_rec | 27.09** | 0.804** | 30.02* | 0.814* | 25.87** | 0.784** | 30.09** | 0.907** | |
TTSR | 25.53 | 0.765 | 28.59 | 0.774 | 24.62 | 0.747 | 28.70 | 0.886 | |
SRCA_rec | 27.09* | 0.807* | 29.93** | 0.813** | 25.93* | 0.786* | 30.25* | 0.909* | |
SRCA | 25.87 | 0.771 | 28.75 | 0.777 | 25.04 | 0.757 | 29.33 | 0.891 |
1 | FREEMAN W T, PASZTOR E C. Learning low-level vision [C]// Proceedings of the 1999 7th IEEE International Conference on Computer Vision. Piscataway: IEEE, 1999: 1182-1189. 10.1109/iccv.1999.790414 |
2 | 苏秉华,金伟其,牛丽红,等.超分辨率图像复原及其进展[J].光学技术,2001,27(1):6-9. 10.3321/j.issn:1002-1582.2001.01.018 |
SU B H, JIN W Q, NIU L H, et al. Super-resolution image restoration and progress [J]. Optical Technique, 2001, 27(1): 6-9. 10.3321/j.issn:1002-1582.2001.01.018 | |
3 | FREEMAN W T, JONES T R, PASZTOR E C. Example-based super-resolution [J]. IEEE Computer Graphics and Applications, 2002, 22(2): 56-65. 10.1109/38.988747 |
4 | DONG C, LOY C C, HE K M, et al. Learning a deep convolutional network for image super-resolution [C]// Proceedings of the 2014 European Conference on Computer Vision, LNCS 8692. Cham: Springer, 2014: 184-199. |
5 | KIM J, LEE J K, LEE K M. Deeply-recursive convolutional network for image super-resolution [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 1637-1645. 10.1109/cvpr.2016.181 |
6 | CAO C S, LIU X M, YANG Y, et al. Look and think twice: capturing top-down visual attention with feedback convolutional neural networks [C]// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 2956-2964. 10.1109/iccv.2015.338 |
7 | WANG F, JIANG M Q, QIAN C, et al. Residual attention network for image classification [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6450-6458. 10.1109/cvpr.2017.683 |
8 | HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141. 10.1109/cvpr.2018.00745 |
9 | LU Y, ZHOU Y, JIANG Z Q, et al. Channel attention and multi-level features fusion for single image super-resolution [C]// Proceedings of the 2018 IEEE International Conference on Visual Communications and Image Processing. Piscataway: IEEE, 2018: 1-4. 10.1109/vcip.2018.8698663 |
10 | ZHANG Z F, WANG Z W, LIN Z, et al. Image super-resolution by neural texture transfer [C]// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 7974-7983. 10.1109/cvpr.2019.00817 |
11 | YANG F Z, YANG H, FU J L, et al. Learning texture transformer network for image super-resolution [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5790-5799. 10.1109/cvpr42600.2020.00583 |
12 | WANG Q L, WU B G, ZHU P F, et al. ECA-Net: efficient channel attention for deep convolutional neural networks [C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11531-11539. 10.1109/cvpr42600.2020.01155 |
13 | 赵荣椿,赵忠明,赵歆波.数字图像处理与分析[M].北京:清华大学出版社,2013:36-40. |
ZHAO R C, ZHAO Z M, ZHAO X B. Digital Image Processing and Analysis [M]. Beijing: Tsinghua University Press, 2013: 36-40. | |
14 | SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition [EB/OL]. [2021-02-23].. 10.5244/c.28.6 |
15 | KINGMA D P, BA J L. Adam: a method for stochastic optimization [EB/OL]. [2021-02-23]. . |
16 | SUN L B, HAYS J. Super-resolution from internet-scale scene matching [C]// Proceedings of the 2012 IEEE International Conference on Computational Photography. Piscataway: IEEE, 2012: 1-12. 10.1109/iccphot.2012.6215221 |
17 | HUANG J B, SINGH A, AHUJA N. Single image super-resolution from transformed self-exemplars [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 5197-5206. 10.1109/cvpr.2015.7299156 |
18 | MATSUI Y, ITO K, ARAMAKI Y, et al. Sketch-based manga retrieval using Manga109 dataset [J]. Multimedia Tools and Applications, 2017, 76(20): 21811-21838. 10.1007/s11042-016-4020-z |
19 | LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway: IEEE, 2017: 1132-1140. 10.1109/cvprw.2017.151 |
20 | ZHANG Y L, TIAN Y P, KONG Y, et al. Residual dense network for image super-resolution [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2472-2481. 10.1109/cvpr.2018.00262 |
21 | ZHANG Y L, LI K P, LI K, et al. Image super-resolution using very deep residual channel attention networks [C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018: 294-310. 10.1007/978-3-030-01234-2_18 |
22 | LEDIG C, THEIS L, HUSZÁR F, et al. Photo-realistic single image super-resolution using a generative adversarial network [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Piscataway: IEEE, 2017: 105-114. 10.1109/cvpr.2017.19 |
23 | PASZKE A, CHAURASIA A, KIM S, et al. ENet: a deep neural network architecture for real-time semantic segmentation [EB/OL]. [2021-02-23]. . 10.1109/icsip49896.2020.9339426 |
24 | WANG X T, YU K, WU S X, et al. ESRGAN: enhanced super-resolution generative adversarial networks [C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11133. Cham: Springer, 2018: 63-79. |
25 | ZHANG W L, LIU Y H, DONG C, et al. RankSRGAN: generative adversarial networks with ranker for image super-resolution [C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 3096-3105. 10.1109/iccv.2019.00319 |
26 | ZHENG H T, JI M Q, WANG H Q, et al. CrossNet: an end-to-end reference-based super resolution network using cross-scale warping [C]// Proceedings of the2018 European Conference on Computer Vision, LNCS 11210. Cham: Springer, 2018: 87-104. |
[1] | 胡鹤轩, 隋华超, 胡强, 张晔, 胡震云, 马能武. 基于图注意力网络与双阶注意力机制的径流预报模型[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1607-1615. |
[2] | 陈代丽, 许国良. 基于注意力机制学习域内变化的跨域行人重识别方法[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1391-1397. |
[3] | 庄屹, 赵海涛. 面向三维点云单目标跟踪的提案聚合网络[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1407-1416. |
[4] | 屈震, 李堃婷, 冯志玺. 基于有效通道注意力的遥感图像场景分类[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1431-1439. |
[5] | 任炜, 白鹤翔. 基于全局与局部标签关系的多标签图像分类方法[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1383-1390. |
[6] | 杨先凤, 赵家和, 李自强. 融合字注释的文本分类模型[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1317-1323. |
[7] | 董永峰, 孙跃华, 高立超, 韩鹏, 季海鹏. 基于改进一维卷积和双向长短期记忆神经网络的故障诊断方法[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1207-1215. |
[8] | 胡新荣, 张君宇, 彭涛, 刘军平, 何儒汉, 何凯. 级联跨域特征融合的虚拟试衣[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1269-1274. |
[9] | 顾军华, 王锐, 李宁宁, 张素琪. 融合协同过滤信息的知识图注意力网络[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1087-1092. |
[10] | 刘志华, 陈文洁, 陈爱斌. 基于自注意力机制时频谱同源特征融合的鸟鸣声分类[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1260-1268. |
[11] | 蒋雯静, 熊熙, 李中志, 李斌勇. 基于无采样协作知识图网络的推荐系统[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1057-1064. |
[12] | 张锦, 屈佩琪, 孙程, 罗蒙. 基于改进YOLOv5的安全帽佩戴检测算法[J]. 《计算机应用》唯一官方网站, 2022, 42(4): 1292-1300. |
[13] | 罗圣钦, 陈金怡, 李洪均. 基于注意力机制的多尺度残差UNet实现乳腺癌灶分割[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 818-824. |
[14] | 朱文球, 邹广, 曾志高. 融合层次特征和混合注意力的目标跟踪算法[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 833-843. |
[15] | 余娜, 刘彦, 魏雄炬, 万源. 基于注意力机制和金字塔融合的RGB-D室内场景语义分割[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 844-853. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||