Image super-resolution reconstruction network based on multi-channel attention mechanism

doi:10.11772/j.issn.1001-9081.2021030498

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (5): 1563-1569.DOI: 10.11772/j.issn.1001-9081.2021030498

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles Next Articles

Image super-resolution reconstruction network based on multi-channel attention mechanism

Ye ZHANG¹, Rong LIU¹, Ming LIU²(), Ming CHEN¹

^1.College of Physical Science and Technology，Central China Normal University，Wuhan Hubei 430079，China
^2.School of Computer Science，Central China Normal University，Wuhan Hubei 430079，China

Received:2021-04-02 Revised:2021-06-28 Accepted:2021-07-01 Online:2022-06-11 Published:2022-05-10
Contact: Ming LIU
About author:ZHANG Ye， born in 1997，M. S. candidate. Her research interestsinclude pattern recognition，intelligent information processing.
LIU Rong， born in 1969，Ph. D.，associate professor. Her researchinterests include intelligent information processing，pattern recognition.
LIU Ming， born in 1967，Ph. D.，professor. His research interestsinclude internet of things， computer system structure， intelligent information processing and visualization.
CHEN Ming， born in 1995，M. S. candidate. His research interestsinclude pattern recognition，intelligent information processing.
Supported by:
National Social Science Fund of China(19BTQ005)

基于多通道注意力机制的图像超分辨率重建网络

张晔¹, 刘蓉¹, 刘明²(), 陈明¹

^1.华中师范大学物理科学与技术学院，武汉 430079
^2.华中师范大学计算机学院，武汉 430079

通讯作者: 刘明
作者简介:张晔（1997—），女，河北石家庄人，硕士研究生，主要研究方向：模式识别、智能信息处理
刘蓉（1969—），女，湖南安化人，副教授，博士，主要研究方向：智能信息处理、模式识别
刘明（1967—），男，湖北仙桃人，教授，博士，CCF会员，主要研究方向：物联网、计算机系统结构、智能信息处理及可视化 lium@mail.ccnu.edu.cn
陈明（1995—），男，湖北十堰人，硕士研究生，主要研究方向：模式识别、智能信息处理。
基金资助:
国家社会科学基金资助项目(19BTQ005)

Abstract

Abstract:

The existing image super-resolution reconstruction methods are affected by texture distortion and details blurring of generated images. To address these problems， a new image super-resolution reconstruction network based on multi-channel attention mechanism was proposed. Firstly， in the texture extraction module of the proposed network， a multi-channel attention mechanism was designed to realize the cross-channel information interaction by combining one-dimensional convolution， thereby achieving the purpose of paying attention to important feature information. Then， in the texture recovery module of the proposed network， the dense residual blocks were introduced to recover part of high-frequency texture details as many as possible to improve the performance of model and generate high-quality reconstructed images. The proposed network is able to improve visual effects of reconstructed images effectively. Besides， the results on benchmark dataset CUFED5 show that the proposed network has achieved the 1.76 dB and 0.062 higher in Peak Signal-to-Noise Ratio （PSNR） and Structural SIMilarity （SSIM） compared with the classic Super-Resolution using Convolutional Neural Network （SRCNN） method. Experimental results show that the proposed network can increase the accuracy of texture migration， and effectively improve the quality of generated images.

Key words: image super-resolution reconstruction, texture transfer, attention mechanism, one-dimensional convolution, dense residual block

摘要：

针对现有的图像超分辨率重建方法存在生成图像纹理扭曲、细节模糊等问题，提出了一种基于多通道注意力机制的图像超分辨率重建网络。首先，该网络中的纹理提取模块通过设计多通道注意力机制并结合一维卷积实现跨通道的信息交互，以关注重要特征信息；然后，该网络中的纹理恢复模块引入密集残差块来尽可能恢复部分高频纹理细节，从而提升模型性能并产生优质重建图像。所提网络不仅能够有效提升图像的视觉效果，而且在基准数据集CUFED5上的结果表明所提网络与经典的基于卷积神经网络的超分辨率重建（SRCNN）方法相比，峰值信噪比（PSNR）和结构相似度（SSIM）分别提升了1.76 dB和0.062。实验结果表明，所提网络可提高纹理迁移的准确性，并有效提升生成图像的质量。

关键词: 图像超分辨率重建, 纹理迁移, 注意力机制, 一维卷积, 密集残差块

CLC Number:

TP391.4

Ye ZHANG, Rong LIU, Ming LIU, Ming CHEN. Image super-resolution reconstruction network based on multi-channel attention mechanism[J]. Journal of Computer Applications, 2022, 42(5): 1563-1569.

张晔, 刘蓉, 刘明, 陈明. 基于多通道注意力机制的图像超分辨率重建网络[J]. 《计算机应用》唯一官方网站, 2022, 42(5): 1563-1569.

Figures/Tables 10

References 26

1	FREEMAN W T， PASZTOR E C. Learning low-level vision ［C］// Proceedings of the 1999 7th IEEE International Conference on Computer Vision. Piscataway： IEEE， 1999： 1182-1189. 10.1109/iccv.1999.790414
2	苏秉华，金伟其，牛丽红，等.超分辨率图像复原及其进展［J］.光学技术，2001，27（1）：6-9. 10.3321/j.issn:1002-1582.2001.01.018
	SU B H， JIN W Q， NIU L H， et al. Super-resolution image restoration and progress ［J］. Optical Technique， 2001， 27（1）： 6-9. 10.3321/j.issn:1002-1582.2001.01.018
3	FREEMAN W T， JONES T R， PASZTOR E C. Example-based super-resolution ［J］. IEEE Computer Graphics and Applications， 2002， 22（2）： 56-65. 10.1109/38.988747
4	DONG C， LOY C C， HE K M， et al. Learning a deep convolutional network for image super-resolution ［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS 8692. Cham： Springer， 2014： 184-199.
5	KIM J， LEE J K， LEE K M. Deeply-recursive convolutional network for image super-resolution ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1637-1645. 10.1109/cvpr.2016.181
6	CAO C S， LIU X M， YANG Y， et al. Look and think twice： capturing top-down visual attention with feedback convolutional neural networks ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 2956-2964. 10.1109/iccv.2015.338
7	WANG F， JIANG M Q， QIAN C， et al. Residual attention network for image classification ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6450-6458. 10.1109/cvpr.2017.683
8	HU J， SHEN L， SUN G. Squeeze-and-excitation networks ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
9	LU Y， ZHOU Y， JIANG Z Q， et al. Channel attention and multi-level features fusion for single image super-resolution ［C］// Proceedings of the 2018 IEEE International Conference on Visual Communications and Image Processing. Piscataway： IEEE， 2018： 1-4. 10.1109/vcip.2018.8698663
10	ZHANG Z F， WANG Z W， LIN Z， et al. Image super-resolution by neural texture transfer ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 7974-7983. 10.1109/cvpr.2019.00817
11	YANG F Z， YANG H， FU J L， et al. Learning texture transformer network for image super-resolution ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 5790-5799. 10.1109/cvpr42600.2020.00583
12	WANG Q L， WU B G， ZHU P F， et al. ECA-Net： efficient channel attention for deep convolutional neural networks ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 11531-11539. 10.1109/cvpr42600.2020.01155
13	赵荣椿，赵忠明，赵歆波.数字图像处理与分析［M］.北京：清华大学出版社，2013：36-40.
	ZHAO R C， ZHAO Z M， ZHAO X B. Digital Image Processing and Analysis ［M］. Beijing： Tsinghua University Press， 2013： 36-40.
14	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognition ［EB/OL］. ［2021-02-23］.. 10.5244/c.28.6
15	KINGMA D P， BA J L. Adam： a method for stochastic optimization ［EB/OL］. ［2021-02-23］. .
16	SUN L B， HAYS J. Super-resolution from internet-scale scene matching ［C］// Proceedings of the 2012 IEEE International Conference on Computational Photography. Piscataway： IEEE， 2012： 1-12. 10.1109/iccphot.2012.6215221
17	HUANG J B， SINGH A， AHUJA N. Single image super-resolution from transformed self-exemplars ［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 5197-5206. 10.1109/cvpr.2015.7299156
18	MATSUI Y， ITO K， ARAMAKI Y， et al. Sketch-based manga retrieval using Manga109 dataset ［J］. Multimedia Tools and Applications， 2017， 76（20）： 21811-21838. 10.1007/s11042-016-4020-z
19	LIM B， SON S， KIM H， et al. Enhanced deep residual networks for single image super-resolution ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1132-1140. 10.1109/cvprw.2017.151
20	ZHANG Y L， TIAN Y P， KONG Y， et al. Residual dense network for image super-resolution ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2472-2481. 10.1109/cvpr.2018.00262
21	ZHANG Y L， LI K P， LI K， et al. Image super-resolution using very deep residual channel attention networks ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer， 2018： 294-310. 10.1007/978-3-030-01234-2_18
22	LEDIG C， THEIS L， HUSZÁR F， et al. Photo-realistic single image super-resolution using a generative adversarial network ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition， Piscataway： IEEE， 2017： 105-114. 10.1109/cvpr.2017.19
23	PASZKE A， CHAURASIA A， KIM S， et al. ENet： a deep neural network architecture for real-time semantic segmentation ［EB/OL］. ［2021-02-23］. . 10.1109/icsip49896.2020.9339426
24	WANG X T， YU K， WU S X， et al. ESRGAN： enhanced super-resolution generative adversarial networks ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11133. Cham： Springer， 2018： 63-79.
25	ZHANG W L， LIU Y H， DONG C， et al. RankSRGAN： generative adversarial networks with ranker for image super-resolution ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 3096-3105. 10.1109/iccv.2019.00319
26	ZHENG H T， JI M Q， WANG H Q， et al. CrossNet： an end-to-end reference-based super resolution network using cross-scale warping ［C］// Proceedings of the2018 European Conference on Computer Vision， LNCS 11210. Cham： Springer， 2018： 87-104.

方法	算法	CUFED5		Sun80		Urban100		Manga109
方法	算法	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SISR	SRCNN	25.33	0.745	28.26	0.781	24.41	0.738	27.12	0.850
	MDSR	25.93	0.777	28.52	0.792	25.51	0.783	28.93	0.891
	RDN	25.95	0.769	29.63	0.806	25.38	0.768	29.24	0.894
	RCAN	26.06	0.769	29.86	0.810	25.42	0.768	29.38	0.895
	SRGAN	24.40	0.702	26.76	0.725	24.07	0.729	25.12	0.802
	ENet	24.24	0.695	26.24	0.702	23.63	0.711	25.25	0.802
	ESRGAN	21.90	0.633	24.18	0.651	20.91	0.620	23.53	0.797
	RSRGAN	22.31	0.635	25.60	0.667	21.47	0.624	25.04	0.803
RefSR	CrossNet	25.48	0.764	28.52	0.793	25.11	0.764	23.36	0.741
	SRNTT_rec	26.24	0.784	28.54	0.793	25.50	0.783	28.95	0.885
	SRNTT	25.61	0.764	27.59	0.756	25.09	0.774	27.54	0.862
	TTSR_rec	27.09**	0.804**	30.02*	0.814*	25.87**	0.784**	30.09**	0.907**
	TTSR	25.53	0.765	28.59	0.774	24.62	0.747	28.70	0.886
	SRCA_rec	27.09*	0.807*	29.93**	0.813**	25.93*	0.786*	30.25*	0.909*
	SRCA	25.87	0.771	28.75	0.777	25.04	0.757	29.33	0.891

方法	算法	CUFED5		Sun80		Urban100		Manga109
方法	算法	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SISR	SRCNN	25.33	0.745	28.26	0.781	24.41	0.738	27.12	0.850
	MDSR	25.93	0.777	28.52	0.792	25.51	0.783	28.93	0.891
	RDN	25.95	0.769	29.63	0.806	25.38	0.768	29.24	0.894
	RCAN	26.06	0.769	29.86	0.810	25.42	0.768	29.38	0.895
	SRGAN	24.40	0.702	26.76	0.725	24.07	0.729	25.12	0.802
	ENet	24.24	0.695	26.24	0.702	23.63	0.711	25.25	0.802
	ESRGAN	21.90	0.633	24.18	0.651	20.91	0.620	23.53	0.797
	RSRGAN	22.31	0.635	25.60	0.667	21.47	0.624	25.04	0.803
RefSR	CrossNet	25.48	0.764	28.52	0.793	25.11	0.764	23.36	0.741
	SRNTT_rec	26.24	0.784	28.54	0.793	25.50	0.783	28.95	0.885
	SRNTT	25.61	0.764	27.59	0.756	25.09	0.774	27.54	0.862
	TTSR_rec	27.09**	0.804**	30.02*	0.814*	25.87**	0.784**	30.09**	0.907**
	TTSR	25.53	0.765	28.59	0.774	24.62	0.747	28.70	0.886
	SRCA_rec	27.09*	0.807*	29.93**	0.813**	25.93*	0.786*	30.25*	0.909*
	SRCA	25.87	0.771	28.75	0.777	25.04	0.757	29.33	0.891

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[3]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[4]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[5]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[6]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[7]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[8]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[9]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[10]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[11]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[12]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[13]	Dahai LI, Zhonghua WANG, Zhendong WANG. Dual-branch low-light image enhancement network combining spatial and frequency domain information [J]. Journal of Computer Applications, 2024, 44(7): 2175-2182.
[14]	Wenliang WEI, Yangping WANG, Biao YUE, Anzheng WANG, Zhe ZHANG. Deep learning model for infrared and visible image fusion based on illumination weight allocation and attention [J]. Journal of Computer Applications, 2024, 44(7): 2183-2191.
[15]	Mei WANG, Xuesong SU, Jia LIU, Ruonan YIN, Shan HUANG. Time series classification method based on multi-scale cross-attention fusion in time-frequency domain [J]. Journal of Computer Applications, 2024, 44(6): 1842-1847.

Image super-resolution reconstruction network based on multi-channel attention mechanism

基于多通道注意力机制的图像超分辨率重建网络

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 26

Related Articles 15

Recommended Articles

Metrics