Mixed-order channel attention network for single image super-resolution reconstruction

doi:10.11772/j.issn.1001-9081.2020020281

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (10): 3048-3053.DOI: 10.11772/j.issn.1001-9081.2020020281

• Virtual reality and multimedia computing • Previous Articles Next Articles

Mixed-order channel attention network for single image super-resolution reconstruction

YAO Lu¹, SONG Huihui², ZHANG Kaihua¹

1. Jiangsu Key Laboratory of Big Data Analysis Technology(Nanjing University of Information Science and Technology), Nanjing Jiangsu 210044 China;
2. Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing Jiangsu 210044, China

Received:2020-03-14 Revised:2020-06-15 Online:2020-10-10 Published:2020-06-24
Supported by:
This work is partially supported by the National Major Project for New Generation of AI (2018AAA0100400), the National Natural Science Foundation of China (6187289, 61876088), the Natural Science Foundation of Jiangsu Province (BK20191397, BK20170040).

混合阶通道注意力网络的单图像超分辨率重建

姚鲁¹, 宋慧慧², 张开华¹

1. 江苏省大数据分析技术重点实验室(南京信息工程大学), 南京 210044;
2. 南京信息工程大学大气环境与装备技术协同创新中心, 南京 210044

通讯作者: 宋慧慧
作者简介:姚鲁(1993-),男,江苏连云港人,硕士研究生,主要研究方向:图像超分辨率重建、深度学习;宋慧慧(1986-),女,山东聊城人,教授,博士,CCF会员,主要研究方向:机器学习、遥感图像处理;张开华(1983-),男,山东日照人,教授,博士,CCF会员,主要研究方向:视频目标跟踪、显著性检测、视频目标分割。
基金资助:
国家新一代人工智能重大项目（2018AAA0100400）；国家自然科学基金资助项目（61872189，61876088）；江苏省自然科学基金资助项目（BK20191397，BK20170040）。

Abstract

Abstract: For the current channel attention mechanism used for super-resolution reconstruction, there are problems that the attention prediction destroys the direct corresponding relationship between each channel and its weight and the mechanism only considers the first-order or second-order channel attention without comprehensive consideration of the advantage complementation. Therefore, a mixed-order channel attention network for image super-resolution reconstruction was proposed. First of all, by using the local cross-channel interaction strategy, increase and reduction in channel dimension used by the first-order and second-order channel attention models were changed into a fast one-dimensional convolution with kernel k, which not only makes the channel attention prediction more direct and accurate but makes the resulting model simpler than before. Besides, the improved first and second-order channel attention models above were adopted to comprehensively take the advantages of channel attentions of different orders, thus improving network discrimination. Experimental results on the benchmark datasets show that compared with the existing super-resolution algorithms, the proposed method has the best recovered texture details and high frequency information of the reconstructed images and the Perceptual Indictor (PI) on Set5 and BSD100 datasets are increased by 0.3 and 0.1 on average respectively. It shows that this network is more accurate in predicting channel attention and comprehensively uses channel attentions of different orders, so as to improve the performance.

Key words: channel attention mechanism, Generative Adversarial Network (GAN), image super-resolution, Convolutional Neural Network (CNN), deep learning

摘要： 目前用于图像超分辨率重建的通道注意力机制存在注意力预测破坏每个通道和其权重的直接对应关系以及仅仅只考虑一阶或二阶通道注意力而没有综合考虑优势互补的问题，因此提出一种混合阶通道注意力网络的单图像超分辨率重建算法。首先，该网络框架利用局部跨通道相互作用策略将之前一、二阶通道注意力模型采用的升降维改为核为k的一维卷积。这样不仅使得通道注意力预测更直接准确，而且得到的模型相比之前的通道注意力模型更简单；同时，采用改进一、二阶通道注意力模型以综合利用不同阶通道注意力的优势，提高网络判别能力。在基准数据集上的实验结果表明，和现有的超分辨率算法相比，所提算法重建图像的纹理细节和高频信息能得到更好的恢复，且在Set5和BSD100数据集上感知指数（PI）分别平均提高0.3和0.1。这表明此网络能更准确地预测通道注意力并综合利用了不同阶通道注意力，一定程度上提升了性能。

关键词: 通道注意力机制, 生成对抗网络, 图像超分辨率重建, 卷积神经网络, 深度学习

CLC Number:

TP391.41

YAO Lu, SONG Huihui, ZHANG Kaihua. Mixed-order channel attention network for single image super-resolution reconstruction[J]. Journal of Computer Applications, 2020, 40(10): 3048-3053.

姚鲁, 宋慧慧, 张开华. 混合阶通道注意力网络的单图像超分辨率重建[J]. 计算机应用, 2020, 40(10): 3048-3053.

References

[1] 欧阳宁, 梁婷, 林乐平. 基于自注意力网络的图像超分辨率重建[J]. 计算机应用,2019,39(8):2391-2395.(OUYANG N, LIANG T,LIN L P. Self-attention network based image superresolution[J]. Journal of Computer Applications,2019,39(8):2391-2395.)
[2] DONG C,LOY C C,HE K,at al. Image super-resolution using deep convolutional network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2016,38(2):295-307.
[3] HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.
[4] KIM J,LEE J K,LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1646-1654.
[5] ZHANG Y,LI K,LI K,et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11211. Cham:Springer,2018:294-310.
[6] ITTI L,KOCH C,NIEBUR E. A model of saliency-based visual attention for rapid scene analysis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,1998,20(11):1254-1259.
[7] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[EB/OL].[2017-06-05]. https://arxiv.org/pdf/1709.01507.pdf.
[8] DAI T,CAI J,ZHANG Y,et al. Second-order attention network for single image super-resolution[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2019:11057-11066.
[9] SHI W,CABALLERO J,HUSZÁR F,et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1874-1883.
[10] WANG Q,WU B,ZHU P,et al. ECA-Net:efficient channel attention for deep convolutional neural networks[EB/OL].[2019-10-08]. https://arxiv.org/pdf/1910.03151.pdf.
[11] VU T,LUU T M,YOO C D. Perception-enhanced image superresolution via relativistic generative adversarial networks[C]//Proceedings of the 2018 European Conference on Computer Vision,LNCS 11133. Cham:Springer,2018:98-113.
[12] AGUSTSSON E,TIMOFTE R. NTIRE 2017 challenge on single image super-resolution:dataset and study[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1122-1131.
[13] BLAU Y,MICHAELI T. The perception-distortion tradeoff[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:6228-6237.
[14] MA C,YANG C Y,YANG X,et al. Learning a no-reference quality metric for single-image super-resolution[J]. Computer Vision and Image Understanding,2017,158:1-16.
[15] MITTAL A,SOUNDARARAJAN R,BOVIK A C. Making a "completely blind" image quality analyzer[J]. IEEE Signal Processing Letters,2013,20(3):209-212.
[16] KEYS R. Cubic convolution interpolation for digital image processing[J]. IEEE Transactions on Acoustics,Speech,and Signal Processing,1981,29(6):1153-1160.
[17] LIM B,SON S,KIM H,et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE,2017:1132-1440.
[18] LEDIG C,THEIS L,HUSZAR F,et al. Photo-realistic single image super-resolution using a generate adversarial network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway,NJ:IEEE,2017:105-114.
[19] SAJJADI M,SCHÖLKOPF B,HIRSCH M. EnhanceNet:single image super-resolution through automated texture synthesis[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:4501-4510.

Mixed-order channel attention network for single image super-resolution reconstruction

混合阶通道注意力网络的单图像超分辨率重建

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	CHEN Chengrui, SUN Ning, HE Shibiao, LIAO Yong. Deep learning-based joint channel estimation and equalization algorithm for C-V2X communications [J]. Journal of Computer Applications, 2021, 41(9): 2687-2693.
[2]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[3]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[4]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[5]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[6]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[7]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[8]	ZHENG Zhiqiang, HU Xin, WENG Zhi, WANG Yuhe, CHENG Xi. Cattle eye image feature extraction method based on improved DenseNet [J]. Journal of Computer Applications, 2021, 41(9): 2780-2784.
[9]	XIE Defeng, JI Jianmin. Syntax-enhanced semantic parsing with syntax-aware representation [J]. Journal of Computer Applications, 2021, 41(9): 2489-2495.
[10]	DAI Yurou, YANG Qing, ZHANG Fengli, ZHOU Fan. Trajectory prediction model of social network users based on self-supervised learning [J]. Journal of Computer Applications, 2021, 41(9): 2545-2551.
[11]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[12]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[13]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[14]	GUAN Qijie, ZHANG Ting, LI Deya, ZHOU Shaojing, DU Yi. Indefinite reconstruction method of spatial data based on multi-resolution generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2306-2311.
[15]	SUN Xiao, XU Jindong. Remote sensing image dehazing method based on cascaded generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2440-2444.