Image super-resolution restoration algorithm based on information distillation network with dual attention mechanism

doi:10.11772/j.issn.1001-9081.2021010134

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 239-244.DOI: 10.11772/j.issn.1001-9081.2021010134

• Multimedia computing and computer simulation • Previous Articles Next Articles

Image super-resolution restoration algorithm based on information distillation network with dual attention mechanism

Suyu WANG¹^,², Jing YANG¹^,²(), Yue LI¹^,²

^1.Beijing Engineering Research Center for IoT Software and Systems （Beijing University of Technology），Beijing 100124，China
^2.Faculty of Information Technology，Beijing University of Technology，Beijing 100124，China

Received:2021-01-26 Revised:2021-04-28 Accepted:2021-04-29 Online:2022-01-11 Published:2022-01-10
Contact: Jing YANG
About author:WANG Suyu， born in 1976， Ph. D.， associate professor. Her research interests include image and video signal processing， computer vision.
YANG Jing， born in 1994， M. S. candidate. Her research interests include image processing.
LI Yue， born in 1995， M. S. candidate. His research interests include image processing.

基于双注意力机制信息蒸馏网络的图像超分辨率复原算法

王素玉¹^,², 杨静¹^,²(), 李越¹^,²

^1.北京市物联网软件与系统工程技术研究中心（北京工业大学），北京 100124
^2.北京工业大学信息学部，北京 100124

通讯作者: 杨静
作者简介:王素玉（1976—），女，河北唐山人，副教授，博士，主要研究方向：图像与视频信号处理、计算机视觉
杨静（1994—），女，河北石家庄人，硕士研究生，主要研究方向：图像处理
李越（1995—），男，山西阳泉人，硕士研究生，主要研究方向：图像处理。

Abstract

Abstract:

Aiming at the problems of network training difficulty and low utilization rate of feature information caused by increasing network layers in super-resolution restoration technology， an image super-resolution restoration algorithm based on dual attention Information Distillation Network （IDN） was designed and implemented. Firstly， by taking the advantage of the low computational complexity of IDN and the advantage of the information distillation module by which more features were extracted， the weights of the features were readjust adaptively by introducing the Residual Attention Module （RAM） and considering the interdependence of image channels， so as to further improve the reconstruction ability of high-resolution details of images. Then， a new mixed loss function sensitive to edge information was designed to refine the image and accelerate the convergence of the network. Test results on Set5， Set14， BSD100 and Urban100 public datasets show that the visual effect and Peak Signal-to-Noise Ratio （PSNR） of the proposed method are superior to those of the current mainstream algorithms.

Key words: Information Distillation Network (IDN), image super-resolution restoration, Spatial Attention (SA), Channel Attention (CA), mixed loss function

摘要：

针对超分辨率复原技术中网络层数不断加深导致的网络训练困难、特征信息利用率低等问题，设计并实现了一种基于双注意力的信息蒸馏网络（IDN）的图像超分辨率复原算法。首先，利用IDN较低的计算复杂度及信息蒸馏模块提取更多特征的优势，通过引入残差注意力模块（RAM）并考虑图像通道之间的相互依赖性来自适应地重新调整特征权重，从而进一步提升图像高分辨率细节的重建能力；然后，设计了对于边缘信息敏感的新型混合损失函数对图像进行细化处理，以加速网络收敛。在Set5、Set14、BSD100和Urban100公共数据集上的测试结果表明，该方法的视觉效果和峰值信噪比（PSNR）均优于当前主流算法。

关键词: 信息蒸馏网络, 图像超分辨率复原, 空间注意力, 通道注意力, 混合损失函数

CLC Number:

TP391.4

Suyu WANG, Jing YANG, Yue LI. Image super-resolution restoration algorithm based on information distillation network with dual attention mechanism[J]. Journal of Computer Applications, 2022, 42(1): 239-244.

王素玉, 杨静, 李越. 基于双注意力机制信息蒸馏网络的图像超分辨率复原算法[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 239-244.

Figures/Tables 8

References 16

1	DONG C， LOY C C， HE K M， et al. Image super-resolution using deep convolutional networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（2）：295-307. 10.1109/tpami.2015.2439281
2	HUI Z， WANG X M， GAO X B. Fast and accurate single image super-resolution via information distillation network［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018：723-731. 10.1109/cvpr.2018.00082
3	ZHAO H， GALLO O， FROSIO I， et al. Loss functions for image restoration with neural networks［J］. IEEE Transactions on Computational Imaging， 2017， 3（1）：47-57. 10.1109/tci.2016.2644865
4	ZHAO H， GALLO O， FROSIO I， et al. Loss functions for neural networks for image processing［EB/OL］. （2018-04-20）［2020-10-15］.. 10.1109/tci.2016.2644865
5	YANG J C， WRIGHT J， HUANG T S， et al. Image super-resolution via sparse representation［J］. IEEE Transactions on Image Processing， 2010， 19（11）：2861-2873. 10.1109/tip.2010.2050625
6	ZEYDE R， ELAD M， PROTTER M. On single image scale-up using sparse-representations［C］// Proceedings of the 2010 International Conference Curves and Surfaces， LNCS6920. Berlin： Springer， 2012： 711-730.
7	KIM J， LEE J K， LEE K M. Accurate image super-resolution using very deep convolutional networks［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1646-1654. 10.1109/cvpr.2016.182
8	NOH H， HONG S， HAN B. Learning deconvolution network for semantic segmentation［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1520-1528. 10.1109/iccv.2015.178
9	LIM B， SON S， KIM H， et al. Enhanced deep residual networks for single image super-resolution［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1132-1140. 10.1109/cvprw.2017.151
10	KIM J H， CHOI J H， CHEON M， et al. RAM： residual attention module for single image super-resolution［EB/OL］. （2018-11-29）［2020-10-15］..
11	MARTIN D， FOWLKES C， TAL D， et al. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics［C］// Proceedings of the 8th IEEE International Conference on Computer Vision. Piscataway： IEEE， 2001：416-423. 10.1109/iccv.2001.937655
12	HU J， SHEN L， ALBANIE S， et al. Squeeze-and-excitation networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（8）：2011-2023. 10.1109/tpami.2019.2913372
13	WOO S， PARK J， LEE J Y. CBAM： convolutional block attention module［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11211. Cham： Springer， 2018：3-19.
14	DONG C， LOY C C， HE K M， et al. Learning a deep convolutional network for image super-resolution［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS8692. Cham： Springer， 2014：184-199.
15	KIM J， LEE J K， LEE K M. Deeply-recursive convolutional network for image super-resolution［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016：1637-1645. 10.1109/cvpr.2016.181
16	LAI W S， HUANG J B， AHUJA N， et al. Deep Laplacian pyramid networks for fast and accurate super-resolution［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017：5835-5843. 10.1109/cvpr.2017.618

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
/	×4	31.57	0.886 5	28.18	0.772 3	27.20	0.728 0	25.33	0.762 5
SE	×4	31.77	0.889 3	28.23	0.772 8	27.40	0.728 9	25.38	0.763 0
CBAM	×4	31.79	0.891 0	28.24	0.772 9	27.42	0.730 0	25.42	0.763 3
RAM	×4	31.80	0.890 2	28.26	0.772 9	27.43	0.730 2	25.41	0.770 0

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
/	×4	31.57	0.886 5	28.18	0.772 3	27.20	0.728 0	25.33	0.762 5
SE	×4	31.77	0.889 3	28.23	0.772 8	27.40	0.728 9	25.38	0.763 0
CBAM	×4	31.79	0.891 0	28.24	0.772 9	27.42	0.730 0	25.42	0.763 3
RAM	×4	31.80	0.890 2	28.26	0.772 9	27.43	0.730 2	25.41	0.770 0

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
RAM+L1	×4	31.80	0.890 2	28.26	0.772 9	27.43	0.730 2	25.41	0.770 0
RAM+（MS-SSIM）	×4	31.77	0.890 3	28.26	0.773 3	27.40	0.730 1	25.40	0.770 3
RAM+（（MS-SSIM）+L1）	×4	31.79	0.890 3	28.28	0.773 2	27.42	0.730 2	25.41	0.768 7

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
RAM+L1	×4	31.80	0.890 2	28.26	0.772 9	27.43	0.730 2	25.41	0.770 0
RAM+（MS-SSIM）	×4	31.77	0.890 3	28.26	0.773 3	27.40	0.730 1	25.40	0.770 3
RAM+（（MS-SSIM）+L1）	×4	31.79	0.890 3	28.28	0.773 2	27.42	0.730 2	25.41	0.768 7

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
RAM	×4	31.80	0.890 2	28.26	0.772 9	27.43	0.730 2	25.41	0.770 0
（MS-SSIM）+L1	×4	31.59	0.891 0	28.25	0.772 6	27.26	0.730 2	25.35	0.762 9
RAM+（（MS-SSIM）+L1）	×4	31.79	0.890 3	28.28	0.773 2	27.42	0.730 2	25.41	0.768 7

Image super-resolution restoration algorithm based on information distillation network with dual attention mechanism

基于双注意力机制信息蒸馏网络的图像超分辨率复原算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 16

Related Articles 2

Recommended Articles

Metrics

[1]	Juan WANG, Xuliang YUAN, Minghu WU, Liquan GUO, Zishan LIU. Real-time semantic segmentation method based on squeezing and refining network [J]. Journal of Computer Applications, 2022, 42(7): 1993-2000.
[2]	Guangkai LIAO, Zheng ZHANG, Zhiguo SONG. Convolutional network-based vehicle re-identification combining wavelet features and attention mechanism [J]. Journal of Computer Applications, 2022, 42(6): 1876-1883.

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
Bicubic	×2	33.66	0.929 9	30.24	0.868 8	29.56	0.843 1	26.88	0.840 3
	×3	30.39	0.868 2	27.55	0.774 2	27.21	0.738 5	24.46	0.734 9
	×4	28.42	0.810 4	26.00	0.702 7	25.96	0.667 5	23.14	0.657 7
SRCNN	×2	36.66	0.954 2	32.42	0.906 3	31.34	0.887 9	29.50	0.894 6
	×3	32.75	0.909 0	29.28	0.820 9	28.40	0.786 3	26.24	0.798 9
	×4	30.48	0.862 8	27.49	0.750 3	26.90	0.710 1	24.52	0.722 1
VDSR	×2	37.53	0.958 7	33.03	0.912 4	31.90	0.896 0	30.76	0.914 0
	×3	33.66	0.921 3	29.77	0.831 4	28.82	0.797 6	27.14	0.827 9
	×4	31.35	0.883 8	28.01	0.767 4	27.29	0.725 1	25.18	0.752 4
DRCN	×2	37.63	0.958 4	33.06	0.910 8	31.85	0.894 7	30.76	0.914 7
	×3	33.85	0.921 5	29.89	0.831 7	28.81	0.795 4	27.16	0.831 1
	×4	31.56	0.881 0	28.15	0.762 7	27.24	0.715 0	25.15	0.753 0
LapSRN	×2	37.52	0.958 1	33.08	0.910 9	31.80	0.894 9	30.41	0.911 2
	×3	33.82	0.920 7	29.89	0.830 4	28.82	0.795 0	27.07	0.829 8
	×4	31.54	0.881 1	28.19	0.763 5	27.32	0.716 2	25.21	0.756 4
IDN	×2	37.83	0.960 0	33.30	0.914 8	32.08	0.898 5	31.27	0.919 6
	×3	34.11	0.925 3	29.99	0.835 4	28.95	0.801 3	27.42	0.835 9
	×4	31.82	0.890 3	28.25	0.773 0	27.41	0.729 7	25.41	0.763 2
本文算法	×2	37.85	0.958 2	33.35	0.915 4	32.07	0.899 3	31.29	0.919 4
	×3	34.13	0.928 9	29.97	0.835 1	28.89	0.821 1	27.41	0.841 7
	×4	31.79	0.890 3	28.28	0.773 2	27.42	0.730 2	25.41	0.768 7

算法	Scale	Set5		Set14		BSD100		Urban100
算法	Scale	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
Bicubic	×2	33.66	0.929 9	30.24	0.868 8	29.56	0.843 1	26.88	0.840 3
	×3	30.39	0.868 2	27.55	0.774 2	27.21	0.738 5	24.46	0.734 9
	×4	28.42	0.810 4	26.00	0.702 7	25.96	0.667 5	23.14	0.657 7
SRCNN	×2	36.66	0.954 2	32.42	0.906 3	31.34	0.887 9	29.50	0.894 6
	×3	32.75	0.909 0	29.28	0.820 9	28.40	0.786 3	26.24	0.798 9
	×4	30.48	0.862 8	27.49	0.750 3	26.90	0.710 1	24.52	0.722 1
VDSR	×2	37.53	0.958 7	33.03	0.912 4	31.90	0.896 0	30.76	0.914 0
	×3	33.66	0.921 3	29.77	0.831 4	28.82	0.797 6	27.14	0.827 9
	×4	31.35	0.883 8	28.01	0.767 4	27.29	0.725 1	25.18	0.752 4
DRCN	×2	37.63	0.958 4	33.06	0.910 8	31.85	0.894 7	30.76	0.914 7
	×3	33.85	0.921 5	29.89	0.831 7	28.81	0.795 4	27.16	0.831 1
	×4	31.56	0.881 0	28.15	0.762 7	27.24	0.715 0	25.15	0.753 0
LapSRN	×2	37.52	0.958 1	33.08	0.910 9	31.80	0.894 9	30.41	0.911 2
	×3	33.82	0.920 7	29.89	0.830 4	28.82	0.795 0	27.07	0.829 8
	×4	31.54	0.881 1	28.19	0.763 5	27.32	0.716 2	25.21	0.756 4
IDN	×2	37.83	0.960 0	33.30	0.914 8	32.08	0.898 5	31.27	0.919 6
	×3	34.11	0.925 3	29.99	0.835 4	28.95	0.801 3	27.42	0.835 9
	×4	31.82	0.890 3	28.25	0.773 0	27.41	0.729 7	25.41	0.763 2
本文算法	×2	37.85	0.958 2	33.35	0.915 4	32.07	0.899 3	31.29	0.919 4
	×3	34.13	0.928 9	29.97	0.835 1	28.89	0.821 1	27.41	0.841 7
	×4	31.79	0.890 3	28.28	0.773 2	27.42	0.730 2	25.41	0.768 7