Single image super-resolution reconstruction method based on dense Inception

doi:10.11772/j.issn.1001-9081.2021010070

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (12): 3666-3671.DOI: 10.11772/j.issn.1001-9081.2021010070

• Multimedia computing and computer simulation • Previous Articles Next Articles

Single image super-resolution reconstruction method based on dense Inception

Haiyong WANG¹, Kaixin ZHANG²(), Weizheng GUAN²

^1.College of Computer Science，Nanjing University of Posts and Telecommunications，Nanjing Jiangsu 210023，China
^2.College of Internet of Things，Nanjing University of Posts and Telecommunications，Nanjing Jiangsu 210003，China

Received:2021-01-15 Revised:2021-03-29 Accepted:2021-04-06 Online:2021-04-15 Published:2021-12-10
Contact: Kaixin ZHANG
About author:WANG Haiyong， born in 1979， Ph. D.， associate research fellow. His research interests include computer network and security， information network.
GUAN Weizheng， born in 1992， M. S. candidate. His research interests include machine learning.
Supported by:
the CERNET Innovation Project(NGII20190612);the Postdoctoral Research Funding Program of Jiangsu Province(2020Z058)

基于密集Inception的单图像超分辨率重建方法

王海勇¹, 张开心²(), 管维正²

^1.南京邮电大学计算机学院，南京 210023
^2.南京邮电大学物联网学院，南京 210003

通讯作者: 张开心
作者简介:王海勇（1979—），男，江苏连云港人，副研究员，博士，CCF会员，主要研究方向：计算机网络与安全、信息网络
管维正（1992—），男，湖北孝感人，硕士研究生，主要研究方向：机器学习。
基金资助:
赛尔网络下一代互联网技术创新项目(NGII20190612);江苏省博士后科研资助计划项目(2020Z058)

Abstract

Abstract:

In recent years， the single image Super-Resolution （SR） reconstruction methods based on Convolutional Neural Network （CNN） have become mainstream. Under normal circumstances， the deeper network layers of the reconstruction model have， the more features are extracted， and the better reconstruction effect is. However， as the number of network layers increases， the reconstruction model will not only have the vanishing gradient problem， but also significantly increase the number of parameters and increase the difficulty of training. To solve the above problems， a single image SR reconstruction method based on dense Inception was proposed. In the proposed method， the image features were extracted by introducing the Inception-Residual Network （Inception-ResNet） structure， and the simplified dense network was adopted globally. And only the path that each module outputs to the reconstruction layer was constructed， avoiding the increase of computation amount caused by the generation of redundant data. When the magnification was 4， the dataset Set5 was used to test the model performance. The results show that， the Structural SIMilarity （SSIM） of the proposed model is 0.013 6 higher than that of accurate image Super-Resolution using Very Deep convolutional network （VDSR）， and the proposed method has the SSIM 0.002 9 higher and the model parameters 78% smaller than Multi-scale residual Network for Image Super-Resolution （MSRN）. The experimental results show that， under the premise of ensuring the depth and width of the model， the proposed method significantly reduces the number of parameters and the difficulty of training. In the meantime， the proposed method can achieve better Peak Signal-to-Noise Ratio （PSNR） and SSIM than the comparison methods.

Key words: image super-resolution reconstruction, Convolutional Neural Network (CNN), dense network, Inception-Residual Network (Inception-ResNet), residual module

摘要：

近几年，基于卷积神经网络（CNN）的单图像超分辨率（SR）重建方法成为了主流。通常情况下，重建模型的网络层数越深，提取的特征越多，重建效果越好；然而随着网络层数的加深，不仅会出现梯度消失的问题，还会显著增加参数量，增加训练的难度。针对以上问题，提出了一种基于密集Inception的单图像SR重建方法。该方法引入Inception-残差网络（Inception-ResNet）结构提取图像特征，全局采用简化后的密集网络，且仅构建每一个模块输出到重建层的路径，从而避免产生冗余数据来增加计算量。在放大倍数为4时，采用数据集Set5测试模型性能，结果显示与超深卷积神经网络的图像超分辨率（VDSR）相比，所提方法的结构相似性（SSIM）高了0.013 6；与基于多尺度残差网络的图像SR（MSRN）相比，SSIM高了0.002 9，模型参数量少了78%。实验结果表明，所提方法在保证模型的深度和宽度的情况下，显著减少了参数量，从而降低了训练的难度，而且取得了比对比方法更好的峰值信噪比（PSNR）和SSIM。

关键词: 图像超分辨率重建, 卷积神经网络, 密集网络, Inception-残差网络, 残差模块

CLC Number:

TP181

Haiyong WANG, Kaixin ZHANG, Weizheng GUAN. Single image super-resolution reconstruction method based on dense Inception[J]. Journal of Computer Applications, 2021, 41(12): 3666-3671.

王海勇, 张开心, 管维正. 基于密集Inception的单图像超分辨率重建方法[J]. 《计算机应用》唯一官方网站, 2021, 41(12): 3666-3671.

Figures/Tables 8

References 25

1	曾接贤，倪申龙.改进的卷积神经网络单幅图像超分辨率重建［J］.计算机工程与应用，2019，55（13）：1-7.
	ZENG J X， NI S L. Improved super-resolution reconstruction of single image based on convolution neural network ［J］. Computer Engineering and Applications， 2019， 55（13）： 1-7.
2	CHEN C， QI F. Single image super-resolution using deep CNN with dense skip connections and inception-ResNet ［C］// Proceedings of the 2018 9th International Conference on Information Technology in Medicine and Education. Piscataway： IEEE， 2018： 999-1003. 10.1109/itme.2018.00222
3	HUANG G， LIU Z， MAATEN L VAN DER， et al. Densely connected convolutional networks ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269. 10.1109/cvpr.2017.243
4	SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions ［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 1-9. 10.1109/cvpr.2015.7298594
5	SZEGEDY C， IOFFE S， VANHOUCKE V， et al. Inception-v4， Inception-ResNet and the impact of residual connections on learning ［C］// Proceedings of the 2017 31st AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2017： 4278-4284.
6	DONG C， LOY C C， HE K M， et al. Learning a deep convolutional network for image super-resolution ［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS8692. Cham： Springer， 2014： 184-199.
7	KIM J， LEE J K， LEE K M. Accurate image super-resolution using very deep convolutional networks ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1646-1654. 10.1109/cvpr.2016.182
8	DONG C， LOY C C， TANG X O. Accelerating the super-resolution convolutional neural network ［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS9906. Cham： Springer， 2016： 391-407.
9	KIM J， LEE J K， LEE K M. Deeply-recursive convolutional network for image super-resolution ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1637-1645. 10.1109/cvpr.2016.181
10	TONG T， LI G， LIU X J， et al. Image super-resolution using dense skip connections ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 4809-4817. 10.1109/iccv.2017.514
11	ZHANG Y L， TIAN Y P， KONG Y， et al. Residual dense network for image super-resolution ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2472-2481. 10.1109/cvpr.2018.00262
12	TANG Z M， LI S H， LUO L K， et al. Image super-resolution via simplified dense network with non-degenerate layers ［J］. IEEE Access， 2019， 7： 24775-24787. 10.1109/access.2019.2898846
13	IOFFE S， SZEGEDY C. Batch normalization： accelerating deep network training by reducing internal covariate shift ［C］// Proceedings of the 2015 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 448-456.
14	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826. 10.1109/cvpr.2016.308
15	LI J C， FANG F M， MEI K F， et al. Multi-scale residual network for image super-resolution ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11212. Cham： Springer， 2018： 527-542.
16	QIN J Y， SUN X F， YAN Y T， et al. Multi-resolution space-attended residual dense network for single image super-resolution ［J］. IEEE Access， 2020， 8： 40499-40511. 10.1109/access.2020.2976478
17	TIMOFTE R， AGUSTSSON E， GOOL L VAN， et al. NTIRE 2017 challenge on single image super-resolution： methods and results ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1110-1121.
18	BEVILACQUA M， ROUMY A， GUILLEMOT C， et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding ［C］// Proceedings of the 2012 British Machine Vision Conference. Durham： BMVA Press， 2012： Article No.135. 10.5244/c.26.135
19	ZEYDE R， ELAD M， PROTTER M. On single image scale-up using sparse-representations ［C］// Proceedings of the 2010 7th International Conference on Curves and Surfaces， LNCS6920. Berlin： Springer， 2010： 711-730.
20	HUANG J B， SINGH A， AHUJA N. Single image super-resolution from transformed self-exemplars ［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 5197-5206. 10.1109/cvpr.2015.7299156
21	KINGMA D P， BA J. Adam： a method for stochastic optimization ［EB/OL］. ［2020-12-22］. .
22	SHI W Z， CABALLERO J， HUSZÁR F， et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1874-1883. 10.1109/cvpr.2016.207
23	LAI W S， HUANG J B， AHUJA N， et al. Deep Laplacian pyramid networks for fast and accurate super-resolution ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5835-5843. 10.1109/cvpr.2017.618
24	TAI Y， YANG J， LIU X M. Image super-resolution via deep recursive residual network ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2790-2798. 10.1109/cvpr.2017.298
25	TAI Y， YANG J， LIU X M， et al. MemNet： a persistent memory network for image restoration ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 4549-4557. 10.1109/iccv.2017.486

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	36.71	0.953 6	32.32	0.905 2	29.54	0.896 2
ESPCN	37.00	0.955 9	32.75	0.909 8	29.87	0.906 5
FSRCNN	37.06	0.955 4	32.76	0.907 8	29.88	0.902 4
VDSR	37.53	0.958 3	33.05	0.910 7	30.79	0.915 7
DRCN	37.63	0.958 4	33.06	0.910 8	30.76	0.914 7
LapSRN	37.52	0.958 1	33.08	0.910 9	30.41	0.911 2
DRRN	37.74	0.959 1	33.23	0.913 6	31.23	0.918 8
MemNet	37.78	0.959 7	33.28	0.914 2	31.31	0.919 5
MSRN	38.08	0.960 5	33.74	0.917 0	32.22	0.932 6
本文方法	37.97	0.960 3	33.50	0.916 5	31.88	0.925 8

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	36.71	0.953 6	32.32	0.905 2	29.54	0.896 2
ESPCN	37.00	0.955 9	32.75	0.909 8	29.87	0.906 5
FSRCNN	37.06	0.955 4	32.76	0.907 8	29.88	0.902 4
VDSR	37.53	0.958 3	33.05	0.910 7	30.79	0.915 7
DRCN	37.63	0.958 4	33.06	0.910 8	30.76	0.914 7
LapSRN	37.52	0.958 1	33.08	0.910 9	30.41	0.911 2
DRRN	37.74	0.959 1	33.23	0.913 6	31.23	0.918 8
MemNet	37.78	0.959 7	33.28	0.914 2	31.31	0.919 5
MSRN	38.08	0.960 5	33.74	0.917 0	32.22	0.932 6
本文方法	37.97	0.960 3	33.50	0.916 5	31.88	0.925 8

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	32.47	0.906 7	29.23	0.820 1	26.25	0.802 8
ESPCN	33.02	0.913 5	29.49	0.827 1	26.41	0.816 1
FSRCNN	33.20	0.914 9	29.54	0.827 7	26.48	0.817 5
VDSR	33.68	0.920 1	29.86	0.831 2	27.15	0.831 5
DRCN	33.85	0.921 5	29.89	0.831 7	27.16	0.831 1
LapSRN	33.82	0.920 7	29.89	0.830 4	27.07	0.829 8
DRRN	34.03	0.924 4	29.96	0.834 9	27.53	0.837 8
MemNet	34.09	0.924 8	30.00	0.835 0	27.56	0.837 6
MSRN	34.38	0.926 2	30.34	0.839 5	28.08	0.855 4
本文方法	34.32	0.926 7	30.27	0.840 9	28.02	0.849 3

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	32.47	0.906 7	29.23	0.820 1	26.25	0.802 8
ESPCN	33.02	0.913 5	29.49	0.827 1	26.41	0.816 1
FSRCNN	33.20	0.914 9	29.54	0.827 7	26.48	0.817 5
VDSR	33.68	0.920 1	29.86	0.831 2	27.15	0.831 5
DRCN	33.85	0.921 5	29.89	0.831 7	27.16	0.831 1
LapSRN	33.82	0.920 7	29.89	0.830 4	27.07	0.829 8
DRRN	34.03	0.924 4	29.96	0.834 9	27.53	0.837 8
MemNet	34.09	0.924 8	30.00	0.835 0	27.56	0.837 6
MSRN	34.38	0.926 2	30.34	0.839 5	28.08	0.855 4
本文方法	34.32	0.926 7	30.27	0.840 9	28.02	0.849 3

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	30.50	0.857 3	27.62	0.745 3	24.53	0.723 6
ESPCN	30.66	0.864 6	27.71	0.756 2	24.60	0.736 0
FSRCNN	30.73	0.860 1	27.71	0.748 8	24.62	0.727 2
VDSR	31.36	0.879 6	28.11	0.762 4	25.18	0.754 3
DRCN	31.56	0.881 0	28.15	0.762 7	25.15	0.753 0
LapSRN	31.54	0.881 1	28.19	0.763 5	25.21	0.756 4
DRRN	31.68	0.888 8	28.21	0.772 1	25.44	0.763 8
MemNet	31.74	0.889 3	28.26	0.772 3	25.50	0.763 0
MSRN	32.07	0.890 3	28.60	0.775 1	26.04	0.789 6
本文方法	32.04	0.893 2	28.52	0.780 5	25.92	0.780 9

Single image super-resolution reconstruction method based on dense Inception

基于密集Inception的单图像超分辨率重建方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 25

Related Articles 15

Recommended Articles

Metrics

模型	参数量	平均训练时间/s	SSIM
SRCNN	0.057×10⁶	0.540	0.906 7
ESPCN	0.020×10⁶	0.410	0.913 5
FSRCNN	0.012×10⁶	0.360	0.914 9
VDSR	0.665×10⁶	2.900	0.920 1
DRCN	1.775×10⁶	5.800	0.921 5
LapSRN	0.812×10⁶	0.077	0.920 7
DRRN	0.297×10⁶	1.740	0.924 4
MemNet	0.677×10⁶	0.037	0.924 8
MSRN	6.300×10⁶	0.007	0.926 2
本文方法	1.400×10⁶	0.003	0.926 7

[1]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[2]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[3]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[4]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[5]	Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance [J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.
[6]	Jianjing LI, Guanfeng LI, Feizhou QIN, Weijun LI. Multi-relation approximate reasoning model based on uncertain knowledge graph embedding [J]. Journal of Computer Applications, 2024, 44(6): 1751-1759.
[7]	Min SUN, Qian CHENG, Xining DING. CBAM-CGRU-SVM based malware detection method for Android [J]. Journal of Computer Applications, 2024, 44(5): 1539-1545.
[8]	Wenshuo GAO, Xiaoyun CHEN. Point cloud classification network based on node structure [J]. Journal of Computer Applications, 2024, 44(5): 1471-1478.
[9]	Tianhua CHEN, Jiaxuan ZHU, Jie YIN. Bird recognition algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1114-1120.
[10]	Lijun XU, Hui LI, Zuyang LIU, Kansong CHEN, Weixuan MA. 3D-GA-Unet： MRI image segmentation algorithm for glioma based on 3D-Ghost CNN [J]. Journal of Computer Applications, 2024, 44(4): 1294-1302.
[11]	Jie WANG, Hua MENG. Image classification algorithm based on overall topological structure of point cloud [J]. Journal of Computer Applications, 2024, 44(4): 1107-1113.
[12]	Yongfeng DONG, Jiaming BAI, Liqin WANG, Xu WANG. Chinese named entity recognition combining prior knowledge and glyph features [J]. Journal of Computer Applications, 2024, 44(3): 702-708.
[13]	Ruifeng HOU, Pengcheng ZHANG, Liyuan ZHANG, Zhiguo GUI, Yi LIU, Haowen ZHANG, Shubin WANG. Iterative denoising network based on total variation regular term expansion [J]. Journal of Computer Applications, 2024, 44(3): 916-921.
[14]	Jingxian ZHOU, Xina LI. UAV detection and recognition based on improved convolutional neural network and radio frequency fingerprint [J]. Journal of Computer Applications, 2024, 44(3): 876-882.
[15]	Rui ZHANG, Siqi SONG, Jing HU, Yongmei ZHANG, Yanfeng CHAI. Performance evaluation of industry-university-research based on statistics and adaptive ParNet [J]. Journal of Computer Applications, 2024, 44(2): 628-637.