基于密集Inception的单图像超分辨率重建方法

doi:10.11772/j.issn.1001-9081.2021010070

《计算机应用》唯一官方网站 ›› 2021, Vol. 41 ›› Issue (12): 3666-3671.DOI: 10.11772/j.issn.1001-9081.2021010070

• 多媒体计算与计算机仿真 • 上一篇下一篇

基于密集Inception的单图像超分辨率重建方法

王海勇¹, 张开心²(), 管维正²

^1.南京邮电大学计算机学院，南京 210023
^2.南京邮电大学物联网学院，南京 210003

收稿日期:2021-01-15 修回日期:2021-03-29 接受日期:2021-04-06 发布日期:2021-04-15 出版日期:2021-12-10
通讯作者: 张开心
作者简介:王海勇（1979—），男，江苏连云港人，副研究员，博士，CCF会员，主要研究方向：计算机网络与安全、信息网络
管维正（1992—），男，湖北孝感人，硕士研究生，主要研究方向：机器学习。
基金资助:
赛尔网络下一代互联网技术创新项目(NGII20190612);江苏省博士后科研资助计划项目(2020Z058)

Single image super-resolution reconstruction method based on dense Inception

Haiyong WANG¹, Kaixin ZHANG²(), Weizheng GUAN²

^1.College of Computer Science，Nanjing University of Posts and Telecommunications，Nanjing Jiangsu 210023，China
^2.College of Internet of Things，Nanjing University of Posts and Telecommunications，Nanjing Jiangsu 210003，China

Received:2021-01-15 Revised:2021-03-29 Accepted:2021-04-06 Online:2021-04-15 Published:2021-12-10
Contact: Kaixin ZHANG
About author:WANG Haiyong， born in 1979， Ph. D.， associate research fellow. His research interests include computer network and security， information network.
GUAN Weizheng， born in 1992， M. S. candidate. His research interests include machine learning.
Supported by:
the CERNET Innovation Project(NGII20190612);the Postdoctoral Research Funding Program of Jiangsu Province(2020Z058)

摘要/Abstract

摘要：

近几年，基于卷积神经网络（CNN）的单图像超分辨率（SR）重建方法成为了主流。通常情况下，重建模型的网络层数越深，提取的特征越多，重建效果越好；然而随着网络层数的加深，不仅会出现梯度消失的问题，还会显著增加参数量，增加训练的难度。针对以上问题，提出了一种基于密集Inception的单图像SR重建方法。该方法引入Inception-残差网络（Inception-ResNet）结构提取图像特征，全局采用简化后的密集网络，且仅构建每一个模块输出到重建层的路径，从而避免产生冗余数据来增加计算量。在放大倍数为4时，采用数据集Set5测试模型性能，结果显示与超深卷积神经网络的图像超分辨率（VDSR）相比，所提方法的结构相似性（SSIM）高了0.013 6；与基于多尺度残差网络的图像SR（MSRN）相比，SSIM高了0.002 9，模型参数量少了78%。实验结果表明，所提方法在保证模型的深度和宽度的情况下，显著减少了参数量，从而降低了训练的难度，而且取得了比对比方法更好的峰值信噪比（PSNR）和SSIM。

关键词: 图像超分辨率重建, 卷积神经网络, 密集网络, Inception-残差网络, 残差模块

Abstract:

In recent years， the single image Super-Resolution （SR） reconstruction methods based on Convolutional Neural Network （CNN） have become mainstream. Under normal circumstances， the deeper network layers of the reconstruction model have， the more features are extracted， and the better reconstruction effect is. However， as the number of network layers increases， the reconstruction model will not only have the vanishing gradient problem， but also significantly increase the number of parameters and increase the difficulty of training. To solve the above problems， a single image SR reconstruction method based on dense Inception was proposed. In the proposed method， the image features were extracted by introducing the Inception-Residual Network （Inception-ResNet） structure， and the simplified dense network was adopted globally. And only the path that each module outputs to the reconstruction layer was constructed， avoiding the increase of computation amount caused by the generation of redundant data. When the magnification was 4， the dataset Set5 was used to test the model performance. The results show that， the Structural SIMilarity （SSIM） of the proposed model is 0.013 6 higher than that of accurate image Super-Resolution using Very Deep convolutional network （VDSR）， and the proposed method has the SSIM 0.002 9 higher and the model parameters 78% smaller than Multi-scale residual Network for Image Super-Resolution （MSRN）. The experimental results show that， under the premise of ensuring the depth and width of the model， the proposed method significantly reduces the number of parameters and the difficulty of training. In the meantime， the proposed method can achieve better Peak Signal-to-Noise Ratio （PSNR） and SSIM than the comparison methods.

Key words: image super-resolution reconstruction, Convolutional Neural Network (CNN), dense network, Inception-Residual Network (Inception-ResNet), residual module

中图分类号:

TP181

王海勇, 张开心, 管维正. 基于密集Inception的单图像超分辨率重建方法[J]. 计算机应用, 2021, 41(12): 3666-3671.

Haiyong WANG, Kaixin ZHANG, Weizheng GUAN. Single image super-resolution reconstruction method based on dense Inception[J]. Journal of Computer Applications, 2021, 41(12): 3666-3671.

图/表 8

参考文献 25

1	曾接贤，倪申龙.改进的卷积神经网络单幅图像超分辨率重建［J］.计算机工程与应用，2019，55（13）：1-7.
	ZENG J X， NI S L. Improved super-resolution reconstruction of single image based on convolution neural network ［J］. Computer Engineering and Applications， 2019， 55（13）： 1-7.
2	CHEN C， QI F. Single image super-resolution using deep CNN with dense skip connections and inception-ResNet ［C］// Proceedings of the 2018 9th International Conference on Information Technology in Medicine and Education. Piscataway： IEEE， 2018： 999-1003. 10.1109/itme.2018.00222
3	HUANG G， LIU Z， MAATEN L VAN DER， et al. Densely connected convolutional networks ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269. 10.1109/cvpr.2017.243
4	SZEGEDY C， LIU W， JIA Y Q， et al. Going deeper with convolutions ［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 1-9. 10.1109/cvpr.2015.7298594
5	SZEGEDY C， IOFFE S， VANHOUCKE V， et al. Inception-v4， Inception-ResNet and the impact of residual connections on learning ［C］// Proceedings of the 2017 31st AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2017： 4278-4284.
6	DONG C， LOY C C， HE K M， et al. Learning a deep convolutional network for image super-resolution ［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS8692. Cham： Springer， 2014： 184-199.
7	KIM J， LEE J K， LEE K M. Accurate image super-resolution using very deep convolutional networks ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1646-1654. 10.1109/cvpr.2016.182
8	DONG C， LOY C C， TANG X O. Accelerating the super-resolution convolutional neural network ［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS9906. Cham： Springer， 2016： 391-407.
9	KIM J， LEE J K， LEE K M. Deeply-recursive convolutional network for image super-resolution ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1637-1645. 10.1109/cvpr.2016.181
10	TONG T， LI G， LIU X J， et al. Image super-resolution using dense skip connections ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 4809-4817. 10.1109/iccv.2017.514
11	ZHANG Y L， TIAN Y P， KONG Y， et al. Residual dense network for image super-resolution ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2472-2481. 10.1109/cvpr.2018.00262
12	TANG Z M， LI S H， LUO L K， et al. Image super-resolution via simplified dense network with non-degenerate layers ［J］. IEEE Access， 2019， 7： 24775-24787. 10.1109/access.2019.2898846
13	IOFFE S， SZEGEDY C. Batch normalization： accelerating deep network training by reducing internal covariate shift ［C］// Proceedings of the 2015 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 448-456.
14	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826. 10.1109/cvpr.2016.308
15	LI J C， FANG F M， MEI K F， et al. Multi-scale residual network for image super-resolution ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11212. Cham： Springer， 2018： 527-542.
16	QIN J Y， SUN X F， YAN Y T， et al. Multi-resolution space-attended residual dense network for single image super-resolution ［J］. IEEE Access， 2020， 8： 40499-40511. 10.1109/access.2020.2976478
17	TIMOFTE R， AGUSTSSON E， GOOL L VAN， et al. NTIRE 2017 challenge on single image super-resolution： methods and results ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2017： 1110-1121.
18	BEVILACQUA M， ROUMY A， GUILLEMOT C， et al. Low-complexity single-image super-resolution based on nonnegative neighbor embedding ［C］// Proceedings of the 2012 British Machine Vision Conference. Durham： BMVA Press， 2012： Article No.135. 10.5244/c.26.135
19	ZEYDE R， ELAD M， PROTTER M. On single image scale-up using sparse-representations ［C］// Proceedings of the 2010 7th International Conference on Curves and Surfaces， LNCS6920. Berlin： Springer， 2010： 711-730.
20	HUANG J B， SINGH A， AHUJA N. Single image super-resolution from transformed self-exemplars ［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 5197-5206. 10.1109/cvpr.2015.7299156
21	KINGMA D P， BA J. Adam： a method for stochastic optimization ［EB/OL］. ［2020-12-22］. .
22	SHI W Z， CABALLERO J， HUSZÁR F， et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1874-1883. 10.1109/cvpr.2016.207
23	LAI W S， HUANG J B， AHUJA N， et al. Deep Laplacian pyramid networks for fast and accurate super-resolution ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5835-5843. 10.1109/cvpr.2017.618
24	TAI Y， YANG J， LIU X M. Image super-resolution via deep recursive residual network ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2790-2798. 10.1109/cvpr.2017.298
25	TAI Y， YANG J， LIU X M， et al. MemNet： a persistent memory network for image restoration ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 4549-4557. 10.1109/iccv.2017.486

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	36.71	0.953 6	32.32	0.905 2	29.54	0.896 2
ESPCN	37.00	0.955 9	32.75	0.909 8	29.87	0.906 5
FSRCNN	37.06	0.955 4	32.76	0.907 8	29.88	0.902 4
VDSR	37.53	0.958 3	33.05	0.910 7	30.79	0.915 7
DRCN	37.63	0.958 4	33.06	0.910 8	30.76	0.914 7
LapSRN	37.52	0.958 1	33.08	0.910 9	30.41	0.911 2
DRRN	37.74	0.959 1	33.23	0.913 6	31.23	0.918 8
MemNet	37.78	0.959 7	33.28	0.914 2	31.31	0.919 5
MSRN	38.08	0.960 5	33.74	0.917 0	32.22	0.932 6
本文方法	37.97	0.960 3	33.50	0.916 5	31.88	0.925 8

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	36.71	0.953 6	32.32	0.905 2	29.54	0.896 2
ESPCN	37.00	0.955 9	32.75	0.909 8	29.87	0.906 5
FSRCNN	37.06	0.955 4	32.76	0.907 8	29.88	0.902 4
VDSR	37.53	0.958 3	33.05	0.910 7	30.79	0.915 7
DRCN	37.63	0.958 4	33.06	0.910 8	30.76	0.914 7
LapSRN	37.52	0.958 1	33.08	0.910 9	30.41	0.911 2
DRRN	37.74	0.959 1	33.23	0.913 6	31.23	0.918 8
MemNet	37.78	0.959 7	33.28	0.914 2	31.31	0.919 5
MSRN	38.08	0.960 5	33.74	0.917 0	32.22	0.932 6
本文方法	37.97	0.960 3	33.50	0.916 5	31.88	0.925 8

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	32.47	0.906 7	29.23	0.820 1	26.25	0.802 8
ESPCN	33.02	0.913 5	29.49	0.827 1	26.41	0.816 1
FSRCNN	33.20	0.914 9	29.54	0.827 7	26.48	0.817 5
VDSR	33.68	0.920 1	29.86	0.831 2	27.15	0.831 5
DRCN	33.85	0.921 5	29.89	0.831 7	27.16	0.831 1
LapSRN	33.82	0.920 7	29.89	0.830 4	27.07	0.829 8
DRRN	34.03	0.924 4	29.96	0.834 9	27.53	0.837 8
MemNet	34.09	0.924 8	30.00	0.835 0	27.56	0.837 6
MSRN	34.38	0.926 2	30.34	0.839 5	28.08	0.855 4
本文方法	34.32	0.926 7	30.27	0.840 9	28.02	0.849 3

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	32.47	0.906 7	29.23	0.820 1	26.25	0.802 8
ESPCN	33.02	0.913 5	29.49	0.827 1	26.41	0.816 1
FSRCNN	33.20	0.914 9	29.54	0.827 7	26.48	0.817 5
VDSR	33.68	0.920 1	29.86	0.831 2	27.15	0.831 5
DRCN	33.85	0.921 5	29.89	0.831 7	27.16	0.831 1
LapSRN	33.82	0.920 7	29.89	0.830 4	27.07	0.829 8
DRRN	34.03	0.924 4	29.96	0.834 9	27.53	0.837 8
MemNet	34.09	0.924 8	30.00	0.835 0	27.56	0.837 6
MSRN	34.38	0.926 2	30.34	0.839 5	28.08	0.855 4
本文方法	34.32	0.926 7	30.27	0.840 9	28.02	0.849 3

模型	Set5		Set14		Urban100
模型	PSNR/dB	SSIM	PSNR/dB	SSIM	PSNR/dB	SSIM
SRCNN	30.50	0.857 3	27.62	0.745 3	24.53	0.723 6
ESPCN	30.66	0.864 6	27.71	0.756 2	24.60	0.736 0
FSRCNN	30.73	0.860 1	27.71	0.748 8	24.62	0.727 2
VDSR	31.36	0.879 6	28.11	0.762 4	25.18	0.754 3
DRCN	31.56	0.881 0	28.15	0.762 7	25.15	0.753 0
LapSRN	31.54	0.881 1	28.19	0.763 5	25.21	0.756 4
DRRN	31.68	0.888 8	28.21	0.772 1	25.44	0.763 8
MemNet	31.74	0.889 3	28.26	0.772 3	25.50	0.763 0
MSRN	32.07	0.890 3	28.60	0.775 1	26.04	0.789 6
本文方法	32.04	0.893 2	28.52	0.780 5	25.92	0.780 9

基于密集Inception的单图像超分辨率重建方法

Single image super-resolution reconstruction method based on dense Inception

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 8

参考文献 25

相关文章 15

编辑推荐

Metrics

模型	参数量	平均训练时间/s	SSIM
SRCNN	0.057×10⁶	0.540	0.906 7
ESPCN	0.020×10⁶	0.410	0.913 5
FSRCNN	0.012×10⁶	0.360	0.914 9
VDSR	0.665×10⁶	2.900	0.920 1
DRCN	1.775×10⁶	5.800	0.921 5
LapSRN	0.812×10⁶	0.077	0.920 7
DRRN	0.297×10⁶	1.740	0.924 4
MemNet	0.677×10⁶	0.037	0.924 8
MSRN	6.300×10⁶	0.007	0.926 2
本文方法	1.400×10⁶	0.003	0.926 7

[1]	李云, 王富铕, 井佩光, 王粟, 肖澳. 基于不确定度感知的帧关联短视频事件检测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2903-2910.
[2]	秦璟, 秦志光, 李发礼, 彭悦恒. 基于概率稀疏自注意力神经网络的重性抑郁疾患诊断[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2970-2974.
[3]	陈虹, 齐兵, 金海波, 武聪, 张立昂. 融合1D-CNN与BiGRU的类不平衡流量异常检测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2493-2499.
[4]	赵宇博, 张丽萍, 闫盛, 侯敏, 高茂. 基于改进分段卷积神经网络和知识蒸馏的学科知识实体间关系抽取[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2421-2429.
[5]	张春雪, 仇丽青, 孙承爱, 荆彩霞. 基于两阶段动态兴趣识别的购买行为预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2365-2371.
[6]	高阳峄, 雷涛, 杜晓刚, 李岁永, 王营博, 闵重丹. 基于像素距离图和四维动态卷积网络的密集人群计数与定位方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 2233-2242.
[7]	王东炜, 刘柏辰, 韩志, 王艳美, 唐延东. 基于低秩分解和向量量化的深度网络压缩方法[J]. 《计算机应用》唯一官方网站, 2024, 44(7): 1987-1994.
[8]	姚迅, 秦忠正, 杨捷. 生成式标签对抗的文本分类模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1781-1785.
[9]	黄梦源, 常侃, 凌铭阳, 韦新杰, 覃团发. 基于层间引导的低光照图像渐进增强算法[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1911-1919.
[10]	李健京, 李贯峰, 秦飞舟, 李卫军. 基于不确定知识图谱嵌入的多关系近似推理模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1751-1759.
[11]	沈君凤, 周星辰, 汤灿. 基于改进的提示学习方法的双通道情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1796-1806.
[12]	高文烁, 陈晓云. 基于节点结构的点云分类网络[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1471-1478.
[13]	孙敏, 成倩, 丁希宁. 基于CBAM-CGRU-SVM的Android恶意软件检测方法[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1539-1545.
[14]	席治远, 唐超, 童安炀, 王文剑. 基于双路时空网络的驾驶员行为识别[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1511-1519.
[15]	王杰, 孟华. 基于点云整体拓扑结构的图像分类算法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1107-1113.