Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (5): 1440-1447.DOI: 10.11772/j.issn.1001-9081.2018091887

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Single image super-resolution reconstruction method based on improved convolutional neural network

LIU Yuefeng, YANG Hanxi, CAI Shuang, ZHANG Chenrong   

  1. School of Information Engineering, Inner Mongolia University of Science & Technology, Baotou Inner Mongolia 014000, China
  • Received:2018-09-10 Revised:2018-11-19 Online:2019-05-14 Published:2019-05-10
  • Supported by:
    This work is partially supported by the Inner Mongolia Natural Science Foundation (2018MS06019).

基于改进卷积神经网络的单幅图像超分辨率重建方法

刘月峰, 杨涵晰, 蔡爽, 张晨荣   

  1. 内蒙古科技大学 信息工程学院, 内蒙古 包头 014000
  • 通讯作者: 刘月峰
  • 作者简介:刘月峰(1977-),男,内蒙古包头人,副教授,博士,主要研究方向:深度学习、图像处理;杨涵晰(1994-),女,内蒙古包头人,硕士研究生,主要研究方向:深度学习、图像处理;蔡爽(1993-),女,山东菏泽人,硕士研究生,主要研究方向:深度学习、入侵检测;张晨荣(1993-),男,内蒙古巴彦淖尔人,硕士研究生,主要研究方向:深度学习、知识图谱。
  • 基金资助:
    内蒙古自然科学基金资助项目(2018MS06019)。

Abstract: Aiming at the problem of edge distortion and fuzzy texture detail information in reconstructed images, an image super-resolution reconstruction method based on improved Convolutional Neural Network (CNN) was proposed. Firstly, various preprocessing operations were performed on the underlying feature extraction layer by three interpolation methods and five sharpening methods, and the images which were only subjected to one interpolation operation and the images which were sharpened after interpolation operation were arranged into a 3D matrix. Then, the 3D feature map formed by the preprocessing was used as the multi-channel input of a deep residual network in the nonlinear mapping layer to obtain deeper texture detail information. Finally, for reducing image reconstruction time, sub-pixel convolution was introduced into the reconstruction layer to complete image reconstruction operation. Experimental results on several common datasets show that the proposed method achieves better restored texture detail information and high-frequency information in the reconstructed image compared with the classical methods. Furthermore, the Peak Signal-to-Noise Ratio (PSNR) was increased by 0.23 dB on average, and the structural similarity was increased by 0.0066 on average. The proposed method can better maintain the texture details of the reconstructed image and reduce the image edge distortion under the premise of ensuring the image reconstruction time, improving the performance of image reconstruction.

Key words: single image super-resolution reconstruction, deep learning, Convolutional Neural Network (CNN), multi-channel convolution, sub-pixel convolution

摘要: 对于重建图像存在的边缘失真和纹理细节信息模糊的问题,提出一种基于改进卷积神经网络(CNN)的图像超分辨率重建方法。首先在底层特征提取层以三种插值方法和五种锐化方法进行多种预处理操作,并将只进行一次插值操作的图像和先进行一次插值后进行一次锐化的图像合并排列成三维矩阵;然后在非线性映射层将预处理后构成的三维特征映射作为深层残差网络的多通道输入,以获取更深层次的纹理细节信息;最后在重建层为减少图像重建时间在网络结构中引入亚像素卷积来完成图像重建操作。在多个常用数据集上的实验结果表明,与经典方法相比,所提方法重建图像的纹理细节信息和高频信息能得到更好的恢复,峰值信噪比(PSNR)平均增加0.23 dB,结构相似性(SSIM)平均增加0.0066。在保证图像重建时间的前提下,所提方法更好地保持重建图像的纹理细节并减少图像边缘失真,提升重建图像的性能。

关键词: 单幅图像超分辨率重建, 深度学习, 卷积神经网络, 多通道卷积, 亚像素卷积

CLC Number: