Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (3): 825-831.DOI: 10.11772/j.issn.1001-9081.2019081471

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Image inpainting based on dilated convolution

FENG Lang1,2,3, ZHANG Ling1,2,3, ZHANG Xiaolong1,2,3   

  1. 1. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan Hubei 430065, China;
    2. Institute of Big Data Science and Engineering, Wuhan University of Science and Technology, Wuhan Hubei 430065, China;
    3. Hubei Key Laboratory of Intelligent Information Processing and Real-Time Industrial System(Wuhan University of Science and Technology), Wuhan Hubei 430065, China
  • Received:2019-08-23 Revised:2019-10-16 Online:2020-03-10 Published:2019-10-31
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61702381, U1803262).

基于扩张卷积的图像修复

冯浪1,2,3, 张玲1,2,3, 张晓龙1,2,3   

  1. 1. 武汉科技大学 计算机科学与技术学院, 武汉 430065;
    2. 武汉科技大学 大数据科学与工程研究院, 武汉 430065;
    3. 智能信息处理与实时工业系统湖北省重点实验室(武汉科技大学), 武汉 430065
  • 通讯作者: 张晓龙
  • 作者简介:冯浪(1994-),男,湖北仙桃人,硕士研究生,主要研究方向:深度学习、图像处理;张玲(1986-),女,湖北黄冈人,博士,主要研究方向:图像和视频处理、计算机视觉;张晓龙(1963-),男,江西吉安人,教授,博士,主要研究方向:人工智能、机器学习、数据挖掘、生物信息处理。
  • 基金资助:
    国家自然科学基金资助项目(61702381,U1803262)。

Abstract: Although the existing image inpainting methods can recover the content of the missing area of the image, there are still some problems, such as structure distortion, texture blurring and content discontinuity, so that the inpainted images cannot meet people’s visual requirements. To solve these problems, an image inpainting method based on dilated convolution was proposed. By introducing the idea of dilated convolution to increase the receptive field, the quality of image inpainting was improved. This method was based on the idea of Generative Adversarial Network (GAN), which was divided into generative network and adversarial network. The generative network included global content inpainting network and local detail inpainting network, and gated convolution was used to realize the dynamical learning of the image features, solving the problem that the traditional convolution neural network method was not able to complete the large irregular missing areas well. Firstly, the global content inpainting network was used to obtain an initial content completion result, and then the local texture details were repaired by the local detail inpainting network. The adversarial network was composed of SN-PatchGAN discriminator, and was used to evaluate the image inpainting effect. Experimental results show that compared with the current image inpainting methods, the proposed method has great improvement in Peak Signal-to-Noise Ratio (PSNR), Structural SIMilarity (SSIM) and inception score. Moreover, the method effectively solves the problem of texture blurring in traditional inpainting methods, and meets people’s visual requirements better, verifying the validity and feasibility of the proposed method.

Key words: image inpainting, dilated convolution, Generative Adversarial Network (GAN), texture information, SN-PatchGAN discriminator

摘要: 现有图像修复方法虽然能够补全图像缺失区域的内容,但是仍然存在结构扭曲、纹理模糊、内容不连贯等问题,无法满足人们视觉上的要求。针对这些问题,提出一种基于扩张卷积的图像修复方法,通过引入扩张卷积的思想增大感受野来提升图像修复的质量。该方法基于生成对抗网络(GAN)的思想,分为生成网络和对抗网络。生成网络包括全局内容修复网络和局部细节修复网络,并使用gated卷积动态地学习图像特征,解决了使用传统卷积神经网络方法无法较好地补全大面积不规则缺失区域的问题。首先利用全局内容修复网络获得一个初始的内容补全结果,之后再通过局部细节修复网络对局部纹理细节进行修复。对抗网络由SN-PatchGAN鉴别器构成,用于评判图像修复效果的好坏。实验结果表明,与目前存在的图像修复方法相比,该方法在峰值信噪比(PSNR)、结构相似性(SSIM)、inception分数3个指标上都有较大的提升;而且该方法有效解决了传统修复方法出现的纹理模糊问题,较好地满足了人们的视觉连贯性,证实了提出的方法的有效性和可行性。

关键词: 图像修复, 扩张卷积, 生成对抗网络, 纹理信息, SN-PatchGAN鉴别器

CLC Number: