计算机应用 ›› 2018, Vol. 38 ›› Issue (12): 3557-3562.DOI: 10.11772/j.issn.1001-9081.2018051097

• 虚拟现实与多媒体计算 • 上一篇    下一篇

基于双鉴别网络的生成对抗网络图像修复方法

刘波宁, 翟东海   

  1. 西南交通大学 信息科学与技术学院, 成都 610031
  • 收稿日期:2018-05-28 修回日期:2018-07-12 出版日期:2018-12-10 发布日期:2018-12-15
  • 通讯作者: 刘波宁
  • 作者简介:刘波宁(1993-),男,广东梅州人,硕士研究生,主要研究方向:数字图像处理;翟东海(1974-),男,山西芮城人,副教授,博士,主要研究方向:数字图像处理、海量数据挖掘。
  • 基金资助:
    国家自然科学基金资助项目(61461048,61661047)。

Image completion method of generative adversarial networks based on two discrimination networks

LIU Boning, ZHAI Donghai   

  1. School of Information Science and Technology, Southwest Jiaotong University, Chengdu Sichuan 610031, China
  • Received:2018-05-28 Revised:2018-07-12 Online:2018-12-10 Published:2018-12-15
  • Contact: 刘波宁
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61461048, 61661047).

摘要: 针对现有神经网络图像修复方法的修复结果在视觉连通性上存在结构扭曲、训练过程中易陷入过度学习等问题,提出了一种基于双鉴别网络的生成对抗网络(GAN)图像修复方法。该方法的修复模型使用了修复网络、全局鉴别网络和局部鉴别网络。修复网络将待修复图像破损区域用相似信息填充后作为输入,极大地提高了生成图像的速度与质量;全局鉴别网络综合采用图像全局的边缘结构信息和特征信息以保证修复网络输出的修复图像结果符合视觉连通性;而局部鉴别网络在鉴别输出图像的同时,利用在多个图像中寻找到的辅助特征块来提高鉴别的泛化能力,很好地抑制了修复网络在特征过于集中或单一时容易过度学习的问题。实验结果表明,所提修复方法在人脸类图像上具有较好的修复效果,且在不同种类图像上有非常好的适用性,其峰值信噪比(PSNR)和结构相似性(SSIM)指标比当前基于深度学习且修复效果较好的几种方法更优。

关键词: 生成对抗网络, 边缘结构, 人脸修复, 缓冲池, 卷积神经网络

Abstract: The existing image completion methods have the problems of structural distortion on visual connectivity and easy to overfitting in the process of training. In order to solve the problems, a new image completion method of Generative Adversarial Network (GAN) based on two discrimination networks was proposed. One completion network, one global discrimination network and one local discrimination network were used in the completion model of the proposed method. The broken area of image to be completed was filled by a similar patch as input in the completion network, which greatly improved the speed and quality of the generation images. The global marginal structure information and feature information were used comprehensively in the global discrimination network to ensure that the completed image of completion network conformed visual connectivity. While discriminating the output image, the assisted feature patches found from multiple images were used to improve the generalization ability of discrimination in the local discrimination network, which solved the issue that the completion network was easily overfitting with too concentrated features or single feature. The experimental results show that, the proposed completion method has good completion effect on face images, and has good applicability in different kinds of images. The Peak Signal-to-Noise Ratio (PSNR) and Structural SIMilarity (SSIM) of the proposed method are better than those of the state-of-the-art methods based on deep learning.

Key words: Generative Adversarial Network (GAN), marginal structure, face completion, cache pool, Convolution Neural Network (CNN)

中图分类号: