计算机应用 ›› 2020, Vol. 40 ›› Issue (3): 710-716.DOI: 10.11772/j.issn.1001-9081.2019071178

• 人工智能 • 上一篇    下一篇

基于组合学习的人脸超分辨率算法

许若波1,2, 卢涛1,2, 王宇1,2, 张彦铎1,2   

  1. 1. 武汉工程大学 计算机科学与工程学院, 武汉 430205;
    2. 智能机器人湖北省重点实验室(武汉工程大学), 武汉 430205
  • 收稿日期:2019-07-08 修回日期:2019-09-01 出版日期:2020-03-10 发布日期:2019-09-19
  • 通讯作者: 卢涛,
  • 作者简介:许若波(1990-),男,江苏徐州人,硕士研究生,主要研究方向:图像处理、计算机视觉;卢涛(1980-),男,湖北武汉人,副教授,博士,主要研究方向:图像/视频处理、计算机视觉、人工智能;王宇(1996-),男,湖北荆州人,硕士研究生,主要研究方向:图像处理、计算机视觉;张彦铎(1971-),男,黑龙江肇东人,教授,博士,主要研究方向:智能计算、智能机器人、智能检测、仿真系统。
  • 基金资助:
    国家自然科学基金资助项目(61502354);湖北省自然科学基金资助项目(2015CFB451);中央引导地方科技发展专项(2018ZYYD059);武汉工程大学科研基金资助项目(K201713);武汉工程大学研究生教育创新基金资助项目(CX2018211)。

Face hallucination algorithm via combined learning

XU Ruobo1,2, LU Tao1,2, WANG Yu1,2, ZHANG Yanduo1,2   

  1. 1. School of Computer Science and Engineering, Wuhan Institute of Technology, Wuhan Hubei 430205, China;
    2. Hubei Key Laboratory of Intelligent Robot(Wuhan Institute of Technology), Wuhan Hubei 430205, China
  • Received:2019-07-08 Revised:2019-09-01 Online:2020-03-10 Published:2019-09-19
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61502354), the Natural Science Foundation of Hubei Province (2015CFB451), the Special Fund of Central Guidance for Local Science and Technology Development (2018ZYYD059), the Scientific Research Foundation of Wuhan Institute of Technology (K201713), the Graduate Innovation Foundation of Wuhan Institute of Technology (CX2018211).

摘要: 现有的基于深度学习的人脸超分辨算法大部分仅仅利用一种网络分区重建高分辨率输出图像,并未考虑人脸图像中的结构性信息,导致了在人脸的重要器官重建上缺乏足够的细节信息。针对这一问题,提出一种基于组合学习的人脸超分辨率算法。该算法独立采用不同深度学习模型的优势重建感兴趣的区域,由此在训练网络的过程中每个人脸区域的数据分布不同,不同的子网络能够获得更精确的先验信息。首先,对人脸图像采用超像素分割算法生成人脸组件部分和人脸背景图像;然后,采用人脸组件生成对抗网络(C-GAN)独立重建人脸组件图像块,并采用人脸背景重建网络生成人脸背景图像;其次,使用人脸组件融合网络将两种不同模型重建的人脸组件图像块自适应融合;最后,将生成的人脸组件图像块合并至人脸背景图像中,重建出最终的人脸图像。在FEI数据集上的实验结果表明,与人脸图像超分辨率算法通过组件生成和增强学习幻构人脸图像(LCGE)及判决性增强的生成对抗网络(EDGAN)相比,所提算法的峰值信噪比(PSNR)值分别高出1.23 dB和1.11 dB。所提算法能够采用不同深度学习模型的优势组合学习重建更精准的人脸图像,同时拓展了图像重建先验的来源。

关键词: 组合学习, 人脸幻构, 生成对抗网络, 融合网络, 深度学习

Abstract: Most of the existing deep learning based face hallucination algorithms only use a single network partition to reconstruct high-resolution output images without considering the structural information in the face images, resulting in the lack of sufficient details in the reconstruction of vital organs on the face. Therefore, a face hallucination algorithm based on combined learning was proposed to tackle this problem. In the algorithm, the regions of interest were reconstructed independently by utilizing the advantages of different deep learning models, thus the data distribution of each face region was different to each other in the process of network training, and different sub-networks were able to obtain more accurate prior information. Firstly, for the face image, the superpixel segmentation algorithm was used to generate the facial component parts and facial background image. Secondly, the facial component image patches were independently reconstructed by the Component-Generative Adversarial Network (C-GAN) and the facial background reconstruction network was used to generate the facial background image. Thirdly, the facial component fusion network was used to adaptively fuse the facial component image patches reconstructed by two different models. Finally, the generated facial component image patches were merged into the facial background image to reconstruct the final face image. The experimental results on FEI dataset show that the Peak Signal to Noise Ratio (PSNR) of the proposed algorithm is respectively 1.23 dB and 1.11 dB higher than that of the face image hallucination algorithms Learning to hallucinate face images via Component Generation and Enhancement (LCGE) and Enhanced Discriminative Generative Adversarial Network (EDGAN). The proposed algorithm can perform combined learning of the advantages of different deep learning models to learn and reconstruct more accurate face images as well as expand the sources of image reconstruction prior information.

Key words: combined learning, face hallucination, Generative Adversarial Network (GAN), fusion network, deep learning

中图分类号: