Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (3): 714-720.DOI: 10.11772/j.issn.1001-9081.2020060779

Special Issue: 人工智能

• Artificial intelligence • Previous Articles     Next Articles

Face frontalization generative adversarial network algorithm based on face feature map symmetry

LI Hongxia1,2, QIN Pinle1,2, YAN Hanmei3, ZENG Jianchao1,2, BAO Qianyue1,2, CHAI Rui1,2   

  1. 1. School of Data Science and Technology, North University of China, Taiyuan Shanxi 030051, China;
    2. Shanxi Medical Imaging and Data Analysis Engineering Research Center(North University of China), Taiyuan Shanxi 030051, China;
    3. Science and Technology Department, Shanxi Police College, Taiyuan Shanxi 030401, China
  • Received:2020-06-09 Revised:2020-09-16 Online:2021-03-10 Published:2020-12-22
  • Supported by:
    This work is partially supported by the Shanxi Provincial Key Research and Development Program (201803D31212-1).


李虹霞1,2, 秦品乐1,2, 闫寒梅3, 曾建潮1,2, 鲍骞月1,2, 柴锐1,2   

  1. 1. 中北大学 大数据学院, 太原 030051;
    2. 山西省医学影像与数据分析工程研究中心(中北大学), 太原 030051;
    3. 山西警察学院 刑事科学技术系, 太原 030401
  • 通讯作者: 秦品乐
  • 作者简介:李虹霞(1996-),女,山西大同人,硕士研究生,主要研究方向:深度学习、计算机视觉;秦品乐(1978-),男,山西长治人,教授,博士,CCF会员,主要研究方向:机器视觉、大数据、医学影像分析;闫寒梅(1968-),女,山西天镇人,副教授,主要研究方向:刑事科学技术、刑事图像学;曾建潮(1963-),男,陕西大荔人,教授,博士生导师,博士,CCF会员,主要研究方向:复杂系统的维护决策、健康管理;鲍骞月(1998-),男,山西朔州人,主要研究方向:深度学习、机器视觉;柴锐(1985-),男,山西运城人,讲师,博士,主要研究方向:医学影像处理。
  • 基金资助:

Abstract: At present, the research of face frontalization mainly solves the face yaw problem, and pays less attention to the face frontalization of the side face affected by yaw and pitch at the same time in real scenes such as surveillance video. Aiming at this problem and the problem of incomplete identity information retained in front face image generated by multi-angle side faces, a Generative Adversarial Network (GAN) based on feature map symmetry and periocular feature preserving loss was proposed. Firstly, according to the prior of face symmetry, a symmetry module of the feature map was proposed. The face key point detector was used to detect the position of nasal tip point, and mirror symmetry was performed to the feature map extracted by the encoder according to the nasal tip, so as to alleviate the lack of facial information at the feature level. Finally, benefiting from the idea of periocular recognition, the periocular feature preserving loss was added in the existing identity preserving method of generated image to train the generator to generate realistic and identity-preserving front face image. Experimental results show that the facial details of the images generated by the proposed algorithm were well preserved, and the average Rank-1 recognition rate of faces with all angles under the pitch of CAS-PEAL-R1 dataset is 99.03%, which can effectively solve the frontalization problem of multi-angle side faces.

Key words: face recognition, face frontalization, Generative Adversarial Network (GAN), deep learning, periocular recognition

摘要: 目前人脸正面化研究主要解决人脸偏转问题,而对监控视频等现实场景中同时受偏转和俯仰变化影响的侧脸的正面化生成关注较少,针对这个问题和多角度侧脸生成的正面人脸图存在身份信息保留不全的问题,提出了一种基于特征图对称模块和眼周特征保留损失的生成对抗网络(GAN)。首先,根据人脸对称性先验,提出特征图对称模块,先使用人脸关键点检测器检测出侧脸鼻尖点位置,再将编码器提取到的特征图依照鼻尖位置进行镜像对称,从而在特征层面上缓解面部信息缺失的问题。其次,借鉴眼周识别思想,在现有的生成图身份保留方法中加入了眼周特征保留损失以训练生成器生成逼真的且保留身份信息的人脸正面图像。实验结果表明,所提算法得到的生成图面部细节保留较好,且在CAS-PEAL-R1数据集的所有俯角下人脸的平均Rank-1识别率为99.03%,可见该算法能够有效解决多角度侧脸的正面化问题。

关键词: 人脸识别, 人脸正面化, 生成对抗网络, 深度学习, 眼周识别

CLC Number: