基于面部特征图对称的人脸正面化生成对抗网络算法

doi:10.11772/j.issn.1001-9081.2020060779

计算机应用 ›› 2021, Vol. 41 ›› Issue (3): 714-720.DOI: 10.11772/j.issn.1001-9081.2020060779

所属专题：人工智能

基于面部特征图对称的人脸正面化生成对抗网络算法

李虹霞^1,2, 秦品乐^1,2, 闫寒梅³, 曾建潮^1,2, 鲍骞月^1,2, 柴锐^1,2

1. 中北大学大数据学院, 太原 030051;
2. 山西省医学影像与数据分析工程研究中心(中北大学), 太原 030051;
3. 山西警察学院刑事科学技术系, 太原 030401

收稿日期:2020-06-09 修回日期:2020-09-16 出版日期:2021-03-10 发布日期:2020-12-22
通讯作者: 秦品乐
作者简介:李虹霞(1996-),女,山西大同人,硕士研究生,主要研究方向:深度学习、计算机视觉;秦品乐(1978-),男,山西长治人,教授,博士,CCF会员,主要研究方向:机器视觉、大数据、医学影像分析;闫寒梅(1968-),女,山西天镇人,副教授,主要研究方向:刑事科学技术、刑事图像学;曾建潮(1963-),男,陕西大荔人,教授,博士生导师,博士,CCF会员,主要研究方向:复杂系统的维护决策、健康管理;鲍骞月(1998-),男,山西朔州人,主要研究方向:深度学习、机器视觉;柴锐(1985-),男,山西运城人,讲师,博士,主要研究方向:医学影像处理。
基金资助:
山西省重点研发项目（201803D31212-1）。

Face frontalization generative adversarial network algorithm based on face feature map symmetry

LI Hongxia^1,2, QIN Pinle^1,2, YAN Hanmei³, ZENG Jianchao^1,2, BAO Qianyue^1,2, CHAI Rui^1,2

1. School of Data Science and Technology, North University of China, Taiyuan Shanxi 030051, China;
2. Shanxi Medical Imaging and Data Analysis Engineering Research Center(North University of China), Taiyuan Shanxi 030051, China;
3. Science and Technology Department, Shanxi Police College, Taiyuan Shanxi 030401, China

Received:2020-06-09 Revised:2020-09-16 Online:2021-03-10 Published:2020-12-22
Supported by:
This work is partially supported by the Shanxi Provincial Key Research and Development Program (201803D31212-1).

摘要/Abstract

摘要： 目前人脸正面化研究主要解决人脸偏转问题，而对监控视频等现实场景中同时受偏转和俯仰变化影响的侧脸的正面化生成关注较少，针对这个问题和多角度侧脸生成的正面人脸图存在身份信息保留不全的问题，提出了一种基于特征图对称模块和眼周特征保留损失的生成对抗网络（GAN）。首先，根据人脸对称性先验，提出特征图对称模块，先使用人脸关键点检测器检测出侧脸鼻尖点位置，再将编码器提取到的特征图依照鼻尖位置进行镜像对称，从而在特征层面上缓解面部信息缺失的问题。其次，借鉴眼周识别思想，在现有的生成图身份保留方法中加入了眼周特征保留损失以训练生成器生成逼真的且保留身份信息的人脸正面图像。实验结果表明，所提算法得到的生成图面部细节保留较好，且在CAS-PEAL-R1数据集的所有俯角下人脸的平均Rank-1识别率为99.03%，可见该算法能够有效解决多角度侧脸的正面化问题。

关键词: 人脸识别, 人脸正面化, 生成对抗网络, 深度学习, 眼周识别

Abstract: At present, the research of face frontalization mainly solves the face yaw problem, and pays less attention to the face frontalization of the side face affected by yaw and pitch at the same time in real scenes such as surveillance video. Aiming at this problem and the problem of incomplete identity information retained in front face image generated by multi-angle side faces, a Generative Adversarial Network (GAN) based on feature map symmetry and periocular feature preserving loss was proposed. Firstly, according to the prior of face symmetry, a symmetry module of the feature map was proposed. The face key point detector was used to detect the position of nasal tip point, and mirror symmetry was performed to the feature map extracted by the encoder according to the nasal tip, so as to alleviate the lack of facial information at the feature level. Finally, benefiting from the idea of periocular recognition, the periocular feature preserving loss was added in the existing identity preserving method of generated image to train the generator to generate realistic and identity-preserving front face image. Experimental results show that the facial details of the images generated by the proposed algorithm were well preserved, and the average Rank-1 recognition rate of faces with all angles under the pitch of CAS-PEAL-R1 dataset is 99.03%, which can effectively solve the frontalization problem of multi-angle side faces.

Key words: face recognition, face frontalization, Generative Adversarial Network (GAN), deep learning, periocular recognition

中图分类号:

TP389.1

李虹霞, 秦品乐, 闫寒梅, 曾建潮, 鲍骞月, 柴锐. 基于面部特征图对称的人脸正面化生成对抗网络算法[J]. 计算机应用, 2021, 41(3): 714-720.

LI Hongxia, QIN Pinle, YAN Hanmei, ZENG Jianchao, BAO Qianyue, CHAI Rui. Face frontalization generative adversarial network algorithm based on face feature map symmetry[J]. Journal of Computer Applications, 2021, 41(3): 714-720.

参考文献

[1] TRAN L,YIN X,LIU X. Disentangled representation learning GAN for pose-invariant face recognition[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:1283-1292.
[2] CAO K,RONG Y,LI C,et al. Pose-robust face recognition via deep residual equivariant mapping[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:5187-5196.
[3] HU Y,WU X,YU B,et al. Pose-guided photorealistic face rotation[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:8398-8406.
[4] 钱一琛. 基于生成对抗的人脸正面化生成[D]. 北京:北京邮电大学,2019:73.(QIAN Y C. Generative adversarial networks based face frontalization[D]. Beijing:Beijing University of Posts and Telecommunications,2019:73.)
[5] KAN M,SHAN S,CHANG H,et al. Stacked Progressive AutoEncoders (SPAE) for face recognition across poses[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2014:1883-1890.
[6] YIM J,JUNG H,YOO B,et al. Rotating your face using multitask deep neural network[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:676-684.
[7] HUANG R,ZHANG S,LI T,et al. Beyond face rotation:global and local perception GAN for photorealistic and identity preserving frontal view synthesis[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:2458-2467.
[8] LI P,WU X,HU Y,et al. M2FPA:a multi-yaw multi-pitch highquality database and benchmark for facial pose analysis[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway:IEEE,2019:10042-10050.
[9] HASSNER T, HAREL S, PAZ E, et al. Effective face frontalization in unconstrained images[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:4295-4304.
[10] ZHU X,LEI Z,YAN J,et al. High-fidelity pose and expression normalization for face recognition in the wild[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:787-796.
[11] 丁立军, 冯浩, 华亮. Clifford代数3D人脸姿态矫正方法[J]. 小型微型计算机系统,2013,34(4):906-909.(DING L J,FENG H,HUA L. Clifford algebra approach for 3D face pose correction[J]. Journal of Chinese Computer Systems, 2013, 34(4):906-909.)
[12] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al.Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2014:2672-2680.
[13] ALONSO-FERNANDEZ F,BIGUN J. A survey on periocular biometrics research[J]. Pattern Recognition Letters,2016,82(Pt 2):92-105.
[14] ZHAO Z,KUMAR A. Improving periocular recognition by explicit attention to critical regions in deep neural network[J]. IEEE Transactions on Information Forensics and Security,2018,13(12):2937-2952.
[15] GAO W,CAO B,SHAN S,et al. The CAS-PEAL large-scale Chinese face database and baseline evaluations[J]. IEEE Transactions on Systems,Man,and Cybernetics-Part A:Systems and Humans,2008,38(1):149-161.
[16] YIN Y,JIANG S,ROBINSON J P,et al. Dual-attention GAN for large-pose face frontalization[EB/OL].[2020-04-25]. https://arxiv.org/pdf/2002.07227.pdf.
[17] DENG J,GUO J,ZHOU Y,et al. RetinaFace:single-stage dense face localisation in the wild[EB/OL].[2020-04-25]. https://arxiv.org/pdf/1905.00641.pdf.
[18] WU X,HE R,SUN Z,et al. A light CNN for deep face representation with noisy labels[J]. IEEE Transactions on Information Forensics and Security,2018,13(11):2884-2896.
[19] HERNANDEZ-DIAZ K,ALONSO-FERNANDEZ F,BIGUN J. Periocular recognition using CNN features off-the-shelf[C]//Proceedings of the 2018 International Conference of the Biometrics Special Interest Group. Piscataway:IEEE,2018:1-5
[20] OJALA T,PIETIKAINEN M,MAENPAA T. Multiresolution grayscale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(7):971-987.
[21] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2005:886-893.
[22] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision,2004,60(2):91-110.
[23] HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.
[24] ZHAO J,CHENG Y,XU Y,et al. Towards pose invariant face recognition in the wild[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:2207-2216.

基于面部特征图对称的人脸正面化生成对抗网络算法

Face frontalization generative adversarial network algorithm based on face feature map symmetry

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	陈成瑞, 孙宁, 何世彪, 廖勇. 面向C-V2X通信的基于深度学习的联合信道估计与均衡算法[J]. 计算机应用, 2021, 41(9): 2687-2693.
[2]	谢德峰, 吉建民. 融入句法感知表示进行句法增强的语义解析[J]. 计算机应用, 2021, 41(9): 2489-2495.
[3]	代雨柔, 杨庆, 张凤荔, 周帆. 基于自监督学习的社交网络用户轨迹预测模型[J]. 计算机应用, 2021, 41(9): 2545-2551.
[4]	郑志强, 胡鑫, 翁智, 王雨禾, 程曦. 基于改进DenseNet的牛眼图像特征提取方法[J]. 计算机应用, 2021, 41(9): 2780-2784.
[5]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[6]	高工, 杨红雨, 刘洪. 基于深度学习的三维点云人脸识别[J]. 计算机应用, 2021, 41(9): 2736-2740.
[7]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[8]	管其杰, 张挺, 李德亚, 周绍景, 杜奕. 基于多分辨率生成对抗网络的空间数据不确定性重建方法[J]. 计算机应用, 2021, 41(8): 2306-2311.
[9]	孙潇, 徐金东. 基于级联生成对抗网络的遥感图像去雾方法[J]. 计算机应用, 2021, 41(8): 2440-2444.
[10]	何正海, 线岩团, 王蒙, 余正涛. 融合句法指导与字符注意力机制的案情阅读理解方法[J]. 计算机应用, 2021, 41(8): 2427-2431.
[11]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[12]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[13]	杜炎, 吕良福, 焦一辰. 基于模糊推理的模糊原型网络[J]. 计算机应用, 2021, 41(7): 1885-1890.
[14]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.
[15]	李亚芳, 梁烨, 冯韦玮, 祖宝开, 康玉健. 基于社区优化的深度网络嵌入方法[J]. 计算机应用, 2021, 41(7): 1956-1963.