Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (11): 3306-3313.DOI: 10.11772/j.issn.1001-9081.2020030420

• Virtual reality and multimedia computing • Previous Articles     Next Articles

3D face reconstruction and dense face alignment method based on improved 3D morphable model

ZHOU Jian, HUANG Zhangjin   

  1. School of Computer Science and Technology, University of Science and Technology of China, Hefei Anhui 230027, China
  • Received:2020-04-07 Revised:2020-05-15 Online:2020-11-10 Published:2020-06-03
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61877056), the Fundamental Research Funds for the Central Universities (WK6030000109).

基于改进三维形变模型的三维人脸重建和密集人脸对齐方法

周健, 黄章进   

  1. 中国科学技术大学 计算机科学与技术学院, 合肥 230027
  • 通讯作者: 黄章进(1980-),男,湖北天门人,副教授,博士,主要研究方向:计算机视觉、机器学习、计算机图形学;zhuang@ustc.edu.cn
  • 作者简介:周健(1995-),男,河南卫辉人,硕士研究生,主要研究方向:计算机视觉、计算机图形学、三维人脸重建
  • 基金资助:
    国家自然科学基金资助项目(61877056);中央高校基本科研业务费专项基金资助项目(WK6030000109)。

Abstract: In order to solve the problem that the currently widely used 3D morphable model has insufficient expression ability, resulting in poor generalization performance of the reconstructed 3D face model, a novel method for 3D face reconstruction and dense face alignment based on a single face image under unknown pose, expression and illumination was proposed. First, the existing 3D morphable model was improved by convolutional neural network to improve the expression ability of the 3D face model. Then, based on the smoothness of the face and the similarity of the image, a new loss function was proposed at the feature point and pixel level, and the weakly-supervised learning was used to train the convolutional neural network model. Finally, the trained network model was used to perform the 3D face reconstruction and dense face alignment. Experimental results show that, for 3D face reconstruction, the proposed model has the normalized mean error on AFLW2000-3D reduced to 2.25, and for dense face alignment, the proposed model has the normalized mean errors on AFLW2000-3D and AFLW-LFPA reduced to 3.80 and 3.34 respectively. Compared with the original method using 3D morphable model, the proposed model has the normalized mean errors reduced by 7.4% and 7.8% respectively in 3D face reconstruction and dense face alignment. Therefore, for face images with different lighting environments and angles, this network model is accurate in reconstruction and robust, and has high 3D face reconstruction and dense face alignment quality.

Key words: 3D face reconstruction, dense face alignment, 3D morphable model, weakly-supervised learning, Convolutional Neural Network (CNN)

摘要: 针对现在广泛使用的三维形变模型表达能力不够,导致重建出的三维人脸模型泛化性能不佳的问题,提出了一种在姿态、表情和光照未知的条件下的基于单张人脸图片的三维人脸重建和密集人脸对齐的新方法。首先,通过卷积神经网络对现有的三维形变模型进行改进,以提高三维人脸模型的表达能力;然后,基于人脸光滑性和图像相似性,在特征点和像素层面提出新的损失函数,并使用弱监督学习训练卷积神经网络模型;最后,通过训练出的网络模型进行三维人脸重建和密集人脸对齐。实验结果表明,对于三维人脸重建任务,所提模型在AFLW2000-3D上实现了2.25的归一化平均误差;对于密集人脸对齐任务,所提模型在AFLW2000-3D和AFLW-LFPA上分别实现了3.80和3.34的归一化平均误差。与原始使用三维形变模型的方法相比,所提模型在三维人脸重建和密集人脸对齐上的归一化平均误差分别降低了7.4%和7.8%。针对不同光照环境以及角度的人脸图片,该网络模型的重建准确,鲁棒性好,且具有较高的三维人脸重建和密集人脸对齐质量。

关键词: 三维人脸重建, 密集人脸对齐, 三维形变模型, 弱监督学习, 卷积神经网络

CLC Number: