Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (7): 2089-2095.DOI: 10.11772/j.issn.1001-9081.2019112059

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Interactive liveness detection combining with head pose and facial expression

HUANG Jun1, ZHANG Nana2, ZHANG Hui1   

  1. 1. College of Information Technology, Shanghai Ocean University, Shanghai 201306, China;
    2. College of Information Technology, Shanghai Jian Qiao University, Shanghai 201306, China
  • Received:2019-12-05 Revised:2020-01-20 Online:2020-07-10 Published:2020-05-20
  • Supported by:
    This work is partially supported by the Shanghai Municipal Education Commission s "Morning Plan" (AASH1702).


黄俊1, 张娜娜2, 章惠1   

  1. 1. 上海海洋大学 信息学院, 上海 201306;
    2. 上海建桥学院 信息技术学院, 上海 201306
  • 通讯作者: 张娜娜
  • 作者简介:黄俊(1996-),男,浙江温州人,硕士研究生,主要研究方向:图像处理、计算机视觉;张娜娜(1979-),女,山东莱阳人,副教授,硕士,主要研究方向:图像处理、计算机视觉;章惠(1996-),女,浙江温州人,硕士研究生,主要研究方向:图像处理、计算机视觉。
  • 基金资助:

Abstract: In order to prevent photo and video attacks in the face recognition system, an interactive liveness detection algorithm was proposed which combines the head pose and facial expression. Firstly, the number of convolution kernels, network layers, and regularization of VGGNet were adjusted and optimized, and a multi-layer convolutional head pose estimation network was constructed. Secondly, the methods such as global average pooling, local response normalization and convolutional replacement pooling were introduced to improve VGGNet and build an expression recognition network. Finally, the above two networks were fused to realize an interactive liveness detection system, which sends random instructions to users to complete liveness detection in real time. The experimental results show that the proposed head pose estimation network and expression recognition network achieve 99.87% and 99.60% accuracy on CAS-PEAL-R1 dataset and CK+ dataset respectively, and the liveness detection system has the comprehensive accuracy reached 96.70%, the running speed reaches 20-28 frames per second, which make the generalization ability of the system outstanding in the practical application.

Key words: head pose estimation, expression recognition, liveness detection, random instruction, VGGNet

摘要: 为了阻挡人脸识别系统中的照片及视频攻击,提出了一种将头部姿态和面部表情融合的互动式活体检测算法。首先,对VGGNet的卷积核数目、网络层数、正则化等进行了调整优化,构建了一个多层卷积的头部姿态估计网络;其次,引入全局平均池化、局部响应归一化和卷积替代池化等方法对VGGNet进行改进,构建了一个表情识别网络;最后,融合上述两个网络实现了互动式活体检测系统,对用户发出随机指令实时完成活体检测。实验结果表明,所提出的头部姿态估计网络和表情识别网络分别在CAS-PEAL-R1数据集和CK+数据集上取得了99.87%和99.60%的准确率,而活体检测系统的综合准确率达到了96.70%,运行速度达到了每秒20~28帧,在实际应用中泛化能力突出。

关键词: 头部姿态估计, 表情识别, 活体检测, 随机指令, VGGNet

CLC Number: