Interactive liveness detection combining with head pose and facial expression

doi:10.11772/j.issn.1001-9081.2019112059

Abstract

Abstract: In order to prevent photo and video attacks in the face recognition system, an interactive liveness detection algorithm was proposed which combines the head pose and facial expression. Firstly, the number of convolution kernels, network layers, and regularization of VGGNet were adjusted and optimized, and a multi-layer convolutional head pose estimation network was constructed. Secondly, the methods such as global average pooling, local response normalization and convolutional replacement pooling were introduced to improve VGGNet and build an expression recognition network. Finally, the above two networks were fused to realize an interactive liveness detection system, which sends random instructions to users to complete liveness detection in real time. The experimental results show that the proposed head pose estimation network and expression recognition network achieve 99.87% and 99.60% accuracy on CAS-PEAL-R1 dataset and CK+ dataset respectively, and the liveness detection system has the comprehensive accuracy reached 96.70%, the running speed reaches 20-28 frames per second, which make the generalization ability of the system outstanding in the practical application.

Key words: head pose estimation, expression recognition, liveness detection, random instruction, VGGNet

摘要： 为了阻挡人脸识别系统中的照片及视频攻击，提出了一种将头部姿态和面部表情融合的互动式活体检测算法。首先，对VGGNet的卷积核数目、网络层数、正则化等进行了调整优化，构建了一个多层卷积的头部姿态估计网络；其次，引入全局平均池化、局部响应归一化和卷积替代池化等方法对VGGNet进行改进，构建了一个表情识别网络；最后，融合上述两个网络实现了互动式活体检测系统，对用户发出随机指令实时完成活体检测。实验结果表明，所提出的头部姿态估计网络和表情识别网络分别在CAS-PEAL-R1数据集和CK+数据集上取得了99.87%和99.60%的准确率，而活体检测系统的综合准确率达到了96.70%，运行速度达到了每秒20~28帧，在实际应用中泛化能力突出。

关键词: 头部姿态估计, 表情识别, 活体检测, 随机指令, VGGNet

CLC Number:

TP183

HUANG Jun, ZHANG Nana, ZHANG Hui. Interactive liveness detection combining with head pose and facial expression[J]. Journal of Computer Applications, 2020, 40(7): 2089-2095.

黄俊, 张娜娜, 章惠. 融合头部姿态和面部表情的互动式活体检测[J]. 计算机应用, 2020, 40(7): 2089-2095.

References

[1] ZHAO W,CHELLAPPA R,PHILLIPS P J,et al. Face recognition:a literature survey[J]. ACM Computing Surveys,2003,35(4):430-458.
[2] MARCEL S,NIXON M S,LI S Z. Handbook of Biometric AntiSpoofing:Trusted Biometrics under Spoofing Attacks[M]. London:Springer,2014:65-82.
[3] MANJANI I,TARIYAL S,VATSA M,et al. Detecting silicone mask-based presentation attack via deep dictionary learning[J]. IEEE Transactions on Information Forensics and Security,2017,12(7):1713-1723.
[4] 邓雄, 王洪春, 赵立军, 等. 人脸识别活体检测研究方法综述[J/OL]. 计算机应用研究.[2019-09-23]. http://kns.cnki.net/kcms/detail/51.1196.TP.20190828.1154.023.html. (DENG X, WANG H C,ZHAO L J,et al. Survey on face anti-spoofing in face recognition[J/OL]. Application Research of Computers.[2019-09-23]. http://kns.cnki.net/kcms/detail/51.1196.TP.20190828.1154.023.html.)
[5] SINGH A K,JOSHI P,NANDI G C. Face recognition with liveness detection using eye and mouth movement[C]//Proceedings of the 2014 International Conference on Signal Propagation and Computer Technology. Piscataway:IEEE,2014:592-597.
[6] 田野, 项世军. 基于LBP和多层DCT的人脸活体检测算法[J]. 计算机研究与发展,2018,55(3):643-650. (TIAN Y,XIANG S J. LBP and multilayer DCT based anti-spoofing countermeasure in face liveness detection[J]. Journal of Computer Research and Development,2018,55(3):643-650.)
[7] 徐渊, 许晓亮, 李才年, 等. 结合SVM分类器与HOG特征提取的行人检测[J]. 计算机工程,2016,42(1):56-60,65.(XU Y,XU X L,LI C N,et al. Pedestrian detection combining with SVM classifier and HOG feature extraction[J]. Computer Engineering, 2016,42(1):56-60,65.)
[8] 胡斐, 文畅, 谢凯, 等. 基于微调策略的多线索融合人脸活体检测[J]. 计算机工程,2019,45(5):256-260.(HU F,WEN C, XIE K,et al. Multi-cue fusion face detection based on fine-tuning strategy[J]. Computer Engineering,2019,45(5):256-260.)
[9] 冯岩, 薛瑞. 剪切波理论及其应用研究进展[J]. 信阳师范学院学报(自然科学版),2014(3):463-468. (FENG Y,XUE R. Advances in theory and application of shearlets[J]. Journal of Xinyang Normal University(Natural Science Edition),2014(3):463-468.)
[10] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2019-04-10]. https://arxiv.org/pdf/1409.1556.pdf.
[11] KRIZHEVSKY A,SUTSKEVER I,HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook,NY:Curran Associates Inc., 2012:1097-1105.
[12] 李勇, 林小竹, 蒋梦莹. 基于跨连接LeNet-5网络的面部表情识别[J]. 自动化学报,2018,44(1):176-182. (LI Y,LIN X Z, JIANG M Y. Facial expression recognition based on cross-connected LeNet-5 network[J]. Acta Automatica Sinica,2018,44(1):176-182.)
[13] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. The Journal of Machine Learning Research,2014,15(1):1929-1958.
[14] SPRINGENBERG J T,DOSOVITSKIY A,BROX T,et al. Striving for simplicity:the all convolutional net[EB/OL].[2018-12-21]. https://arxiv.org/pdf/1412.6806v3.pdf.
[15] LIN M,CHEN Q,YAN S. Network in network[EB/OL].[2019-03-04]. https://arxiv.org/pdf/1312.4400.pdf.
[16] ZHANG K,ZHANG Z,LI Z,et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters,2016,23(10):1499-1503.
[17] KINGMA D P,BA J L. Adam:a method for stochastic optimization[EB/OL].[2018-12-22]. https://arxiv.org/pdf/1412.6980v8.pdf.
[18] 梁令羽, 张天天, 何为. 多尺度卷积神经网络的头部姿态估计[J]. 激光与光电子学进展,2019,56(13):No. 131003. (LIANG L Y,ZHANG T T,HE W. Head pose estimation based on multi-scale convolutional neural network[J]. Laser and Optoelectronics Progress,2019,56(13):No. 131003.)
[19] 陆清正, 周宇, 叶庆卫, 等. 基于局部二值特征和BP神经网络的头部姿态估计[J]. 传感器与微系统,2019,38(2):45-48, 58.(LU Q Z,ZHOU Y,YE Q W,et al. Head pose estimation based on local binary features and BP neural network[J]. Transducer and Microsystem Technologies,2019,38(2):45-48,58.)
[20] KAFAI M,BHANU B,AN L. Cluster-classification Bayesian networks for head pose estimation[C]//Proceedings of the 21st International Conference on Pattern Recognition. Piscataway:IEEE, 2012:2869-2872.
[21] HUANG C,DING X,FANG C. Head pose estimation based on random forests for multiclass classification[C]//Proceedings of the 20th International Conference on Pattern Recognition. Piscataway:IEEE,2010:934-937.
[22] 徐琳琳, 张树美, 赵俊莉. 构建并行卷积神经网络的表情识别算法[J]. 中国图象图形学报,2019,24(2):227-236. (XU L L, ZHANG S M,ZHAO J L. Expression recognition algorithm for parallel convolutional neural networks[J]. Journal of Image and Graphics,2019,24(2):227-236.)
[23] 姚丽莎, 徐国明, 赵凤. 基于CNN局部特征融合的人脸表情识别[J/OL]. 激光与光电子学进展.[2020-01-04]. http://kns.cnki.net/kcms/detail/31.1690.TN.20190508.1150.016.html. (YAO L S,XU G M,ZHAO F. Facial expression recognition based on CNN local feature fusion[J/OL]. Laser and Optoelectronics Progress.[2020-01-04]. http://kns.cnki.net/kcms/detail/31.1690.TN.20190508.1150.016.html.)
[24] SADEGHI H,RAIE A A,MOHAMMADI M R. Facial expression recognition using geometric normalization and appearance representation[C]//Proceedings of the 8th Iranian Conference on Machine Vision and Image Processing. Piscataway:IEEE,2013:856-860.
[25] KIM G,EUM S,SUHR J K,et al. Face liveness detection based on texture and frequency analyses[C]//Proceedings of the 5th IAPR International Conference on Biometrics. Piscataway:IEEE, 2012:67-72.
[26] SMIATACZ M. Liveness measurements using optical flow for biometric person authentication[J]. Metrology and Measurement Systems,2012,19(2):257-268.
[27] NG E S,CHIA A Y S. Face verification using temporal affective cues[C]//Proceedings of the 21st International Conference on Pattern Recognition. Piscataway:IEEE,2012:1249-1252.