Face liveness detection algorithm based on deep learning and feature fusion

doi:10.11772/j.issn.1001-9081.2019091595

Abstract

Abstract: Aiming at the problem that the existing liveness detection algorithms based on deep learning are mostly based on large convolutional neural network,a liveness detection algorithm based on lightweight network MobileNetV2 and feature fusion was proposed. Firstly,the improved MobileNetV2 was used as the basic network to extract features from RGB,HSV and LBP images respectively. Then,the obtained feature maps were stacked together to perform the feature layer fusion. Finally,the features were extracted from the merged feature maps,and the Softmax layer was used to make the judgment whether the face was real or fake. Simulation results show that the Equal Error Rate(EER)of the proposed algorithm on NUAA dataset was 0. 02%,the Average Classification Error Rate(ACER)on Siw dataset was 0. 75%,and the time to test single image costed 6 ms. Experimental results verify that:the fusion of different information can obtain a lower error rate, and the improved lightweight network guarantees the efficiency of the algorithm and meets the real-time requirement.

Key words: face liveness detection, feature fusion, MobileNetV2, lightweight network

摘要： 针对目前基于深度学习的活体检测算法大都基于大型卷积神经网络的问题，提出一种基于轻量级网络MobileNetV2和特征融合的活体检测算法。首先，以改进的MobileNetV2为基础网络分别从RGB、HSV、LBP图中提取特征；然后，将得到的特征图堆叠在一起以进行特征层的融合；最后，从融合后的特征图中继续提取特征，并利用Softmax层作出真假人脸的判断。仿真结果显示，所提算法在NUAA数据集上的等错误率（EER）为0.02%，在Siw数据集上的ACER（Average Classification Error Rate）为0.75%，而且测试单张图像仅用时6 ms。实验结果表明：融合不同的信息可以获得更低的错误率，改进的轻量化网络保证了算法的高效性并满足实时性需求。

关键词: 人脸活体检测, 特征融合, MobileNetV2, 轻量化网络

CLC Number:

TP37

DENG Xiong, WANG Hongchun. Face liveness detection algorithm based on deep learning and feature fusion[J]. Journal of Computer Applications, 2020, 40(4): 1009-1015.

邓雄, 王洪春. 基于深度学习和特征融合的人脸活体检测算法[J]. 计算机应用, 2020, 40(4): 1009-1015.

References

[1] LU J,LIONG V E,WANG G,et al. Joint feature learning for face recognition[J]. IEEE Transactions on Information Forensics and Security,2015,10(7):1371-1383.
[2] 邓雄, 王洪春, 赵立军, 等. 人脸识别活体检测研究方法综述[J/OL]. 计算机应用研究,2020,37(9).[2019-09-10]. http://www.arocmag.com/article/02-2020-09-003.html. (DENG X,(WANG H C,ZHAO L J,et al. Survey on face anti-spoofing in face recognition[J/OL]. Application Research of Computers,2020,37(9).[2019-09-10]. http://www.arocmag.com/article/02-2020-09-003.html.)
[3] DE FREITAS PEREIRA T,ANJOS A,DE MARTINO J M,et al. LBP-TOP based countermeasure against face spoofing attacks[C]//Proceedings of the 2012 Asian Conference on Computer Vision, LNCS 7728. Berlin:Springer,2012:121-132.
[4] BOULKENAFET Z, KOMULAINEN J, HADID A. Face anti-spoofing based on color texture analysis[C]//Proceedings of the 2015 IEEE International Conference on Image Processing. Piscataway:IEEE,2015:2636-2640.
[5] BOULKENAFET Z,KOMULAINEN J,HADID A. Face spoofing detection using color texture analysis[J]. IEEE Transactions on Information Forensics and Security,2016,11(8):1818-1830.
[6] WANG T,YANG J,LEI Z,et al. Face liveness detection using 3D structure recovered from a single camera[C]//Proceedings of the 2013 International Conference on Biometrics. Piscataway:IEEE, 2013:1-6.
[7] WANG Y,NIAN F D,LI T,et al. Robust face anti-spoofing with depth information[J]. Journal of Visual Communication and Image Representation,2017,49:332-337.
[8] SINGH A K,JOSHI P,NANDI G C. Face recognition with liveness detection using eye and mouth movement[C]//Proceedings of the 2014 International Conference on Signal Propagation and Computer Technology. Piscataway:IEEE,2014:592-597.
[9] BHATTACHARJEE S,MOHAMMADI A,MARCEL S. Spoofing deep face recognition with custom silicone masks[C]//Proceedings of the IEEE 9th International Conference on Biometrics:Theory, Applications and Systems. Piscataway:IEEE,2018:1-7.
[10] GALBALLY J,MARCEL S,FIERREZ J. Image quality assessment for fake biometric detection:application to iris,fingerprint, and face recognition[J]. IEEE Transactions on Image Processing, 2014,23(2):710-724.
[11] KOMULAINEN J,HADID A,PIETIKÄINEN M. Context based face anti-spoofing[C]//Proceedings of the IEEE 6th International Conference on Biometrics:Theory,Applications and Systems. Piscataway:IEEE,2013:1-8.
[12] YANG J,LEI Z,LI S Z. Learn convolutional neural network for face anti-spoofing[EB/OL].[2019-05-22]. https://arxiv.org/pdf/1408.5601.pdf.
[13] LUCENA O,JUNIOR A,MOIA V,et al. Transfer learning using convolutional neural networks for face anti-spoofing[C]//Proceedings of the 2017 International Conference Image Analysis and Recognition,LNCS 10317. Cham:Springer,2017:27-34.
[14] MANJANI I,TARIYAL S,VATSA M,et al. Detecting silicone mask based presentation attack via deep dictionary learning[J]. IEEE Transactions on Information Forensics and Security,2017, 12(7):1713-1723.
[15] GAN J,LI S,ZHAI Y,et al. 3D convolutional neural network based on face anti-spoofing[C]//Proceedings of the 2nd International Conference on Multimedia and Image Processing. Piscataway:IEEE,2017:1-5.
[16] TRONCI R,MUNTONI D,FADDA G,et al. Fusion of multiple clues for photo-attack detection in face recognition systems[C]//Proceedings of the 2011 IEEE International Joint Conference on Biometrics. Piscataway:IEEE,2011:1-6.
[17] KOMULAINEN J,HADID A,PIETIKÄINEN M,et al. Complementary countermeasures for detecting scenic face spoofing attacks[C]//Proceedings of the 2013 IEEE International Conference on Biometrics. Piscataway:IEEE,2013:1-7.
[18] TANG Y,WANG X,JIA X,et al. Fusing multiple deep features for face anti-spoofing[C]//Proceedings of the 2018 Chinese Conference on Biometric Recognition,LNCS 10996. Cham:Springer, 2018:321-330.
[19] 刘渭滨, 邹智元, 邢薇薇. 模式分类中的特征融合方法[J]. 北京邮电大学学报,2017,40(4):5-12. (LIU W B,ZOU Z Y, XING W W. Feature fusion methods in pattern classification[J]. Journal of Beijing University of Posts and Telecommunications, 2017,40(4):1-8.)
[20] FEICHTENHOFER C,PINZ A,ZISSERMAN A. Convolutional two-stream network fusion for video action recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:1933-1941.
[21] 彭明阳, 王建华, 闻祥鑫, 等. 结合HSV空间的水面图像特征水岸线检测[J]. 中国图象图形学报,2018,23(4):526-533. (PENG M Y,WANG J H,WEN X X,et al. Shoreline detection method by combining HSV spatial water image feature[J]. Journal of Image and Graphics,2018,23(4):526-533.)
[22] OJALA T,PIETIKÄINEN M,MAENPAA T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2002,24(7):971-987.
[23] LIU L,FIEGUTH P,GUO Y,et al. Local binary features for texture classification:taxonomy and experimental study[J]. Pattern Recognition,2017,62:135-160.
[24] SANDLER M,HOWARD A,ZHU M,et al. MobileNetV2:inverted residuals and linear bottlenecks[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:4510-4520.
[25] CHOLLET F. Xception:deep learning with depthwise separable convolutions[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:1800-1807.
[26] HOWARD A G,ZHU M,CHEN B,et al. MobileNets:efficient convolutional neural networks for mobile vision applications[EB/OL].[2019-06-03]. https://arxiv.org/pdf/1704.04861.pdf.
[27] HE K,ZHANG X,REN S,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2016:770-778.
[28] TAN X,LI Y,LIU J,et al. Face liveness detection from a single image with sparse low rank bilinear discriminative model[C]//Proceedings of the 2010 European Conference on Computer Vision, LNCS 6316. Berlin:Springer,2010:504-517.
[29] LIU Y,JOURABLOO A,LIU X. Learning deep models for face anti-spoofing:binary or auxiliary supervision[C]//Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2018:389-398.
[30] Information technology-biometric presentation attack detection, part 1:framework:ISO/IEC 30107-1:2016[S]. Geneva:International Organization for Standardization,2016-01-15.
[31] MÄÄTTÄ J,HADID A,PIETIKÄINEN M. Face spoofing detection from single images using micro-texture analysis[C]//Proceedings of the 2011 International Joint Conference on Biometrics. Piscataway:IEEE,2011:1-7.
[32] YANG J,LEI Z,LIAO S,et al. Face liveness detection with component dependent descriptor[C]//Proceedings of the 2013 International Conference on Biometrics. Piscataway:IEEE,2013:1-6.
[33] DE SOUZA G B,DA SILVA SANTOS D F,PIRES R G,et al. Deep texture features for robust face spoofing detection[J]. IEEE Transactions on Circuits and Systems II:Express Briefs,2017,64(12):1397-1401.
[34] ZHANG S,WANG X,LIU A,et al. CASIA-SURF-a dataset and benchmark for large-scale multi-modal face anti-spoofing[EB/OL].[2019-07-12]. https://arxiv.org/pdf/1812.00408.pdf.
[35] WANG Z,ZHAO C,QIN Y,et al. Exploiting temporal and depth information for multi-frame face anti-spoofing[EB/OL].[2019-02-12]. https://arxiv.org/pdf/1811.05118.pdf.