Face liveness detection algorithm based on GhostNet and feature fusion

doi:10.11772/j.issn.1001-9081.2022071100

Abstract

Abstract:

The wide application of face recognition technology not only brings convenience to users， but also brings problems such as face spoofing and presentation attacks. Aiming at the frequent presentation attacks and print attacks， a face liveness detection algorithm based on GhostNet and feature fusion was proposed. Firstly， the feature extraction process of GhostNet model was divided into three different stages， namely， low-level feature， medium-level feature and high-level feature. Then， the feature map information of each stage was output respectively. Finally， the feature maps with different semantic information were sent into the feature fusion module for adaptive weighted fusion， so as to obtain more discriminative feature mapping. Experiments were conducted on public datasets NUAA and CelebA-Spoof. The results show that the accuracy of the proposed algorithm is 99.97% and 93.41% respectively， which is increased by 8.00 and 9.20 percentage points respectively compared with the algorithm of direct training of GhostNet model. Compared with Heterogeneous Kernel-Convolutional Neural Network （HK-CNN）， lightweight convolutional neural network FeatherNet， block based multi-stream network FaceBageNet and other algorithms， the proposed algorithm shows better performance on NUAA and CelebA-Spoof datasets. And， as GhostNet is a lightweight network model， the proposed algorithm only takes 3.6 ms on single image inference on CelebA-Spoof dataset.

Key words: face recognition, live detection, GhostNet, adaptive weighted fusion, CelebA-Spoof dataset

摘要：

人脸识别技术的广泛应用在为用户带来方便的同时，也带来了人脸欺骗和展示攻击等问题。针对经常出现的展示攻击和打印攻击问题，提出了一种基于GhostNet和特征融合的人脸活体检测算法。首先，将GhostNet模型的特征提取过程分为三种不同的阶段，即低等特征、中等特征和高等特征；然后，分别输出每个阶段的特征图信息；最后，将具有不同语义信息的特征图送入特征融合模块进行自适应加权融合，以获得更加具有辨别性的特征映射。在NUAA和CelebA-Spoof两个公开数据集上进行实验，实验结果表明所提算法的准确率分别为99.97%和93.41%，相较于GhostNet模型直接进行训练的算法分别提高了8.00和9.20个百分点。与异构内核的卷积神经网络（HK-CNN）、轻量级卷积神经网络FeatherNet、基于分块的多流网络FaceBagNet等算法相比，所提算法在NUAA和CelebA-Spoof数据集上表现出更好的性能；并且，由于GhostNet是一种轻量化的网络模型，所提算法在CelebA-Spoof数据集上对单张图像进行推理的时间仅需3.6 ms。

关键词: 人脸识别, 活体检测, GhostNet, 自适应加权融合, CelebA-Spoof数据集

CLC Number:

TP183

Chungang HAN, Yonghui LIU. Face liveness detection algorithm based on GhostNet and feature fusion[J]. Journal of Computer Applications, 2023, 43(8): 2588-2592.

韩春港, 刘永辉. 基于GhostNet和特征融合的人脸活体检测算法[J]. 《计算机应用》唯一官方网站, 2023, 43(8): 2588-2592.

Figures/Tables 11

Fig. 1 GhostNet network block used to output feature maps with the same number as normal convolutional layers

Fig. 2 Structures used in GhostNet

Fig. 3 Structure of GhostNet network

Fig. 4 Structure of GhostNet-AFF network

Fig. 5 Visualization of data augmentation

Fig. 6 Accuracy of GhostNet-AFF on NUAA dataset

Fig. 7 Accuracy of GhostNet-AFF on CelebA-Spoof dataset

Tab. 1 Comparison of test results of two models

模型	数据集	TPR（FPR=0.005）	Recall
GhostNet	NUAA	92.60	95.46
GhostNet-AFF	NUAA	99.73	99.93
GhostNet	CelebA-Spoof	85.20	90.50
GhostNet-AFF	CelebA-Spoof	92.14	93.44

Tab. 2 Accuracy comparison of different algorithms on NUAA dataset

算法	准确率	算法	准确率
ND-CNN^［14］	99.00	HK-CNN^［18］	99.82
DK^［15］	99.30	LBP+模糊特征+色彩纹理特征^［19］	99.27
ELM-LRF^［16］	99.67	GhostNet^［10］	91.97
ResNet50+depth+SE^［17］	95.91	本文算法	99.97

Tab. 3 Accuracy comparison of different algorithms on CelebA-Spoof dataset

算法	准确率	算法	准确率
FeatherNet^［20］	89.08	GhostNet^［10］	84.21
FaceBagNet^［21］	91.25	本文算法	93.41

Fig. 8 Recognition results of real and fake faces

References 21

1	YU B Y， LU J W， LI X， et al. Salience-aware face presentation attack detection via deep reinforcement learning［J］. IEEE Transactions on Information Forensics and Security， 2022， 17： 413-427. 10.1109/tifs.2021.3135748
2	LI J W， WANG Y H， TAN T N， et al. Live face detection based on the analysis of Fourier spectra［C］// Proceedings of the SPIE 5404， Biometric Technology for Human Identification 2004. Bellingham， WA： SPIE， 2004： No.541955. 10.1117/12.541955
3	BOULKENAFET Z， KOMULAINEN J， HADID A. Face spoofing detection using colour texture analysis［J］. IEEE Transactions on Information Forensics and Security， 2016， 11（8）： 1818-1830. 10.1109/tifs.2016.2555286
4	BOULKENAFET Z， KOMULAINEN J， FENG X Y， et al. Scale space texture analysis for face anti-spoofing［C］// Proceedings of the 2016 International Conference on Biometrics. Piscataway： IEEE， 2016： 1-6. 10.1109/icb.2016.7550078
5	RAGHAVENDRA R J， SANJEEV KUNTE R. Extended local ternary co-relation pattern： a novel feature descriptor for face anti-spoofing［J］. Journal of Information Security and Applications， 2020， 52： No.102482. 10.1016/j.jisa.2020.102482
6	YANG X， LUO W H， BAO L C， et al. Face anti-spoofing： model matters， so does data［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 3502-3511. 10.1109/cvpr.2019.00362
7	SUN W Y， SONG Y， CHEN C S， et al. Face spoofing detection based on local ternary label supervision in fully convolutional networks［J］. IEEE Transactions on Information Forensics and Security， 2020， 15： 3181-3196. 10.1109/tifs.2020.2985530
8	DEB D， JAIN A K. Look locally infer globally： a generalizable face anti-spoofing approach［J］. IEEE Transactions on Information Forensics and Security， 2021， 16： 1143-1157. 10.1109/tifs.2020.3029879
9	杨瑞杰，郑贵林. 基于InceptionV3和特征融合的人脸活体检测［J］. 计算机应用， 2022，42（7）：2037-2042.
	YANG R J， ZHENG G L. Face liveness detection based on InceptionV3 and feature fusion［J］. Journal of Computer Applications， 2022， 42（7）：2037-2042.
10	HAN K， WANG Y H， TIAN Q， et al. GhostNet： more features from cheap operations［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 1577-1586. 10.1109/cvpr42600.2020.00165
11	ZEILER M D， FERGUS R. Visualizing and understanding convolutional networks［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS 8689. Cham： Springer， 2014： 818-833.
12	TAN X Y， LI Y， LIU J， et al. Face liveness detection from a single image with sparse low rank bilinear discriminative model［C］// Proceedings of the 2010 European Conference on Computer Vision， LNCS 6316. Berlin： Springer， 2010： 504-517.
13	ZHANG Y H， YIN Z F， LI Y D， et al. CelebA-Spoof： large-scale face anti-spoofing dataset with rich annotations［C］// Proceedings of the 2020 European Conference on Computer Vision， LNCS 12357. Cham： Springer， 2020： 70-85.
14	ALOTAIBI A， MAHMOOD A. Deep face liveness detection based on nonlinear diffusion using convolution neural network［J］. Signal， Image and Video Processing， 2017， 11（4）： 713-720. 10.1007/s11760-016-1014-2
15	YU C Y， JIA Y D. Anisotropic diffusion-based kernel matrix model for face liveness detection［EB/OL］. （2017-07-10）［2022-05-22］.. 10.1016/j.imavis.2019.06.009
16	蔡祥云. 人脸识别中的活体检测及应用研究［D］. 兰州：兰州交通大学， 2020：34-48.
	CAI X Y. Research on spoofing detection and application in face recognition［D］. Lanzhou： Lanzhou Jiaotong University， 2020：34-48.
17	陈路燕. 人脸识别中的活体检测算法研究［D］. 北京：北京邮电大学， 2020：23-38.
	CHEN L Y. Research on face anti-spoofing algorithm in face recognition［D］. Beijing： Beijing University of Posts and Telecommunications， 2020：23-38.
18	卢鑫. 基于改进卷积神经网络的人脸活体检测研究［D］. 鞍山：辽宁科技大学， 2020：39-48. 10.1109/iwecai50956.2020.00028
	LU X. Research of face liveness detection based on improved convolutional neural network［D］. Anshan： University of Science and Technology Liaoning， 2020：39-48. 10.1109/iwecai50956.2020.00028
19	李新豆，高陈强，周风顺，等. 基于图像扩散速度模型和纹理信息的人脸活体检测［J］. 计算机科学， 2020， 47（2）：112-117. 10.11896/jsjkx.181202339
	LI X D， GAO C Q， ZHOU F S， et al. Face liveness detection based on image diffusion speed model and texture information［J］. Computer Science， 2020， 47（2）：112-117. 10.11896/jsjkx.181202339
20	ZHANG P， ZOU F H， WU Z W， et al. FeatherNets： convolutional neural networks as light as feather for face anti-spoofing［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vison and Pattern Recognition Workshops. Piscataway： IEEE， 2019：1574-1583. 10.1109/cvprw.2019.00199
21	SHEN T， HUANG Y Y， TONG Z J. FaceBagNet： bag-of-local-features model for multi-modal face anti-spoofing［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vison and Pattern Recognition Workshops. Piscataway： IEEE， 2019：1611-1616. 10.1109/cvprw.2019.00203

[1]	Kansong CHEN, Yuan ZHENG, Lijun XU, Zhouyu WANG, Zhe ZHANG, Fujuan YAO. ShuffaceNet： face recognition neural network based on ThetaMEX global pooling [J]. Journal of Computer Applications, 2023, 43(8): 2572-2580.
[2]	Juming HAO, Jingyu YANG, Shumei HAN, Yangping WANG. YOLOv4 highway pavement crack detection method using Ghost module and ECA [J]. Journal of Computer Applications, 2023, 43(4): 1284-1290.
[3]	Ruijie YANG, Guilin ZHENG. Face liveness detection based on InceptionV3 and feature fusion [J]. Journal of Computer Applications, 2022, 42(7): 2037-2042.
[4]	Huaiqing HE, Jianqing YAN, Kanghua HUI. Lightweight face recognition method based on deep residual network [J]. Journal of Computer Applications, 2022, 42(7): 2030-2036.
[5]	GAO Gong, YANG Hongyu, LIU Hong. 3D point cloud face recognition based on deep learning [J]. Journal of Computer Applications, 2021, 41(9): 2736-2740.
[6]	CHEN Zhengtao, HUANG Can, YANG Bo, ZHAO Li, LIAO Yong. Yak face recognition algorithm of parallel convolutional neural network based on transfer learning [J]. Journal of Computer Applications, 2021, 41(5): 1332-1336.
[7]	LI Hongxia, QIN Pinle, YAN Hanmei, ZENG Jianchao, BAO Qianyue, CHAI Rui. Face frontalization generative adversarial network algorithm based on face feature map symmetry [J]. Journal of Computer Applications, 2021, 41(3): 714-720.
[8]	Han DU, Xianzhong LONG, Yun LI. Graph learning regularized discriminative non-negative matrix factorization based face recognition [J]. Journal of Computer Applications, 2021, 41(12): 3455-3461.
[9]	FANG Shuya, LIU Shouyin. Non-perception class attendance method based on student body detection [J]. Journal of Computer Applications, 2020, 40(9): 2519-2524.
[10]	ZHAO Qing, YU Yuanhui. 3D face recognition based on hierarchical feature network [J]. Journal of Computer Applications, 2020, 40(9): 2514-2518.
[11]	WANG Jinkai, JIA Xu. Vehicle face recognition algorithm based on NMF with weighted and orthogonal constraints [J]. Journal of Computer Applications, 2020, 40(4): 1050-1055.
[12]	LI Dongbo, HUANG Lyuwen. Reweighted sparse principal component analysis algorithm and its application in face recognition [J]. Journal of Computer Applications, 2020, 40(3): 717-722.
[13]	GONG Rui, DING Sheng, ZHANG Chaohua, SU Hao. Lightweight and multi-pose face recognition method based on deep learning [J]. Journal of Computer Applications, 2020, 40(3): 704-709.
[14]	CHEN Fang, LIU Xiaorui, YANG Mingye. Face recognition security system based on liveness detection and authentication [J]. Journal of Computer Applications, 2020, 40(12): 3666-3672.
[15]	REN Zhenwen, WU Mingna. Correntropy self-weighted based joint regularized nearest points for images set classification algorithm [J]. Journal of Computer Applications, 2019, 39(9): 2547-2551.