Deep face verification under pose interference

doi:10.11772/j.issn.1001-9081.2021122214

Abstract

Abstract:

Face verification is widely used in various scenes in life， and the acquisition of ordinary RGB images is extremely dependent on illumination conditions. In order to solve the interference of illumination and head pose， a convolutional neural network based Siamese network L2-Siamese was proposed. Firstly， the paired depth images were taken as input. Then， after using two convolutional neural networks that share weights to extract facial features respectively， L2 norm was introduced to constrain the facial features with different poses on a hypersphere with a fixed radius. Finally， the fully connected layer was used to map the difference between the features to the probability value in （0，1） to determine whether the group of images belonged to the same object. In order to verify the effectiveness of L2-Siamese， a test was conducted on the public dataset Pandora. Experimental results show that L2-Siamese has good overall performance. After the dataset was grouped according to the size of head pose interference， the test results show that the prediction accuracy of L2-Siamese is 4 percentage points higher than that of the state-of-the-art algorithm fully-convolutional Siamese network under the maximum head pose interference， illustrating that the accuracy of prediction has been significantly improved.

Key words: face verification, depth image, deep learning, Siamese network, neural network

摘要：

人脸验证广泛应用于生活中各种场景，而普通RGB图像的获取依赖于光照条件。为解决光照和头部姿态对任务的干扰，提出了一个基于卷积神经网络的孪生网络L2-Siamese。首先，直接将成对的深度图作为输入；然后，用两个共享权重的卷积神经网络分别提取面部特征后，引入L2范数将不同姿态的人脸特征约束在一个半径固定的超球上；最后，通过全连接层将特征之间的差异映射为（0，1）区间的概率值来判断该组图像是否属于同一对象。为了验证L2-Siamese的有效性，在公共数据集Pandora上进行了测试。实验结果显示，L2-Siamese整体性能良好。将Pandora根据头部姿态干扰大小进行分组后的测试结果表明，在头部最大姿态干扰下，与当前最好的算法全卷积孪生网络相比，该网络预测准确率提高了4个百分点，有明显提升。

关键词: 人脸验证, 深度图, 深度学习, 孪生网络, 神经网络

CLC Number:

TP391.1

Qi WANG, Hang LEI, Xupeng WANG. Deep face verification under pose interference[J]. Journal of Computer Applications, 2023, 43(2): 595-600.

王奇, 雷航, 王旭鹏. 姿态干扰下的深度人脸验证[J]. 《计算机应用》唯一官方网站, 2023, 43(2): 595-600.

Figures/Tables 12

Fig.1 Structure of Siamese network

Fig.2 Structure of convolutional neural network

Fig. 3 Depth head images with different poses

Tab.1 Head pose and L2 norm

对象	第（1）组		第（2）组		第（3）组
对象	图像1	图像2	图像1	图像2	图像1	图像2
对象1	0.62	0.62	0.51	0.52	0.42	0.43
对象2	0.50	0.50	0.51	0.51	0.53	0.53
对象3	0.37	0.37	0.33	0.33	0.39	0.39

Fig. 4 Schematic diagram of head feature mapping

Fig. 5 Overall structure of L2-Siamese

Tab. 2 Accuracy corresponding to different threshold value

概率阈值	准确率/%	概率阈值	准确率/%
0.1	73.33	0.6	89.15
0.2	79.68	0.7	83.11
0.3	83.36	0.8	78.07
0.4	86.74	0.9	76.56
0.5	88.43

Tab.3 Accuracy corresponding to different scaling parameters α

$α$	准确率/%	$α$	准确率/%
0.2	87.62	10.0	89.98
0.4	88.77	12.0	89.91
0.6	89.26	14.0	89.89
0.8	89.58	16.0	89.68
1.0	89.74	18.0	89.63
2.0	89.70	20.0	89.25
4.0	89.72	24.0	88.08
6.0	89.66	28.0	88.12
8.0	89.81	不采用超球约束特征	89.15

Tab.3 Accuracy corresponding to different scaling parameters α

$α$	准确率/%	$α$	准确率/%
0.2	87.62	10.0	89.98
0.4	88.77	12.0	89.91
0.6	89.26	14.0	89.89
0.8	89.58	16.0	89.68
1.0	89.74	18.0	89.63
2.0	89.70	20.0	89.25
4.0	89.72	24.0	88.08
6.0	89.66	28.0	88.12
8.0	89.81	不采用超球约束特征	89.15

Fig. 6 Curves of loss value and accuracy

Tab. 4 Comparison of experimental results of different methods on Pandora dataset

方法	输入图像			模型		GPU：1080ti
方法	训练	测试	图像规格	模型参数/10⁶	准确率/%	速度/fps
JanusNet^［13］	RGB+Depth	Depth	$100 × 100$	4.8	81.4	202
Siamese^［3］	Depth	Depth	variable	1.8	85.3	604
L2⁃Siamese	Depth	Depth	$105 × 105$	56.0	89.9	148

Tab. 4 Comparison of experimental results of different methods on Pandora dataset

方法	输入图像			模型		GPU：1080ti
方法	训练	测试	图像规格	模型参数/10⁶	准确率/%	速度/fps
JanusNet^［13］	RGB+Depth	Depth	$100 × 100$	4.8	81.4	202
Siamese^［3］	Depth	Depth	variable	1.8	85.3	604
L2⁃Siamese	Depth	Depth	$105 × 105$	56.0	89.9	148

Fig. 7 Examples of dataset grouping

Tab.5 Accuracy comparison of experimental results of different methods after grouping Pandora dataset according to poses

训练集	测试集
	JanusNet^［13］				Siamese^［3］				L2-Siamese
	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$
$A 1$	0.80	0.66	0.62	0.69	0.86	0.75	0.71	0.77	0.90	0.83	0.78	0.82
$A 2$	0.83	0.79	0.77	0.80	0.87	0.84	0.81	0.85	0.90	0.88	0.87	0.89
$A 3$	0.50	0.50	0.50	0.50	0.75	0.70	0.67	0.71	0.82	0.78	0.73	0.75
${A 1, A 2}$	0.80	0.75	0.72	0.76	0.88	0.84	0.81	0.85	0.91	0.88	0.87	0.90

Tab.5 Accuracy comparison of experimental results of different methods after grouping Pandora dataset according to poses

训练集	测试集
	JanusNet^［13］				Siamese^［3］				L2-Siamese
	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$
$A 1$	0.80	0.66	0.62	0.69	0.86	0.75	0.71	0.77	0.90	0.83	0.78	0.82
$A 2$	0.83	0.79	0.77	0.80	0.87	0.84	0.81	0.85	0.90	0.88	0.87	0.89
$A 3$	0.50	0.50	0.50	0.50	0.75	0.70	0.67	0.71	0.82	0.78	0.73	0.75
${A 1, A 2}$	0.80	0.75	0.72	0.76	0.88	0.84	0.81	0.85	0.91	0.88	0.87	0.90

References 18

1	WANG M， DENG W H. Deep face recognition： a survey［J］. Neurocomputing， 2021， 429： 215-244. 10.1016/j.neucom.2020.10.081
2	BORGHI G， FABBRI M， VEZZANI R， et al. Face-from-depth for head pose estimation on depth images［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（3）： 596-609. 10.1109/tpami.2018.2885472
3	BORGHI G， PINI S， VEZZANI R， et al. Driver face verification with depth maps［J］. Sensors， 2019， 19（15）： No.3361. 10.3390/s19153361
4	HUANG G B， RAMESH M， BERG T， et al. Labeled faces in the wild： a database for studying face recognition in unconstrained environments［EB/OL］. ［2021-10-13］.. 10.1117/12.2080393
5	SCHROFF F， KALENICHENKO D， PHILBIN J. FaceNet： a unified embedding for face recognition and clustering［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 815-823. 10.1109/cvpr.2015.7298682
6	YANG J L， REN P R， ZHANG D Q， et al. Neural aggregation network for video face recognition［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5216-5225. 10.1109/cvpr.2017.554
7	CHEN J C， PATEL V M， CHELLAPPA R. Unconstrained face verification using deep CNN features［C］// Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2016： 1-9. 10.1109/wacv.2016.7477557
8	HUANG C， LI Y Y， LOY C C， et al. Deep imbalanced learning for face recognition and attribute prediction［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（11）：2781-2794. 10.1109/tpami.2019.2914680
9	JIAO Q F， LI R， CAO W M， et al. DDAT： dual domain adaptive translation for low-resolution face verification in the wild［J］. Pattern Recognition， 2021， 120： No.108107. 10.1016/j.patcog.2021.108107
10	BALLOTTA D， BORGHI G， VEZZANI R， et al. Fully convolutional network for head detection with depth images［C］// Proceedings of the 24th International Conference on Pattern Recognition. Piscataway： IEEE， 2018： 752-757. 10.1109/icpr.2018.8545332
11	BALLOTTA D， BORGHI G， VEZZANI R， et al. Head detection with depth images in the wild［C］// Proceedings of the 13th International Conference on Computer Vision， Imaging and Computer Graphics Theory and Applications - Volume 4 VISAPP. Setúbal： SciTePress， 2018： 56-63. 10.5220/0006541000560063
12	KIM D， HERNANDEZ M， CHOI J， et al. Deep 3D face identification［C］// Proceedings of the 2017 IEEE International Joint Conference on Biometrics. Piscataway： IEEE， 2017： 133-142. 10.1109/btas.2017.8272691
13	BORGHI G， PINI S， GRAZIOLI F， et al. Face verification from depth using privileged information［C］// Proceedings of the 2018 British Machine Vision Conference. Durham： BMVA Press， 2018： No.192.
14	BROMLEY J， GUYON I， LeCUN Y， et al. Signature verification using a “Siamese” time delay neural network［C］// Proceedings of the 6th International Conference on Neural Information Processing Systems. San Francisco： Morgan Kaufmann Publishers Inc.， 1993： 737-744. 10.1142/9789812797926_0003
15	KOCH G， ZEMEL R， SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition［EB/OL］. ［2021-11-07］..
16	RANJAN R， CASTILLO C D， CHELLAPPA R. L₂-constrained softmax loss for discriminative face verification［EB/OL］. 2017-06-07］［2021-11-07］..
17	BORGHI G， VENTURELLI M， VEZZANI R， et al. POSEidon： face-from-depth for driver pose estimation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5494-5503. 10.1109/cvpr.2017.583
18	VENTURELLI M， BORGHI G， VEZZANI R， et al. From depth data to head pose estimation： a Siamese approach［EB/OL］.（2017-03-10）［2021-19-20］./ . 10.5220/0006104501940201

[1]	Xingyao YANG, Yu CHEN, Jiong YU, Zulian ZHANG, Jiaying CHEN, Dongxiao WANG. Recommendation model combining self-features and contrastive learning [J]. Journal of Computer Applications, 2024, 44(9): 2704-2710.
[2]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[3]	Yu DU, Yan ZHU. Constructing pre-trained dynamic graph neural network to predict disappearance of academic cooperation behavior [J]. Journal of Computer Applications, 2024, 44(9): 2726-2731.
[4]	Na WANG, Lin JIANG, Yuancheng LI, Yun ZHU. Optimization of tensor virtual machine operator fusion based on graph rewriting and fusion exploration [J]. Journal of Computer Applications, 2024, 44(9): 2802-2809.
[5]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[6]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[7]	Tingjie TANG, Jiajin HUANG, Jin QIN. Session-based recommendation with graph auxiliary learning [J]. Journal of Computer Applications, 2024, 44(9): 2711-2718.
[8]	Rui ZHANG, Pengyun ZHANG, Meirong GAO. Self-optimized dual-modal multi-channel non-deep vestibular schwannoma recognition model [J]. Journal of Computer Applications, 2024, 44(9): 2975-2982.
[9]	Jinjin LI, Guoming SANG, Yijia ZHANG. Multi-domain fake news detection model enhanced by APK-CNN and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2674-2682.
[10]	Guanglei YAO, Juxia XIONG, Guowu YANG. Flower pollination algorithm based on neural network optimization [J]. Journal of Computer Applications, 2024, 44(9): 2829-2837.
[11]	Ying HUANG, Jiayu YANG, Jiahao JIN, Bangrui WAN. Siamese mixed information fusion algorithm for RGBT tracking [J]. Journal of Computer Applications, 2024, 44(9): 2878-2885.
[12]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[13]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[14]	Hang YANG, Wanggen LI, Gensheng ZHANG, Zhige WANG, Xin KAI. Multi-layer information interactive fusion algorithm based on graph neural network for session-based recommendation [J]. Journal of Computer Applications, 2024, 44(9): 2719-2725.
[15]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.