Deep face verification under pose interference

doi:10.11772/j.issn.1001-9081.2021122214

Abstract

Abstract:

Face verification is widely used in various scenes in life， and the acquisition of ordinary RGB images is extremely dependent on illumination conditions. In order to solve the interference of illumination and head pose， a convolutional neural network based Siamese network L2-Siamese was proposed. Firstly， the paired depth images were taken as input. Then， after using two convolutional neural networks that share weights to extract facial features respectively， L2 norm was introduced to constrain the facial features with different poses on a hypersphere with a fixed radius. Finally， the fully connected layer was used to map the difference between the features to the probability value in （0，1） to determine whether the group of images belonged to the same object. In order to verify the effectiveness of L2-Siamese， a test was conducted on the public dataset Pandora. Experimental results show that L2-Siamese has good overall performance. After the dataset was grouped according to the size of head pose interference， the test results show that the prediction accuracy of L2-Siamese is 4 percentage points higher than that of the state-of-the-art algorithm fully-convolutional Siamese network under the maximum head pose interference， illustrating that the accuracy of prediction has been significantly improved.

Key words: face verification, depth image, deep learning, Siamese network, neural network

摘要：

人脸验证广泛应用于生活中各种场景，而普通RGB图像的获取依赖于光照条件。为解决光照和头部姿态对任务的干扰，提出了一个基于卷积神经网络的孪生网络L2-Siamese。首先，直接将成对的深度图作为输入；然后，用两个共享权重的卷积神经网络分别提取面部特征后，引入L2范数将不同姿态的人脸特征约束在一个半径固定的超球上；最后，通过全连接层将特征之间的差异映射为（0，1）区间的概率值来判断该组图像是否属于同一对象。为了验证L2-Siamese的有效性，在公共数据集Pandora上进行了测试。实验结果显示，L2-Siamese整体性能良好。将Pandora根据头部姿态干扰大小进行分组后的测试结果表明，在头部最大姿态干扰下，与当前最好的算法全卷积孪生网络相比，该网络预测准确率提高了4个百分点，有明显提升。

关键词: 人脸验证, 深度图, 深度学习, 孪生网络, 神经网络

CLC Number:

TP391.1

Qi WANG, Hang LEI, Xupeng WANG. Deep face verification under pose interference[J]. Journal of Computer Applications, 2023, 43(2): 595-600.

王奇, 雷航, 王旭鹏. 姿态干扰下的深度人脸验证[J]. 《计算机应用》唯一官方网站, 2023, 43(2): 595-600.

Figures/Tables 12

Fig.1 Structure of Siamese network

Fig.2 Structure of convolutional neural network

Fig. 3 Depth head images with different poses

Tab.1 Head pose and L2 norm

对象	第（1）组		第（2）组		第（3）组
对象	图像1	图像2	图像1	图像2	图像1	图像2
对象1	0.62	0.62	0.51	0.52	0.42	0.43
对象2	0.50	0.50	0.51	0.51	0.53	0.53
对象3	0.37	0.37	0.33	0.33	0.39	0.39

Fig. 4 Schematic diagram of head feature mapping

Fig. 5 Overall structure of L2-Siamese

Tab. 2 Accuracy corresponding to different threshold value

概率阈值	准确率/%	概率阈值	准确率/%
0.1	73.33	0.6	89.15
0.2	79.68	0.7	83.11
0.3	83.36	0.8	78.07
0.4	86.74	0.9	76.56
0.5	88.43

Tab.3 Accuracy corresponding to different scaling parameters α

$α$	准确率/%	$α$	准确率/%
0.2	87.62	10.0	89.98
0.4	88.77	12.0	89.91
0.6	89.26	14.0	89.89
0.8	89.58	16.0	89.68
1.0	89.74	18.0	89.63
2.0	89.70	20.0	89.25
4.0	89.72	24.0	88.08
6.0	89.66	28.0	88.12
8.0	89.81	不采用超球约束特征	89.15

Tab.3 Accuracy corresponding to different scaling parameters α

$α$	准确率/%	$α$	准确率/%
0.2	87.62	10.0	89.98
0.4	88.77	12.0	89.91
0.6	89.26	14.0	89.89
0.8	89.58	16.0	89.68
1.0	89.74	18.0	89.63
2.0	89.70	20.0	89.25
4.0	89.72	24.0	88.08
6.0	89.66	28.0	88.12
8.0	89.81	不采用超球约束特征	89.15

Fig. 6 Curves of loss value and accuracy

Tab. 4 Comparison of experimental results of different methods on Pandora dataset

方法	输入图像			模型		GPU：1080ti
方法	训练	测试	图像规格	模型参数/10⁶	准确率/%	速度/fps
JanusNet^［13］	RGB+Depth	Depth	$100 × 100$	4.8	81.4	202
Siamese^［3］	Depth	Depth	variable	1.8	85.3	604
L2⁃Siamese	Depth	Depth	$105 × 105$	56.0	89.9	148

Tab. 4 Comparison of experimental results of different methods on Pandora dataset

方法	输入图像			模型		GPU：1080ti
方法	训练	测试	图像规格	模型参数/10⁶	准确率/%	速度/fps
JanusNet^［13］	RGB+Depth	Depth	$100 × 100$	4.8	81.4	202
Siamese^［3］	Depth	Depth	variable	1.8	85.3	604
L2⁃Siamese	Depth	Depth	$105 × 105$	56.0	89.9	148

Fig. 7 Examples of dataset grouping

Tab.5 Accuracy comparison of experimental results of different methods after grouping Pandora dataset according to poses

训练集	测试集
	JanusNet^［13］				Siamese^［3］				L2-Siamese
	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$
$A 1$	0.80	0.66	0.62	0.69	0.86	0.75	0.71	0.77	0.90	0.83	0.78	0.82
$A 2$	0.83	0.79	0.77	0.80	0.87	0.84	0.81	0.85	0.90	0.88	0.87	0.89
$A 3$	0.50	0.50	0.50	0.50	0.75	0.70	0.67	0.71	0.82	0.78	0.73	0.75
${A 1, A 2}$	0.80	0.75	0.72	0.76	0.88	0.84	0.81	0.85	0.91	0.88	0.87	0.90

Tab.5 Accuracy comparison of experimental results of different methods after grouping Pandora dataset according to poses

训练集	测试集
	JanusNet^［13］				Siamese^［3］				L2-Siamese
	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$	$A 1$	$A 2$	$A 3$	${A 1, A 2}$
$A 1$	0.80	0.66	0.62	0.69	0.86	0.75	0.71	0.77	0.90	0.83	0.78	0.82
$A 2$	0.83	0.79	0.77	0.80	0.87	0.84	0.81	0.85	0.90	0.88	0.87	0.89
$A 3$	0.50	0.50	0.50	0.50	0.75	0.70	0.67	0.71	0.82	0.78	0.73	0.75
${A 1, A 2}$	0.80	0.75	0.72	0.76	0.88	0.84	0.81	0.85	0.91	0.88	0.87	0.90

References 18

1	WANG M， DENG W H. Deep face recognition： a survey［J］. Neurocomputing， 2021， 429： 215-244. 10.1016/j.neucom.2020.10.081
2	BORGHI G， FABBRI M， VEZZANI R， et al. Face-from-depth for head pose estimation on depth images［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（3）： 596-609. 10.1109/tpami.2018.2885472
3	BORGHI G， PINI S， VEZZANI R， et al. Driver face verification with depth maps［J］. Sensors， 2019， 19（15）： No.3361. 10.3390/s19153361
4	HUANG G B， RAMESH M， BERG T， et al. Labeled faces in the wild： a database for studying face recognition in unconstrained environments［EB/OL］. ［2021-10-13］.. 10.1117/12.2080393
5	SCHROFF F， KALENICHENKO D， PHILBIN J. FaceNet： a unified embedding for face recognition and clustering［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 815-823. 10.1109/cvpr.2015.7298682
6	YANG J L， REN P R， ZHANG D Q， et al. Neural aggregation network for video face recognition［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5216-5225. 10.1109/cvpr.2017.554
7	CHEN J C， PATEL V M， CHELLAPPA R. Unconstrained face verification using deep CNN features［C］// Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2016： 1-9. 10.1109/wacv.2016.7477557
8	HUANG C， LI Y Y， LOY C C， et al. Deep imbalanced learning for face recognition and attribute prediction［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（11）：2781-2794. 10.1109/tpami.2019.2914680
9	JIAO Q F， LI R， CAO W M， et al. DDAT： dual domain adaptive translation for low-resolution face verification in the wild［J］. Pattern Recognition， 2021， 120： No.108107. 10.1016/j.patcog.2021.108107
10	BALLOTTA D， BORGHI G， VEZZANI R， et al. Fully convolutional network for head detection with depth images［C］// Proceedings of the 24th International Conference on Pattern Recognition. Piscataway： IEEE， 2018： 752-757. 10.1109/icpr.2018.8545332
11	BALLOTTA D， BORGHI G， VEZZANI R， et al. Head detection with depth images in the wild［C］// Proceedings of the 13th International Conference on Computer Vision， Imaging and Computer Graphics Theory and Applications - Volume 4 VISAPP. Setúbal： SciTePress， 2018： 56-63. 10.5220/0006541000560063
12	KIM D， HERNANDEZ M， CHOI J， et al. Deep 3D face identification［C］// Proceedings of the 2017 IEEE International Joint Conference on Biometrics. Piscataway： IEEE， 2017： 133-142. 10.1109/btas.2017.8272691
13	BORGHI G， PINI S， GRAZIOLI F， et al. Face verification from depth using privileged information［C］// Proceedings of the 2018 British Machine Vision Conference. Durham： BMVA Press， 2018： No.192.
14	BROMLEY J， GUYON I， LeCUN Y， et al. Signature verification using a “Siamese” time delay neural network［C］// Proceedings of the 6th International Conference on Neural Information Processing Systems. San Francisco： Morgan Kaufmann Publishers Inc.， 1993： 737-744. 10.1142/9789812797926_0003
15	KOCH G， ZEMEL R， SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition［EB/OL］. ［2021-11-07］..
16	RANJAN R， CASTILLO C D， CHELLAPPA R. L₂-constrained softmax loss for discriminative face verification［EB/OL］. 2017-06-07］［2021-11-07］..
17	BORGHI G， VENTURELLI M， VEZZANI R， et al. POSEidon： face-from-depth for driver pose estimation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 5494-5503. 10.1109/cvpr.2017.583
18	VENTURELLI M， BORGHI G， VEZZANI R， et al. From depth data to head pose estimation： a Siamese approach［EB/OL］.（2017-03-10）［2021-19-20］./ . 10.5220/0006104501940201

[1]	Li’an ZHU, Hong ZHANG. Nonhomogeneous image dehazing based on dual-branch conditional generative adversarial network [J]. Journal of Computer Applications, 2023, 43(2): 567-574.
[2]	Ranyan NI, Yi ZHANG. Action recognition method based on video spatio-temporal features [J]. Journal of Computer Applications, 2023, 43(2): 521-528.
[3]	Yang WANG, Hongliang FU, Huawei TAO, Jing YANG, Yue XIE, Li ZHAO. Cross-corpus speech emotion recognition based on decision boundary optimized domain adaptation [J]. Journal of Computer Applications, 2023, 43(2): 374-379.
[4]	Qing JIA, Laihua WANG, Weisheng WANG. Anomaly detection in video via independently recurrent neural network and variational autoencoder network [J]. Journal of Computer Applications, 2023, 43(2): 507-513.
[5]	Ping WANG, Nan CHEN, Lei LU. Fall detection algorithm based on scene prior and attention guidance [J]. Journal of Computer Applications, 2023, 43(2): 529-535.
[6]	Zhijun SHEN, Lina MU, Jing GAO, Yuanhang SHI, Zhiqiang LIU. Review of fine-grained image categorization [J]. Journal of Computer Applications, 2023, 43(1): 51-60.
[7]	Youxin WANG, Bin CHEN. Print defect detection method based on deep comparison network [J]. Journal of Computer Applications, 2023, 43(1): 250-258.
[8]	Keyou GUO, Xue LI, Min YANG. Real‑time detection method of traffic information based on lightweight YOLOv4 [J]. Journal of Computer Applications, 2023, 43(1): 74-80.
[9]	Yangping LIN, Jia LIU, Pei CHEN, Mingshu ZHANG, Xiaoyuan YANG. Semi-generative video steganography scheme based on deep convolutional generative adversarial net [J]. Journal of Computer Applications, 2023, 43(1): 169-175.
[10]	Haitao YU, Jiansheng LI, Yajiao LIU, Fulong LI, Jiang WANG, Chunhui ZHANG, Lifeng YU. Section steel surface defect detection algorithm based on cascade neural network [J]. Journal of Computer Applications, 2023, 43(1): 232-241.
[11]	Jianzhuang LIN, Wenzhong YANG, Sixiang TAN, Lexin ZHOU, Danni CHEN. Fusing filter enhancement and reverse attention network for polyp segmentation [J]. Journal of Computer Applications, 2023, 43(1): 265-272.
[12]	Jun ZHANG, Pengli WU, Lukui SHI, Jin SHI, Bin PAN. Deep learning model for multi-station temperature prediction combined with MOD11A1 and surface meteorological station data [J]. Journal of Computer Applications, 2023, 43(1): 321-328.
[13]	Bin ZOU, Cong ZHANG. Dense crowd detection algorithm based on Faster R-CNN [J]. Journal of Computer Applications, 2023, 43(1): 61-66.
[14]	Xiaoyu LIU, Huaixin CHEN, Biyuan LIU, Ying LIN, Teng MA. License plate detection algorithm in unrestricted scenes based on adaptive confidence threshold [J]. Journal of Computer Applications, 2023, 43(1): 67-73.
[15]	Yuanyuan GAO, Zhenhua YU, Fang DU, Lijuan SONG. Unlabeled network pruning algorithm based on Bayesian optimization [J]. Journal of Computer Applications, 2023, 43(1): 30-36.