优化LeNet-5网络的多角度头部姿态估计方法

doi:10.11772/j.issn.1001-9081.2020091427

计算机应用 ›› 2021, Vol. 41 ›› Issue (6): 1667-1672.DOI: 10.11772/j.issn.1001-9081.2020091427

所属专题：人工智能

优化LeNet-5网络的多角度头部姿态估计方法

章惠¹, 张娜娜², 黄俊¹

1. 上海海洋大学信息学院, 上海 201306;
2. 上海建桥学院信息技术学院, 上海 201306

收稿日期:2020-09-15 修回日期:2020-12-03 出版日期:2021-06-10 发布日期:2020-12-14
通讯作者: 张娜娜
作者简介:章惠(1996-),女,浙江温州人,硕士研究生,主要研究方向:图像处理、计算机视觉;张娜娜(1979-),女,山东莱阳人,副教授,硕士,主要研究方向:图像处理、计算机视觉、机器学习;黄俊(1996-),男,浙江温州人,硕士研究生,主要研究方向:图像处理、计算机视觉。
基金资助:
上海市教育委员会“晨光计划”基金资助项目（AASH1702）。

Multi-angle head pose estimation method based on optimized LeNet-5 network

ZHANG Hui¹, ZHANG Nana², HUANG Jun¹

1. College of Information Technology, Shanghai Ocean University, Shanghai 201306, China;
2. College of Information Technology, Shanghai Jian Qiao University, Shanghai 201306, China

Received:2020-09-15 Revised:2020-12-03 Online:2021-06-10 Published:2020-12-14
Supported by:
This work is partially supported by the Shanghai Municipal Education Commission's "Morning Plan" Fund (AASH1702).

摘要/Abstract

摘要： 针对在受到部分遮挡或角度过大无法定位面部关键特征点的情况下，传统的头部姿态估计方法的准确率低或无法进行头部姿态估计的问题，提出了优化LeNet-5网络的多角度头部姿态估计方法。首先，通过对卷积神经网络（CNN）的深度、卷积核大小等进行优化来更好地捕捉图像的全局特征；然后，改进池化层，用卷积操作代替池化操作来增强网络的非线性能力；最后，引入AdaBound优化器，并利用Softmax回归模型做姿态分类训练。训练中在自建数据集中增加遮挡头发、做出夸张表情和佩戴眼镜等动作来增强网络的泛化能力。实验结果表明，所提方法不需要定位面部关键特征点，在光照阴影、头发等遮挡情况下也可以实现抬头、低头、偏头等多角度转动下的头部姿态估计，在Pointing04公共数据集和CAS-PEAL-R1公共数据集上准确率达到了98.7%，运行速度平均在每秒22~29帧。

关键词: 头部姿态估计, 面部关键特征点, LeNet-5网络, 卷积神经网络, 姿态分类

Abstract: In order to solve the problems that the accuracy is low or the head pose estimation cannot be performed by traditional head pose estimation methods when the key feature points of the face cannot be located due to partial occlusion or too large angle, a multi-angle head pose estimation method based on optimized LeNet-5 network was proposed. Firstly, the depth, the size of the convolution kernel and other parameters of the Convolutional Neural Network (CNN) were optimized to better capture the global features of the image. Then, the pooling layers were improved, and a convolutional operation was used to replace the pooling operation to increase the nonlinear ability of the network. Finally, the AdaBound optimizer was introduced, and the Softmax regression model was used to perform the pose classification training. During the training, hair occlusion, exaggerated expressions and wearing glasses were added to the self-built dataset to increase the generalization ability of the network. Experimental results show that, the proposed method can realize the head pose estimation under multi-angle rotations, such as head up, head down and head tilting without locating key facial feature points, under the occlusion of light, shadow and hair, with the accuracy of 98.7% on Pointing04 public dataset and CAS-PEAL-R1 public dataset, and the average running speed of 22-29 frames per second.

Key words: head pose estimation, key facial feature point, LeNet-5 network, Convolutional Neural Network (CNN), pose classification

中图分类号:

TP391.41

章惠, 张娜娜, 黄俊. 优化LeNet-5网络的多角度头部姿态估计方法[J]. 计算机应用, 2021, 41(6): 1667-1672.

ZHANG Hui, ZHANG Nana, HUANG Jun. Multi-angle head pose estimation method based on optimized LeNet-5 network[J]. Journal of Computer Applications, 2021, 41(6): 1667-1672.

参考文献

[1] AHN B, CHOI D G, PARK J, et al. Real-time head pose estimation using multi-task deep neural network[J]. Robotics and Autonomous Systems,2018,103:1-12.
[2] 张进, 张娜娜. 优化特征提取的互动式人脸活体检测研究[J]. 计算机工程与应用, 2019, 55(13):193-200.(ZHANG J,ZHANG N N. Research on interactive face detection based on optimized feature extraction[J]. Computer Engineering and Applications, 2019,55(13):193-200.)
[3] PATACCHIOLA M,CANGELOSI A. Head pose estimation in the wild using convolutional neural networks and adaptive gradient methods[J]. Pattern Recognition,2017,71:132-143.
[4] MANJANI I,TARIYAL S,VATSA M,et al. Detecting silicone mask-based presentation attack via deep dictionary learning[J]. IEEE Transactions on Information Forensics and Security,2017,12(7):1713-1723.
[5] 桑高丽, 陈虎, 赵启军. 一种基于深度卷积网络的鲁棒头部姿态估计方法[J]. 四川大学学报(工程科学版), 2016, 48(S1):163-169. (SANG G L,CHEN H,ZHANG Q J. Robust head pose estimation based on deep convolution neural networks[J]. Journal of Sichuan University(Engineering Science Edition),2016,48(S1):163-169.)
[6] CHUN J,KIM W. 3D face pose estimation by a robust real time tracking of facial features[J]. Multimedia Tools and Applications, 2016,75(23):15693-15708.
[7] 闵秋莎, 刘能, 陈雅婷, 等. 基于面部特征点定位的头部姿态估计[J]. 计算机工程, 2018, 44(6):263-269.(MIN Q S,LIU N, CHEN Y T,et al. Head pose estimation based on facial feature point localization[J]. Computer Engineering,2018,44(6):263-269.)
[8] RANJAN R,PATEL V M,CHELLAPPA R. HyperFace:a deep multi-task learning framework for face detection, landmark localization,pose estimation,and gender recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2019, 41(1):121-135.
[9] 贺飞翔, 赵启军. 基于深度学习的头部姿态估计[J]. 计算机技术与发展, 2016, 26(11):1-4.(HE F X,ZHAO Q J. Head pose estimation based on deep learning[J]. Computer Technology and Development,2016,26(11):1-4.)
[10] 李成龙, 钟凡, 马昕, 等. 基于卡尔曼滤波和随机回归森林的实时头部姿态估计[J]. 计算机辅助设计与图形学学报, 2017, 29(12):2309-2316.(LI C L,ZHONG F,MA X,et al. Real-time head pose estimation based on Kalman filter and random regression forest[J]. Journal of Computer-Aided Design and Computer Graphics,2017,29(12):2309-2316.)
[11] YANG T Y,CHEN Y T,LIN Y Y,et al. FSA-Net:learning finegrained structure aggregation for head pose estimation from a single image[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2019:1087-1096.
[12] LECUN Y,BOSER B,DENKER J S,et al. Handwritten digit recognition with a back-propagation network[C]//Proceedings of the 19892nd International Conference on Neural Information Processing Systems. Cambridge:MIT Press,1989:396-404.
[13] GENG X,XIA Y. Head pose estimation based on multivariate label distribution[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2014:1837-1842.
[14] 冯浪, 张玲, 张晓龙. 基于扩张卷积的图像修复[J]. 计算机应用, 2020, 40(3):825-831.(FENG L,ZHANG L,ZHANG X L. Image inpainting based on dilated convolution[J]. Journal of Computer Applications,2020,40(3):825-831.)
[15] SCHROFF F,KALENICHENKO D,PHILBIN J. FaceNet:a unified embedding for face recognition and clustering[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:815-823.
[16] 许凡, 程华, 房一泉. 基于CLSTM的步态分类方法[J]. 华东理工大学学报(自然科学版), 2017, 43(4):553-558.(XU F, CHENG H,FANG Y Q. A gait pattern classification method based on CLSTM[J]. Journal of East China University of Science and Technology (Natural Science Edition), 2017, 43(4):553-558.)
[17] LATHUILIÈRE S,JUGE R,MESEJO P,et al. Deep mixture of linear inverse regressions applied to head-pose estimation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2017:7149-7157.
[18] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2020-10-11]. https://arxiv.org/pdf/1409.1556.pdf.
[19] 朱雯文, 叶西宁. 基于卷积神经网络的手势识别算法[J]. 华东理工大学学报(自然科学版), 2018, 44(2):260-269.(ZHU W W,YE X N. Convolution neural networks for gesture recognition[J]. Journal of East China University of Science and Technology (Natural Science Edition),2018,44(2):260-269.)
[20] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research,2014,15:1929-1958.
[21] ZHANG K,ZHANG Z,LI Z,et al. Joint face detection and alignment using multitask cascaded convolutional networks[J]. IEEE Signal Processing Letters,2016,23(10):1499-1503.
[22] LUO L,XIONG Y,LIU Y,et al. Adaptive gradient methods with dynamic bound of learning rate[EB/OL].[2020-10-11]. https://arxiv.org/pdf/1902.09843.pdf.
[23] 陆清正, 周宇, 叶庆卫, 等. 基于局部二值特征和BP神经网络的头部姿态估计[J]. 传感器与微系统, 2019, 38(2):45-48,58. (LU Q Z,ZHOU Y,YE Q W,et al. Head pose estimation based on local binary features and BP neural network[J]. Transducer and Microsystem Technologies,2019,38(2):45-48,58.)
[24] GUPTA A,THAKKAR K,GANDHI V,et al. Nose,eyes and ears:head pose estimation by locating facial keypoints[C]//Proceedings of the 2019 IEEE Conference on Acoustics,Speech and Signal Processing. Piscataway:IEEE,2019:1977-1981.

优化LeNet-5网络的多角度头部姿态估计方法

Multi-angle head pose estimation method based on optimized LeNet-5 network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[2]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[3]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[4]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[5]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[6]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[7]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[8]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[9]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[10]	黄程程, 董霄霄, 李钊. 基于二维Winograd算法的深流水线5×5卷积方法[J]. 计算机应用, 2021, 41(8): 2258-2264.
[11]	曾祥银, 郑伯川, 刘丹. 基于深度卷积神经网络和聚类的左右轨道线检测[J]. 计算机应用, 2021, 41(8): 2324-2329.
[12]	谭道强, 曾诚, 乔金霞, 张俊. 基于混合注意力模型的阴影检测方法[J]. 计算机应用, 2021, 41(7): 2076-2081.
[13]	武光利, 李雷霆, 郭振洲, 王成祥. 基于改进的双向长短期记忆网络的视频摘要生成模型[J]. 计算机应用, 2021, 41(7): 1908-1914.
[14]	吴则举, 焦翠娟, 陈亮. 基于改进Faster R-CNN的轮胎缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1939-1946.
[15]	杨粟, 欧阳智, 杜逆索. 基于相关度距离的无监督并行哈希图像检索[J]. 计算机应用, 2021, 41(7): 1902-1907.