Lightweight human skeleton key point detection model based on improved convolutional pose machines and SqueezeNet

doi:10.11772/j.issn.1001-9081.2019101866

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (6): 1806-1811.DOI: 10.11772/j.issn.1001-9081.2019101866

• Virtual reality and multimedia computing • Previous Articles Next Articles

Lightweight human skeleton key point detection model based on improved convolutional pose machines and SqueezeNet

QIANG Baohua^1,2, ZHAI Yijie¹, CHEN Jinlong¹, XIE Wu¹, ZHENG Hong¹, WANG Xuewen², ZHANG Shihao¹

1. Guangxi Key Laboratory of Trusted Software (Guilin University of Electronic Technology), Guilin Guangxi 541004, China
2. Guangxi Key Laboratory of Image and Graphics Intelligent Processing (Guilin University of Electronic Technology), Guilin Guangxi 541004, China

Received:2019-11-01 Revised:2019-12-20 Online:2020-06-18 Published:2020-06-10
Contact: CHEN Jinlong, born in 1979, M. S., senior experimentalist. His research interests include image processing, machine learning.
About author:QIANG Baohua, born in 1972, Ph. D., professor. His research interests include big data analysis, image processing.ZHAI Yijie, born in 1995, M. S. candidate. Her research interests include human skeleton key point detection, deep learning.CHEN Jinlong, born in 1979, M. S., senior experimentalist. His research interests include image processing, machine learning.XIE Wu, born in 1979, Ph. D., associate professor. His research interests include data mining, information processing.ZHENG Hong, born in 1975, Ph. D., lecturer. Her research interests include image processing, machine learning.WANG Xuewen, born in 1979, M. S., lecturer. His research interests include machine learning, machine vision.ZHANG Shihao, born in 1991, M. S. His research interests include human skeleton key point detection, image processing.
Supported by:
National Natural Science Foundation of China(61762025), the Guangxi Key Research and Development Program(AB17195053, AB18126063), the Natural Science Foundation of Guangxi(2017GXNSFAA198226), the Guilin Science and Technology Development Program(20180107-4).

基于改进CPMs和SqueezeNet的轻量级人体骨骼关键点检测模型

强保华^1,2, 翟艺杰¹, 陈金龙¹, 谢武¹, 郑虹¹, 王学文², 张世豪¹

1.广西可信软件重点实验室(桂林电子科技大学)，广西桂林 541004
2.广西图像图形与智能处理重点实验室(桂林电子科技大学)，广西桂林 541004

通讯作者: 陈金龙(1979—)
作者简介:强保华(1972—)，男，河南南阳人，教授，博士，CCF会员，主要研究方向：大数据分析、图像处理.翟艺杰(1995—)，女，河南周口人，硕士研究生，主要研究方向：人体骨骼关键点检测、深度学习.陈金龙(1979—)，男，江西高安人，高级实验师，硕士，主要研究方向：图像处理、机器学习.谢武(1979—)，男，江西宜春人，副教授，博士，CCF会员，主要研究方向：数据挖掘、信息处理.郑虹(1975—)，女，江西吉安人，讲师，博士，主要研究方向：图像处理、机器学习.王学文(1979—)，男，湖北黄冈人，讲师，硕士，主要研究方向：机器学习、机器视觉.张世豪(1991—)，男，河南许昌人，硕士，主要研究方向：人体骨骼关键点检测、图像处理.
基金资助:
国家自然科学基金资助项目(61762025);广西重点研究发展计划项目(AB17195053,AB18126063);广西自然科学基金资助项目(2017GXNSFAA198226);桂林科技发展计划项目(20180107-4)。

Abstract

Abstract: In order to solve the problems of too many parameters, long training time and slow detection speed of the existing human skeleton key point detection models, a detection method combining the human skeleton key point detection model called Convolutional Pose Machines (CPMs) and the lightweight convolutional neural network model called SqueezeNet was proposed. Firstly, the CPMs with 4 stages (CPMs-Stage4) was used to detect the key points of the human images. Then, the Fire Module network structure of SqueezeNet was introduced into CPMs-Stage4 to reduce the model parameters greatly, and thus to obtain a new lightweight human skeleton key point detection model called SqueezeNet15-CPMs-Stage4. The verification results on the extended Leeds Sports Pose (LSP) dataset show that, compared with CPMs, SqueezeNet15-CPMs-Stage4 model has the training time reduced by 86.68%, the detection time of single image reduced by 44.27%, and the detection accuracy of 90.4%; and the proposed model performs the best in training time, detection speed and accuracy compared with three reference models improved VGG-16, DeepCut and DeeperCut. The experimental results show that the proposed model achieves high detection accuracy with short training time and fast detection speed, and can effectively reduce the training cost of the human skeleton key point detection model.

Key words: human skeleton key point detection, human pose estimate, deep learning, Convolutional Neural Network (CNN), lightweight, Convolutional Pose Machines (CPMs), SqueezeNet

摘要： 针对目前的人体骨骼关键点检测模型参数多、训练时间长和检测速度慢的问题，提出了一种将人体骨骼关键点检测模型CPMs与小型卷积神经网络模型SqueezeNet相结合的检测方法。首先，采用4个Stage的CPMs(CPMs-Stage4)对人物图像进行关键点检测；然后，在CPMs-Stage4中引入SqueezeNet的Fire Module网络结构，利用Fire Module结构大大压缩模型参数，得到一种新的轻量级人体骨骼关键点检测模型SqueezeNet15-CPMs-Stage4。在扩展的LSP数据集上的验证结果显示，与CPMs相比，SqueezeNet15-CPMs-Stage4模型在训练时间上减少86.68%，在单张图像检测时间上减少44.27%，准确率达到90.4%；与改进的VGG-16、DeepCut和DeeperCut 三种参照模型相比，SqueezeNet15-CPMs-Stage4模型在训练时间、检测速度和准确率方面均是最优的。实验结果表明，所提模型不仅检测准确率高，而且训练时间短、检测速度快，能够有效降低人体骨骼关键点检测模型的训练成本。

关键词: 人体骨骼关键点检测, 人体姿态估计, 深度学习, 卷积神经网络, 轻量级, CPMs, SqueezeNet

CLC Number:

TP391.4

QIANG Baohua, ZHAI Yijie, CHEN Jinlong, XIE Wu, ZHENG Hong, WANG Xuewen, ZHANG Shihao. Lightweight human skeleton key point detection model based on improved convolutional pose machines and SqueezeNet[J]. Journal of Computer Applications, 2020, 40(6): 1806-1811.

强保华, 翟艺杰, 陈金龙, 谢武, 郑虹, 王学文, 张世豪. 基于改进CPMs和SqueezeNet的轻量级人体骨骼关键点检测模型[J]. 计算机应用, 2020, 40(6): 1806-1811.

References

1 YANGX J. Human recognition using multi-frame gait silhouette matching [J]. International Journal of Advancements in Computing Technology, 2012, 4(22): 788-795.
2 MOESLUNDT B, HILTONA, KRüGERV. A survey of advances in vision-based human motion capture and analysis [J]. Computer Vision and Image Understanding, 2006, 104(2/3): 90-126.
3 WUX, LIANGW, JIAY. Action recognition feedback-based framework for human pose reconstruction from monocular images [J]. Pattern Recognition Letters, 2009, 30(12): 1077-1085.
4 HUANGC L, CHUNGC Y. A real-time model-based human motion tracking and analysis for human-computer interface systems [J]. EURASIP Journal on Advances in Signal Processing, 2004, 2004(11): Article No. 616891.
5 郑远攀,李广阳,李晔.深度学习在图像识别中的应用研究综述[J].计算机工程与应用,2019,55(12):20-36. ZHENGY P, LIG Y, LIY. Survey of application of deep learning in image recognition [J]. Computer Engineering and Applications, 2019, 55(12): 20-36.
6 LIFSHITZI, FETAYAE, ULLMANS. Human pose estimation using deep consensus voting [C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9906. Cham: Springer, 2016: 246-260.
7 PISHCHULINL, INSAFUTDINOVE, TANGS, et al. DeepCut: joint subset partition and labeling for multi person pose estimation[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 4929-4937.
8 INSAFUTDINOVE, PISHCHULINL, ANDRESB, et al. DeeperCut: a deeper, stronger, and faster multi-person pose estimation model [C]// Proceedings of the 2016 European Conference on Computer Vision, LNCS 9910. Cham: Springer, 2016: 34-50.
9 WEIS E, RAMAKRISHNAV, KANADET, et al. Convolutional pose machines [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 4724-4732.
10 ANDRILUKAM, PISHCHULINL, GEHLERP, et al. MPII human pose dataset [DB/OL]. [2019-04-07]. http://human-pose.mpi-inf.mpg.de.
11 JOHNSONS, EVERINGHAMM. Leeds sports pose dataset [DB/OL]. [2019-04-07]. http://sam.johnson.io/research/lsp.html.
12 IANDOLAF N, HANS, MOSKEWICZM W, et al. SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size [EB/OL]. [2019-02-21]. https://arxiv.org/pdf/1602.07360.pdf.
13 RAMAKRISHNAV, MUNOZD, HEBERTM, et al. Pose machines: articulated pose estimation via inference machines [C]// Proceedings of the 2014 European Conference on Computer Vision, LNCS 8690. Cham: Springer, 2014: 33-47.
14 张世豪.基于深度学习的人体骨骼关键点检测方法研究[D].桂林:桂林电子科技大学,2019:38-42. ZHANGS H. Research on human pose estimation method based on deep learning [D]. Guilin: Guilin University of Electronic Technology, 2019:38-42.
15 QIANGB, ZHANGS, ZHANY, et al. Improved convolutional pose machines for human pose estimation using image sensor data [J]. Sensors, 2019, 19(3): Article No. 718.
16 JOHNSONS, EVERINGHAMM. Leeds sports pose extended training dataset [DB/OL]. [2019-04-07]. http://sam.johnson.io/research/lspet.html.
17 武妍,张立明.神经网络的泛化能力与结构优化算法研究[J].计算机应用研究,2002,19(6):21-25,84. WUY, ZHANGL M. A survey of research work on neural network generalization and structure optimization algorithms [J]. Application Research of Computers, 2002, 19(6): 21-25, 84.
18 MOHAMEDA, HINTONG, PENNG. Understanding how deep belief networks perform acoustic modeling [C]// Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2012: 4273-4276.

[1]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[2]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[3]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[4]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[5]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[6]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[7]	Yanjun LI, Yaodong GE, Qi WANG, Weiguo ZHANG, Chen LIU. Improved KLEIN algorithm and its quantum analysis [J]. Journal of Computer Applications, 2024, 44(9): 2810-2817.
[8]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[9]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[10]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[11]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[12]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[13]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[14]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[15]	Yongjin ZHANG, Jian XU, Mingxing ZHANG. Lightweight algorithm for impurity detection in raw cotton based on improved YOLOv7 [J]. Journal of Computer Applications, 2024, 44(7): 2271-2278.

Lightweight human skeleton key point detection model based on improved convolutional pose machines and SqueezeNet

基于改进CPMs和SqueezeNet的轻量级人体骨骼关键点检测模型

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics