基于OpenPose-slim模型的人体骨骼关键点检测方法

doi:10.11772/j.issn.1001-9081.2019050954

计算机应用 ›› 2019, Vol. 39 ›› Issue (12): 3503-3509.DOI: 10.11772/j.issn.1001-9081.2019050954

基于OpenPose-slim模型的人体骨骼关键点检测方法

汪检兵^1,2, 李俊^1,2

1. 武汉科技大学计算机科学与技术学院, 武汉 430065;
2. 智能信息处理与实时工业系统湖北省重点实验室(武汉科技大学), 武汉 430065

收稿日期:2019-06-06 修回日期:2019-07-30 出版日期:2019-12-10 发布日期:2019-08-06
作者简介:汪检兵(1998-),男,江西九江人,CCF会员,主要研究方向:计算机视觉、自适应推理;李俊(1978-),男,湖北黄石人,副教授,博士,主要研究方向:智能计算、机器学习。
基金资助:
国家自然科学基金资助项目（61572381）；武汉科技大学智能信息处理与实时工业系统湖北省重点实验室基金资助项目（znxx2018QN06）。

Human skeleton key point detection method based on OpenPose-slim model

WANG Jianbing^1,2, LI Jun^1,2

1. College of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan Hubei 430065, China;
2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System(Wuhan University of Science and Technology), Wuhan Hubei 430065, China

Received:2019-06-06 Revised:2019-07-30 Online:2019-12-10 Published:2019-08-06
Contact: 李俊
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61572381), the Fund from Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System (Wuhan University of Science and Technology) (znxx2018QN06).

摘要/Abstract

摘要： 相较于2017年提出的在当时检测效果近乎最优的RMPE模型与Mask R-CNN模型，原用于人体骨骼关键点检测的OpenPose模型有着在保持精度近乎不变的情况下能大幅缩短检测周期的优势，但同时该模型也存在着参数共享率低、冗余度高、耗时长、模型规模太大等问题。针对上述问题，提出了新的OpenPose-slim模型。该模型减小网络宽度，减少卷积块层数，将原并列式结构改成序列式结构并于内模块加入Dense连接机制，其处理过程主要分为3个模块：1）关键点定位模块，检测出人体骨骼关键点的位置坐标；2）关键点联系模块，把关键点位置连接成肢体；3）肢体匹配模块，进行肢体匹配得到人体轮廓。每一个处理阶段之间关联紧密。在MPII数据集、COCO数据集和AI Challenger数据集上的实验结果表明，所提模型使用4个定位模块和2个联系模块，并于每一个模块内部使用Dense连接机制是最佳结构，与OpenPose模型相比，在保持检测精度基本不变的基础上，测试周期缩短为原来的近1/6，参数量缩小了近50%，模型规模缩小为近1/27。

关键词: 人体骨骼关键点检测, 姿态检测, 肢体, Dense连接机制, 序列式结构

Abstract: The OpenPose model originally used for the detection of key points in human skeleton can greatly shorten the detection cycle while maintaining the accuracy of the Regional Multi-Person Pose Estimation (RMPE) model and the Mask Region-based Convolutional Neural Network (R-CNN) model, which were proposed in 2017 and had the near-optimal detection effect at that time. At the same time, the OpenPose model has the problems such as low parameter sharing rate, high redundancy, long time-consuming and too large model scale. In order to solve the problems, a new OpenPose-slim model was proposed. In the proposed model, the network width was reduced, the number of convolution block layers was decreased, the original parallel structure was changed into sequential structure and the Dense connection mechanism was added to the inner module. The processing process was mainly divided into three modules:1) the position coordinates of human skeleton key points were detected in the key point localization module; 2) the key point positions were connected to the limb in the key point association module; 3) limb matching was performed to obtain the contour of human body in the limb matching module. There is a close correlation between processing stages. The experimental results on the MPII dataset, Common Objects in COntext (COCO) dataset and AI Challenger dataset show that, the use of four localization modules and two association modules as well as the use of Dense connection mechanism inside each module of the proposed model is the best structure. Compared with the OpenPose model, the test cycle of the proposed model is shortened to nearly 1/6, the parameter size is reduced by nearly 50%, and the model size is reduced to nearly 1/27.

Key words: human skeleton key point detection, attitude detection, limb, Dense connection mechanism, sequential structure

中图分类号:

TP183

汪检兵, 李俊. 基于OpenPose-slim模型的人体骨骼关键点检测方法[J]. 计算机应用, 2019, 39(12): 3503-3509.

WANG Jianbing, LI Jun. Human skeleton key point detection method based on OpenPose-slim model[J]. Journal of Computer Applications, 2019, 39(12): 3503-3509.

参考文献

[1] CAO Z, HIDALGO G, SIMON T, et al. OpenPose:realtime multi-person 2D pose estimation using part affinity fields[EB/OL].[2018-12-30]. https://arxiv.org/pdf/1812.08008.pdf.
[2] FANG H, XIE S, TAI Y W, et al. RMPE:regional multi-person pose estimation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:2353-2362.
[3] HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:2980-2988.
[4] WU J, ZHENG H, ZHAO B, et al. AI challenger:a large-scale dataset for going deeper in image understanding[EB/OL].[2018-12-30]. https://arxiv.org/pdf/1711.06475.pdf.
[5] KE L, QI H, CHANG M C, et al. Multi-scale supervised network for human pose estimation[C]//Proceedings of the 2018 IEEE International Conference on Image Processing. Piscataway:IEEE, 2018:564-568.
[6] NEWELL A, YANG K, DENG J. Stacked hourglass networks for human pose estimation[C]//Proceedings of the 2016 European Conference on Computer Vision, LNCS 9912. Cham:Springer, 2016:483-499.
[7] CHEN Y, SHEN C, WEI X, et al. Adversarial PoseNet:a structure-aware convolutional network for human pose estimation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE, 2017:1212-1230.
[8] RAJCHL M, LEE M C H, OKTAY O, et al. DeepCut:object segmentation from bounding box annotations using convolutional neural networks[J]. IEEE Transactions on Medical Imaging, 2017, 36(2):674-683.
[9] INSAFUTDINOV E, PISHCHULIN L, ANDRES B, et al. DeeperCut:a deeper, stronger, and faster multi-person pose estimation model[C]//Proceedings of the 2016 European Conference on Computer Vision, LNCS 9910. Cham:Springer, 2016:34-50.
[10] HENDERSON P, FERRARI V. End-to-end training of object class detectors for mean average precision[C]//Proceedings of the 2016 Asian Conference on Computer Vision, LNCS 10115. Cham:Springer, 2016:198-213.
[11] INSAFUTDINOV E, ANDRILUKA M, PISHCHULIN L, et al. ArtTrack:articulated multi-person tracking in the wild[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:1293-1301.
[12] ZHU X, JIANG Y, LUO Z. Multi-person pose estimation for posetrack with enhanced part affinity fields[EB/OL].[2018-12-30]. https://posetrack.net/workshops/iccv2017/pdfs/ML_Lab.pdf.
[13] HUANG G, LIU Z, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:2261-2269.
[14] 黄奕晖,冯前进.基于三维全卷积DenseNet的脑胶质瘤MRI分割[J].南方医科大学学报,2018,38(6):661-668.(HUANG Y H, FENG Q J. Segmentation of brain tumor on magnetic resonance images using 3D full-convolutional densely connected convolutional networks[J]. Journal of Southern Medical University, 2018, 38(6):661-668.)
[15] HOSANG J, BENENSON R, SCHIELE B. Learning non-maximum suppression[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:6469-6477.
[16] 李默涵,王宏志,李建中,等.一种基于二分图最优匹配的重复记录检测算法[J].计算机研究与发展,2009,46(S2):339-345.(LI M H, WANG H Z, LI J Z, et al. Duplicate record detection method based on optimal bipartite graph matching[J]. Journal of Computer Research and Development, 2009, 46(S2):339-345.)
[17] WANG Z, FENG Z, ZHANG P. An iterative Hungarian algorithm based coordinated spectrum sensing strategy[J]. IEEE Communications Letters, 2011, 15(1):49-51.
[18] PAPANDREOU G, ZHU T, KANAZAWA N, et al. Towards accurate multi-person pose estimation in the wild[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2017:3711-3719.
[19] 黄龙,杨媛,王庆军,等.结合全卷积神经网络的室内场景分割[J].中国图象图形学报,2019,24(1):64-72.(HUANG L, YANG Y, WANG Q J, et al. Indoor scene segmentation based on fully convolutional neural networks[J]. Journal of Image and Graphics, 2019, 24(1):64-72.)
[20] VEIT A, BELONGIE S. Convolutional networks with adaptive inference graphs[C]//Proceedings of the 2018 European Conference on Computer Vision, LNCS 11205. Cham:Springer, 2018:3-18.
[21] 徐毅琼,葛临东,王波,等.基于非监督学习神经网络的自动调制识别研究与实现[J].计算机应用与软件,2011,28(1):79-81,95.(XU Y Q, GE L D, WANG B, et al. On automatic modulation recognition based on unsupervised learning neural networks and its implementation[J]. Computer Applications and Software, 2011, 28(1):79-81, 95.)

基于OpenPose-slim模型的人体骨骼关键点检测方法

Human skeleton key point detection method based on OpenPose-slim model

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics