Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (6): 1806-1811.DOI: 10.11772/j.issn.1001-9081.2019101866

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Lightweight human skeleton key point detection model based on improved convolutional pose machines and SqueezeNet

QIANG Baohua1,2, ZHAI Yijie1, CHEN Jinlong1, XIE Wu1, ZHENG Hong1, WANG Xuewen2, ZHANG Shihao1   

  1. 1. Guangxi Key Laboratory of Trusted Software (Guilin University of Electronic Technology), Guilin Guangxi 541004, China
    2. Guangxi Key Laboratory of Image and Graphics Intelligent Processing (Guilin University of Electronic Technology), Guilin Guangxi 541004, China
  • Received:2019-11-01 Revised:2019-12-20 Online:2020-06-10 Published:2020-06-18
  • Contact: CHEN Jinlong, born in 1979, M. S., senior experimentalist. His research interests include image processing, machine learning.
  • About author:QIANG Baohua, born in 1972, Ph. D., professor. His research interests include big data analysis, image processing.ZHAI Yijie, born in 1995, M. S. candidate. Her research interests include human skeleton key point detection, deep learning.CHEN Jinlong, born in 1979, M. S., senior experimentalist. His research interests include image processing, machine learning.XIE Wu, born in 1979, Ph. D., associate professor. His research interests include data mining, information processing.ZHENG Hong, born in 1975, Ph. D., lecturer. Her research interests include image processing, machine learning.WANG Xuewen, born in 1979, M. S., lecturer. His research interests include machine learning, machine vision.ZHANG Shihao, born in 1991, M. S. His research interests include human skeleton key point detection, image processing.
  • Supported by:
    National Natural Science Foundation of China(61762025), the Guangxi Key Research and Development Program(AB17195053, AB18126063), the Natural Science Foundation of Guangxi(2017GXNSFAA198226), the Guilin Science and Technology Development Program(20180107-4).

基于改进CPMs和SqueezeNet的轻量级人体骨骼关键点检测模型

强保华1,2, 翟艺杰1, 陈金龙1, 谢武1, 郑虹1, 王学文2, 张世豪1   

  1. 1.广西可信软件重点实验室(桂林电子科技大学),广西 桂林 541004
    2.广西图像图形与智能处理重点实验室(桂林电子科技大学),广西 桂林 541004
  • 通讯作者: 陈金龙(1979—)
  • 作者简介:强保华(1972—),男,河南南阳人,教授,博士,CCF会员,主要研究方向:大数据分析、图像处理.翟艺杰(1995—),女,河南周口人,硕士研究生,主要研究方向:人体骨骼关键点检测、深度学习.陈金龙(1979—),男,江西高安人,高级实验师,硕士,主要研究方向:图像处理、机器学习.谢武(1979—),男,江西宜春人,副教授,博士,CCF会员,主要研究方向:数据挖掘、信息处理.郑虹(1975—),女,江西吉安人,讲师,博士,主要研究方向:图像处理、机器学习.王学文(1979—),男,湖北黄冈人,讲师,硕士,主要研究方向:机器学习、机器视觉.张世豪(1991—),男,河南许昌人,硕士,主要研究方向:人体骨骼关键点检测、图像处理.
  • 基金资助:
    国家自然科学基金资助项目(61762025);广西重点研究发展计划项目(AB17195053,AB18126063);广西自然科学基金资助项目(2017GXNSFAA198226);桂林科技发展计划项目(20180107-4)。

Abstract: In order to solve the problems of too many parameters, long training time and slow detection speed of the existing human skeleton key point detection models, a detection method combining the human skeleton key point detection model called Convolutional Pose Machines (CPMs) and the lightweight convolutional neural network model called SqueezeNet was proposed. Firstly, the CPMs with 4 stages (CPMs-Stage4) was used to detect the key points of the human images. Then, the Fire Module network structure of SqueezeNet was introduced into CPMs-Stage4 to reduce the model parameters greatly, and thus to obtain a new lightweight human skeleton key point detection model called SqueezeNet15-CPMs-Stage4. The verification results on the extended Leeds Sports Pose (LSP) dataset show that, compared with CPMs, SqueezeNet15-CPMs-Stage4 model has the training time reduced by 86.68%, the detection time of single image reduced by 44.27%, and the detection accuracy of 90.4%; and the proposed model performs the best in training time, detection speed and accuracy compared with three reference models improved VGG-16, DeepCut and DeeperCut. The experimental results show that the proposed model achieves high detection accuracy with short training time and fast detection speed, and can effectively reduce the training cost of the human skeleton key point detection model.

Key words: human skeleton key point detection, human pose estimate, deep learning, Convolutional Neural Network (CNN), lightweight, Convolutional Pose Machines (CPMs), SqueezeNet

摘要: 针对目前的人体骨骼关键点检测模型参数多、训练时间长和检测速度慢的问题,提出了一种将人体骨骼关键点检测模型CPMs与小型卷积神经网络模型SqueezeNet相结合的检测方法。首先,采用4个Stage的CPMs(CPMs-Stage4)对人物图像进行关键点检测;然后,在CPMs-Stage4中引入SqueezeNet的Fire Module网络结构,利用Fire Module结构大大压缩模型参数,得到一种新的轻量级人体骨骼关键点检测模型SqueezeNet15-CPMs-Stage4。在扩展的LSP数据集上的验证结果显示,与CPMs相比,SqueezeNet15-CPMs-Stage4模型在训练时间上减少86.68%,在单张图像检测时间上减少44.27%,准确率达到90.4%;与改进的VGG-16、DeepCut和DeeperCut 三种参照模型相比,SqueezeNet15-CPMs-Stage4模型在训练时间、检测速度和准确率方面均是最优的。实验结果表明,所提模型不仅检测准确率高,而且训练时间短、检测速度快,能够有效降低人体骨骼关键点检测模型的训练成本。

关键词: 人体骨骼关键点检测, 人体姿态估计, 深度学习, 卷积神经网络, 轻量级, CPMs, SqueezeNet

CLC Number: