Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (12): 3602-3607.DOI: 10.11772/j.issn.1001-9081.2021061063

• The 18th China Conference on Machine Learning • Previous Articles    

Smoking behavior detection algorithm based on human skeleton key points

Wanqing XU1,2,3, Baodong WANG1,2,3, Yimei HUANG1,2,3, Jinping LI1,2,3()   

  1. 1.School of Information Science and Engineering,University of Jinan,Jinan Shandong 250022,China
    2.Shandong Provincial Key Laboratory of Network Based Intelligent Computing (University of Jinan),Jinan Shandong 250022,China
    3.Shandong College and University Key Laboratory of Information Processing and Cognitive Computing in the 13th Five-Year Plan (University of Jinan),Jinan Shandong 250022,China
  • Received:2021-05-12 Revised:2021-07-27 Accepted:2021-08-05 Online:2021-12-28 Published:2021-12-10
  • Contact: Jinping LI
  • About author:XU Wanqing, born in 1998, M. S. candidate. Her research interests include image processing, machine learning.
    WANG Baodong, born in 1995, M. S. candidate. His research interests include image processing, machine learning.
    HUANG Yimei, born in 1980, M. S., lecturer. Her research interests include computer vision, pattern recognition, image processing, machine learning.
  • Supported by:
    the Shandong Provincial Key Research and Development Program(2017CXGC0810)

基于人体骨骼关键点的吸烟行为检测算法

徐婉晴1,2,3, 王保栋1,2,3, 黄艺美1,2,3, 李金屏1,2,3()   

  1. 1.济南大学 信息科学与工程学院,济南 250022
    2.山东省网络环境智能计算技术重点实验室(济南大学),济南 250022
    3.山东省“十三五”高校信息处理与认知计算重点实验室(济南大学),济南 250022
  • 通讯作者: 李金屏
  • 作者简介:徐婉晴(1998—),女,河北石家庄人,硕士研究生,CCF会员,主要研究方向:图像处理、机器学习
    王保栋(1995—),男,山东青岛人,硕士研究生,CCF会员,主要研究方向:图像处理、机器学习
    黄艺美(1980—),女,福建永泰人,讲师,硕士,CCF会员,主要研究方向:计算机视觉、模式识别、图像处理、机器学习;
  • 基金资助:
    山东省重点研发计划项目(2017CXGC0810)

Abstract:

In view of the small target of cigarette butts in surveillance videos of public places and the easy divergence of smoke generated by smoking, it is difficult to determine the smoking behavior only by target detection algorithm. Considering that the algorithm of posture estimation using skeleton key points is becoming more and more mature, a smoking behavior detection algorithm was proposed by using the relationship between human skeleton key points and smoking behavior. Firstly, AlphaPose and RetinaFace were used to detect the key points of human skeleton and face respectively. According to the ratio of distance between wrist and middle point of two corners of mouth and between wrist and the eye on the same side, a method for calculating whether the Smoking Action Ratio (SAR) in humans falls within the Golden Ratio of Smoking Actions (GRSA) to distinguish smoking from non-smoking behaviors was proposed. Then, YOLOv4 was used to detect whether cigarette butts existed in the video. The results of GRSA determination and YOLOv4 were combined to determine the possibility of smoking behavior in the video and make a determination of whether smoking behavior was present. The self-recorded dataset test shows that the proposed algorithm can accurately detect smoking behavior with the accuracy reached 92%.

Key words: smoking behavior detection, key points of human skeleton, AlphaPose, RetinaFace, YOLOv4

摘要:

针对公共场所的监控视频中烟头目标较小并且吸烟产生的烟雾易发散,仅依靠目标检测算法检测烟头或者烟雾来判定吸烟行为存在较大难度的问题,考虑到利用骨骼关键点来进行姿态估计的算法越来越成熟,提出一种利用人体骨骼关键点和吸烟行为之间的关系来进行吸烟行为检测的算法。该算法首先利用AlphaPose和RetinaFace分别检测出人体骨骼关键点和脸部关键点信息,根据手腕到两嘴角中点和手腕到同侧眼睛的距离之比,提出一种计算人体的吸烟动作比例(SAR)是否属于吸烟动作黄金比例(GRSA)的方法以区分吸烟与非吸烟行为;再利用YOLOv4检测视频中是否存在烟头;最后结合GRSA判定和YOLOv4的结果来确定视频中存在吸烟行为的可能性高低,作出是否有吸烟行为的判定。经过笔者录制的数据集测试,结果表明所提算法可以准确检测到吸烟行为,准确率达到92%。

关键词: 吸烟行为检测, 人体骨骼关键点, AlphaPose, RetinaFace, YOLOv4

CLC Number: