基于深度图像与骨骼数据的行为识别

doi:10.11772/j.issn.1001-9081.2016.11.2979

计算机应用 ›› 2016, Vol. 36 ›› Issue (11): 2979-2984.DOI: 10.11772/j.issn.1001-9081.2016.11.2979

• 第十六届中国粗糙集与软计算联合学术会议(CRSSC 2016)论文 • 上一篇下一篇

基于深度图像与骨骼数据的行为识别

陆中秋, 侯振杰, 陈宸, 梁久祯

常州大学信息科学与工程学院, 江苏常州 213164

收稿日期:2016-06-07 修回日期:2016-06-27 出版日期:2016-11-10 发布日期:2016-11-12
通讯作者: 侯振杰
作者简介:陆中秋(1991-),男,江苏张家港人,硕士研究生,CCF会员,主要研究方向:行为识别;侯振杰(1973-),男,内蒙古包头人,教授,博士,CCF会员,主要研究方向:机器视觉、机器学习;陈宸(1982-),男,江苏常州人,博士,主要研究方向:数字图像处理、机器学习;梁久祯(1968-),男,山东济南人,教授,博士,CCF会员,主要研究方向:机器视觉。
基金资助:
国家自然科学基金资助项目（61063021）；江苏省产学研前瞻性联合研究项目（BY2015027-12）。

Action recognition based on depth images and skeleton data

LU Zhongqiu, HOU Zhenjie, CHEN Chen, LIANG Jiuzhen

School of Information Science & Engineering, Changzhou University, Changzhou Jiangsu 213164, China

Received:2016-06-07 Revised:2016-06-27 Online:2016-11-10 Published:2016-11-12
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61063021), Industry, Teaching and Research Prospective Project of Jiangsu Province (BY2015027-12).

摘要/Abstract

摘要： 为了充分利用深度图像与骨骼数据进行人体行为识别，提出了一种基于深度图形与骨骼数据的多特征行为识别方法。该算法的多特征包括深度运动图（DMM）特征与四方形骨骼特征（Quad）。深度图像方面，将深度图像投影到一个笛卡尔坐标系的三个平面获得深度运动图特征。骨骼数据方面，提出四方形骨骼特征，它是骨骼坐标的一种标定方式，得到的结果只与骨骼姿态有关。同时提出一种多模型概率投票的分类策略，减小了噪声数据对分类结果的影响。所提方法在MSR-Action3D和DHA数据库进行实验，实验结果表明，所提算法有着较高的识别率与良好的鲁棒性。

关键词: 深度图像, 骨骼数据, 行为识别, 深度运动图, 四方形骨骼特征

Abstract: In order to make full use of depth images and skeleton data for action detection, a multi-feature human action recognition method based on depth images and skeleton data was proposed. Multi-features included Depth Motion Map (DMM) feature and Quadruples skeletal feature (Quad). In aspect of depth images, DMM could be captured by projecting the depth image onto the three plane of a Descartes coordinate system. In aspect of skeleton data, Quad was a kind of calibration method for skeleton features and the results were only related to the skeleton posture. Meanwhile, a strategy of multi-model probabilistic voting model was proposed to reduce the influence from noise data on the classification. The proposed method was evaluated on Microsoft Research Action 3D dataset and Depth-included Human Action (DHA) database. The results indicate that the method has high accuracy and good robustness.

Key words: depth image, skeleton data, action recognition, depth motion map, Quadruples skeletal feature (Quad)

中图分类号:

TP391.4

陆中秋, 侯振杰, 陈宸, 梁久祯. 基于深度图像与骨骼数据的行为识别[J]. 计算机应用, 2016, 36(11): 2979-2984.

LU Zhongqiu, HOU Zhenjie, CHEN Chen, LIANG Jiuzhen. Action recognition based on depth images and skeleton data[J]. Journal of Computer Applications, 2016, 36(11): 2979-2984.

参考文献

[1] XIA L, CHEN C C, AGGARWAL J K. View invariant human action recognition using histograms of 3D joints[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ:IEEE, 2012:20-27.
[2] YANG X, TIAN Y. Eigen-joints-based action recognition using naive-Bayes-nearest-neighbor[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ:IEEE, 2012:14-19.
[3] LI W, ZHANG Z, LIU Z. Action recognition based on a bag of 3D points[C]//Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ:IEEE, 2010:9-14.
[4] SHOTTON J, FITZGIBBON A, COOK M, et al. Real-time human pose recognition in parts from single depth images[C]//Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2011:1297-1304.
[5] SCHULDT C, LAPTEV I, CAPUTO B. Recognizing human actions:a local SVM approach[C]//Proceedings of the 17th IEEE International Conference on Pattern Recognition. Piscataway, NJ:IEEE, 2004, 3:32-36.
[6] SUN J, WU X, YAN S, et al. Hierarchical spatio-temporal context modeling for action recognition[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2009:2004-2011.
[7] BOBICK A, DAVIS J. The recognition of human movement using temporal templates[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(3):257-267.
[8] 蔡加欣,冯国灿.基于局部轮廓和随机森林的人体行为识别[J].光学学报,2014, 34(10):204-213.(CAI J X, FENG G C. Action recognition based on local contour and random forest[J]. Acta Optica Sinica, 2014, 34(10):204-213.)
[9] LI W, ZHANG Z, LIU Z. Action recognition based on a bag of 3D points[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Piscataway, NJ:IEEE, 2010:9-14.
[10] YANG X, ZHANG C,TIAN Y. Recognizing actions using depth motion maps-based histograms of oriented gradients[C]//Proceedings of the 20th ACM International Conference on Multimedia. New York:ACM, 2012:1057-1060.
[11] WANG J, LIU Z, CHOROWSKI J, et al. Robust 3D action recognition with random occupancy patterns[C]//Proceedings of the 12th European Conference on Computer Vision. Berlin:Springer-Verlag, 2012:872-885.
[12] 郑胤,陈权崎,章毓晋.深度学习及其在目标和行为识别中的新进展[J].中国图象图形学报,2014,19(2):175-184.(ZHENG Y, CHEN Q Q, ZHANG Y J. The new development of deep learning in action recognition[J]. Journal of Image and Graphics, 2014,19(2):175-184.)
[13] CHAARAOUI A A, PADILLA-L'OPEZ J R, CLIMENT-P'EREZ P, et al. Evolutionary joint selection to improve human action recognition with RGB-D devices[J]. Expert Systems with Applications, 2014, 41(3):786-794.
[14] CHAUDHRY R, OFLI F, KURILLO G, et al. Bio-inspired dynamic 3D discriminative skeletal features for human action recognition[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Washington, DC:IEEE Computer Society, 2013:471-478.
[15] LIN Y Y. Depth and skeleton associated action recognition without online accessible RGB-D cameras[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2014:2617-2624.
[16] VIEIRA A W, NASCIMENTO E, OLIVEIRA G. STOP:space-time occupancy patterns for 3D action recognition from depth map sequences[C]//Proceedings of the 17th Iberoamerican Congress on Pattern Recognition. Berlin:Springer-Verlag, 2012:252-259.
[17] GEORAIOS E, GURKIRT S, RADU H. Skeletal quads:human action recognition using joint quadruples[C]//Proceedings of the 201422nd International Conference on Pattern Recognition. Piscataway, NJ:IEEE, 2014:4513-4518.
[18] XIA L, AGGARWAL J. Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2013:2834-2841.
[19] OREIFEI O, LIU Z. HON4D:histogram of oriented 4D normals for activity recognition from depth sequences[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2013:716-723.
[20] WANG J, LIU Z, WU Y. Mining actionlet ensemble for action recognition with depth cameras[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2012:1290-1297.
[21] TRAN Q D, LY N Q. Sparse spatio-temporal representation of joint shape-motion cues for human action recognition in depth sequences[C]//Proceedings of the 2013 IEEE RIVF International Conference on Computing and Communication Technologies. Research, Innovation, and Vision for the Future. Piscataway, NJ:IEEE, 2013:253-258.
[22] RAHMANI H, MAHMOOD A, HUYNH D Q, et al. Real time action recognition using histograms of depth gradients and random decision forests[C]//Proceedings of the 2014 IEEE Winter Conference on Applications of Computer Vision. Piscataway, NJ:IEEE, 2014:626-633.
[23] VEMULAPALLI R, ARRATE F, CHELLAPPA R. Human action recognition by representing 3D human skeletons as points in a lie group[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ:IEEE, 2014:588-595.
[24] CHEN C, LIU K, KEHTARNAVAZ N. Real-time human action recognition based on depth motion maps[J]. Journal of Real-Time Image Processing, 2013, 12(1):155-163.

基于深度图像与骨骼数据的行为识别

Action recognition based on depth images and skeleton data

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李扬志, 袁家政, 刘宏哲. 基于时空注意力图卷积网络模型的人体骨架动作识别算法[J]. 计算机应用, 2021, 41(7): 1915-1921.
[2]	刘世泽, 秦艳君, 王晨星, 高存远, 罗海勇, 赵方, 王宝会. 基于多尺度特征提取的交通模式识别算法[J]. 计算机应用, 2021, 41(6): 1573-1580.
[3]	田志强, 邓春华, 张俊雯. 基于骨骼时序散度特征的人体行为识别算法[J]. 计算机应用, 2021, 41(5): 1450-1457.
[4]	蒋丽, 黄仕建, 严文娟. 基于低秩行为信息和多尺度卷积神经网络的人体行为识别方法[J]. 计算机应用, 2021, 41(3): 721-726.
[5]	戎炜, 蒋哲远, 谢昭, 吴克伟. 基于聚类关联网络的群组行为识别[J]. 计算机应用, 2020, 40(9): 2507-2513.
[6]	周云, 陈淑荣. 基于双流非局部残差网络的行为识别方法[J]. 计算机应用, 2020, 40(8): 2236-2240.
[7]	杨文璐, 于孟孟, 谢宏. 基于关键姿势的双人交互行为识别[J]. 计算机应用, 2020, 40(8): 2231-2235.
[8]	张豪, 张强, 邵思羽, 丁海斌. 深度学习在单图像三维模型重建的应用[J]. 计算机应用, 2020, 40(8): 2351-2357.
[9]	陈皓, 肖利雪, 李广, 潘跃凯, 夏雨. 基于人体关节点数据的攻击性行为识别[J]. 计算机应用, 2019, 39(8): 2235-2241.
[10]	王萍, 庞文浩. 基于视频分段的空时双通道卷积神经网络的行为识别[J]. 计算机应用, 2019, 39(7): 2081-2086.
[11]	喻露, 胡剑锋, 姚磊岳. 基于人体骨架的非标准深蹲姿势检测方法[J]. 计算机应用, 2019, 39(5): 1448-1452.
[12]	郭明祥, 宋全军, 徐湛楠, 董俊, 谢成军. 基于三维残差稠密网络的人体行为识别算法[J]. 计算机应用, 2019, 39(12): 3482-3489.
[13]	王立林, 刘俊. 基于多尺度卷积的船舶行为识别方法[J]. 计算机应用, 2019, 39(12): 3691-3696.
[14]	姬晓飞, 秦琳琳, 王扬扬. 基于RGB和关节点数据融合模型的双人交互行为识别[J]. 计算机应用, 2019, 39(11): 3349-3354.
[15]	吴亮, 何毅, 梅雪, 刘欢. 基于时空兴趣点和概率潜动态条件随机场模型的在线行为识别方法[J]. 计算机应用, 2018, 38(6): 1760-1764.