基于关键姿势的双人交互行为识别

doi:10.11772/j.issn.1001-9081.2019122223

计算机应用 ›› 2020, Vol. 40 ›› Issue (8): 2231-2235.DOI: 10.11772/j.issn.1001-9081.2019122223

基于关键姿势的双人交互行为识别

杨文璐, 于孟孟, 谢宏

上海海事大学信息工程学院, 上海 201306

收稿日期:2020-01-05 修回日期:2020-03-02 出版日期:2020-08-10 发布日期:2020-05-13
通讯作者: 杨文璐(1967-),男,江苏沛县人,副教授,博士,主要研究方向:生物信息处理,wlyang@shmtu.edu.cn
作者简介:于孟孟(1993-),女,河南周口人,硕士研究生,主要研究方向:信号与信息处理;谢宏(1962-),男,陕西汉中人,教授,博士,主要研究方向:人工智能。
基金资助:
国家自然科学基金资助项目（61550110252）。

Recognition of two-person interaction behavior based on key gestures

YANG Wenlu, YU Mengmeng, XIE Hong

College of Information Engineering, Shanghai Maritime University, Shanghai 201306, China

Received:2020-01-05 Revised:2020-03-02 Online:2020-08-10 Published:2020-05-13
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61550110252).

摘要/Abstract

摘要： 针对双人交互行为识别应用领域广但效率低的问题，提出一种基于关键姿势的双人交互行为识别方法。首先，利用帧间差异比较来提取关键帧；然后，利用骨骼点角度变化的方差和空间关系来确定关键帧中的关键姿势；接着，利用关节距离、角度和关节运动等特征表示关键姿势，每一个关键姿势表示为一个特征矩阵；最后，利用不同的降维和分类组合，选取识别率最优的组合。在SBU交互数据集和自建的交互数据集上评估所提出的识别方法，该方法的识别率分别达到92.47%和94.14%。实验结果表明，通过提取关键姿势的特征形成特征矩阵来表示动作的方法可以有效地提高双人交互行为识别结果。

关键词: 行为识别, 双人交互, 关键姿势, 特征矩阵, 体感器

Abstract: Concerning the problem of wide applications and low efficiency of two-person interaction behavior recognition, a method of two-person interaction behavior recognition based on key gestures was proposed. First, the key frames were extracted by comparing the differences between frames. Second, the key gestures in the key frames were determined by using the variance and spatial relationship of the angle changes of the bone points. Then, the key gestures were represented by features such as joint distance, angle, and joint motion. Every key gesture was expressed as a feature matrix. Finally, the combination with the best recognition rate was selected by comparing different dimension reductions and classification combinations. The proposed recognition method was evaluated on the SBU interaction dataset and the self-built interaction dataset, and the recognition rate of it reached 92.47% and 94.14% respectively. Experimental results show that the proposed method of representing actions by extracting the features of key gestures to form feature matrices can effectively improve the recognition result of two-person interaction behavior.

Key words: behavior recognition, two-person interaction, key gesture, feature matrix, body sensor

中图分类号:

TP391.4

杨文璐, 于孟孟, 谢宏. 基于关键姿势的双人交互行为识别[J]. 计算机应用, 2020, 40(8): 2231-2235.

YANG Wenlu, YU Mengmeng, XIE Hong. Recognition of two-person interaction behavior based on key gestures[J]. Journal of Computer Applications, 2020, 40(8): 2231-2235.

参考文献

[1] 胡琼,秦磊,黄庆明.基于视觉的人体动作识别综述[J]. 计算机学报, 2013, 36(12):2512-2524. (HU Q, QIN L, HUANG Q M. A survey on visual human action recognition[J]. Chinese Journal of Computers, 2013, 36(12):2512-2524.)
[2] 姬晓飞,王昌汇,王扬扬. 分层结构的双人交互行为识别方法[J]. 智能系统学报, 2015, 10(6):893-900. (JI X F, WANG C H, WANG Y Y. Human interaction behavior-recognition method based on hierarchical structure[J]. CAAI Transactions on Intelligent Systems, 2015, 10(6):893-900.)
[3] 姬晓飞,秦琳琳,王扬扬. 基于RGB和关节点数据融合模型的双人交互行为识别[J]. 计算机应用, 2019, 39(11):3349-3354. (JI X F,QIN L L,WANG Y Y. Human interaction recognition based on RGB and skeleton data fusion model[J]. Journal of Computer Applications, 2019, 39(11):3349-3354.)
[4] WANG J, ZHOU S C, XIA L M. Human interaction recognition based on sparse representation of feature covariance matrices[J]. Journal of Central South University, 2018, 25(2):304-314.
[5] 金壮壮,曹江涛,姬晓飞. 多源信息融合的双人交互行为识别算法研究[J]. 计算机技术与发展, 2018, 28(10):32-36, 43. (JIN Z Z, CAO J T, JI X F. Research on human interaction recognition algorithm based on multi-source information fusion[J]. Computer Technology and Development, 2018, 28(10):32-36, 43.)
[6] 关百胜.基于卷积神经网络的双人交互行为识别技术研究[D].北京:中国科学院大学,2018:5-6. (GUAN B S. Research on human interaction recognition technology based on convolutional neural network[D]. Beijing:University of Chinese Academy of Sciences, 2018:5-6.)
[7] 袁梦妍. 基于Kinect的动作识别系统[D]. 郑州:郑州大学, 2017:3-7. (YUAN M Y. Action recognition system based on Kinect[D]. Zhengzhou:Zhengzhou University, 2017:3-7.)
[8] PISHARADY P K, SAERBECK M. Kinect based body posture detection and recognition system[C]//Proceedings of the 2013 International Conference on Graphic and Image Processing, SPIE 8768. Bellingham, WA:SPIE, 2013:No.87687F.
[9] XIE C, BI S, DONG M, et al. Key frame extraction of skeleton joint based on Kinect sensor[C]//Proceedings of the IEEE 16th International Conference on Dependable, Autonomic and Secure Computing, 16th International Conference on Pervasive Intelligence and Computing, 4th International Conference on Big Data Intelligence and Computing and Cyber Science and Technology Congress. Piscataway:IEEE, 2018:383-385.
[10] LIU B, CAI H B, JI X, et al. Human-human interaction recognition based on spatial and motion trend feature[C]//Proceedings of the 2017 IEEE International Conference on Image Processing. Piscataway:IEEE, 2017:4547-4551.
[11] JALAL A, QUAID M A K, HASAN A S. Wearable sensor-based human behavior understanding and recognition in daily life for smart environments[C]//Proceedings of the 2018 International Conference on Frontiers of Information Technology. Piscataway:IEEE, 2018:105-110.
[12] JI Y, YE G, CHENG H. Interactive body part contrast mining for human interaction recognition[C]//Proceedings of the 2014 IEEE International Conference on Multimedia and Expo Workshops. Piscataway:IEEE, 2014:1-6.
[13] 邹武合,张雷,戴宁. 基于关键姿态的单次学习动作识别[J]. 半导体光电, 2015, 36(6):999-1005. (ZOU W H, ZHANG L, DAI N. One shot learning action recognition based on key frame[J]. Semiconductor Optoelectronics, 2015, 36(6):999-1005.)
[14] YUN K, HONORIO J, CHATTOPADHYAY D, et al. Two-person interaction detection using body pose features and multiple instance learning[C]//Proceedings of the 2012 Computer Vision and Pattern Recognition Workshops. Piscataway:IEEE, 2012:28-35.
[15] 刘从文.基于Kinect的人的行为识别研究[D].南京:东南大学,2016:22-25. (LIU C W. Research on human behavior recognition based on Kinect[D]. Nanjing:Southeast University, 2016:22-25.)
[16] 石念峰,侯小静,张平,等.姿态估计和跟踪结合的运动视频关键帧提取[J].电视技术,2017,41(4/5):37-46. (SHI N F, HOU X J, ZHANG P, et al. Joint pose estimation and tracking for keyframe extraction of motion video[J]. Video Engineering, 2017, 41(4/5):37-46.)
[17] 李星. 基于3D数据的人体动作识别算法研究[D]. 郑州:郑州大学, 2018:33-37. (LI X. Research on human action recognition algorithm based on 3D data[D]. Zhengzhou:Zhengzhou University, 2018:33-37.)
[18] MÜLLER M, RÖDER T, CLAUSEN M. Efficient content-based retrieval of motion capture data[J]. ACM Transactions on Graphics, 2005, 24(3):677-685.
[19] 张志荣. 人工智能之SVD算法[EB/OL].[2020-02-28].https://mp.ofweek.com/ai/a945673021216. (ZHANG Z R. Artificial intelligence:SVD algorithm[EB/OL].[2020-02-28].https://mp.ofweek.com/ai/a945673021216.)
[20] Matchs. 通俗易懂——决策树算法、随机森林算法讲解(算法+案例)[EB/OL].[2020-02-28].https://blog.csdn.net/weixin_41510260/article/details/84962144?depth_1-utm_source=distribute.pc_relevant.none-task&utm_source=distribute.pc_relevant.none-task. (Matchs. Easy to understand-explanation of decision tree algorithm and random forest algorithm (algorithm + case)[EB/OL].[2020-02-28]. https://blog.csdn.net/weixin_41510260/article/details/84962144?depth_1-utm_source=distribute.pc_relevant.none-task&utm_source=distribute.pc_relevant.none-task.)
[21] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780.
[22] JI Y, CHENG H, ZHENG Y, et al. Learning contrastive feature distribution model for interaction recognition[J]. Journal of Visual Communication and Image Representation, 2015, 33:340-349.

基于关键姿势的双人交互行为识别

Recognition of two-person interaction behavior based on key gestures

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	李扬志, 袁家政, 刘宏哲. 基于时空注意力图卷积网络模型的人体骨架动作识别算法[J]. 计算机应用, 2021, 41(7): 1915-1921.
[2]	刘世泽, 秦艳君, 王晨星, 高存远, 罗海勇, 赵方, 王宝会. 基于多尺度特征提取的交通模式识别算法[J]. 计算机应用, 2021, 41(6): 1573-1580.
[3]	田志强, 邓春华, 张俊雯. 基于骨骼时序散度特征的人体行为识别算法[J]. 计算机应用, 2021, 41(5): 1450-1457.
[4]	蒋丽, 黄仕建, 严文娟. 基于低秩行为信息和多尺度卷积神经网络的人体行为识别方法[J]. 计算机应用, 2021, 41(3): 721-726.
[5]	戎炜, 蒋哲远, 谢昭, 吴克伟. 基于聚类关联网络的群组行为识别[J]. 计算机应用, 2020, 40(9): 2507-2513.
[6]	周云, 陈淑荣. 基于双流非局部残差网络的行为识别方法[J]. 计算机应用, 2020, 40(8): 2236-2240.
[7]	陈皓, 肖利雪, 李广, 潘跃凯, 夏雨. 基于人体关节点数据的攻击性行为识别[J]. 计算机应用, 2019, 39(8): 2235-2241.
[8]	王萍, 庞文浩. 基于视频分段的空时双通道卷积神经网络的行为识别[J]. 计算机应用, 2019, 39(7): 2081-2086.
[9]	郭明祥, 宋全军, 徐湛楠, 董俊, 谢成军. 基于三维残差稠密网络的人体行为识别算法[J]. 计算机应用, 2019, 39(12): 3482-3489.
[10]	王立林, 刘俊. 基于多尺度卷积的船舶行为识别方法[J]. 计算机应用, 2019, 39(12): 3691-3696.
[11]	姬晓飞, 秦琳琳, 王扬扬. 基于RGB和关节点数据融合模型的双人交互行为识别[J]. 计算机应用, 2019, 39(11): 3349-3354.
[12]	吴亮, 何毅, 梅雪, 刘欢. 基于时空兴趣点和概率潜动态条件随机场模型的在线行为识别方法[J]. 计算机应用, 2018, 38(6): 1760-1764.
[13]	田华, 刘俣男, 顾家莹, 陈俏. 基于特征矩阵相似性度量的形状对应性分析[J]. 计算机应用, 2017, 37(6): 1763-1767.
[14]	王智文, 蒋联源, 王宇航, 王日凤, 张灿龙, 黄镇谨, 王鹏涛. 基于尺度自适应局部时空特征的足球比赛视频中的多运动员行为表示[J]. 计算机应用, 2016, 36(8): 2134-2138.
[15]	何传阳, 王平, 张晓华, 宋丹妮. 基于智能监控的中小人群异常行为检测[J]. 计算机应用, 2016, 36(6): 1724-1729.