基于低秩行为信息和多尺度卷积神经网络的人体行为识别方法

doi:10.11772/j.issn.1001-9081.2020060958

计算机应用 ›› 2021, Vol. 41 ›› Issue (3): 721-726.DOI: 10.11772/j.issn.1001-9081.2020060958

所属专题：人工智能

基于低秩行为信息和多尺度卷积神经网络的人体行为识别方法

蒋丽, 黄仕建, 严文娟

长江师范学院电子信息工程学院, 重庆 408100

收稿日期:2020-07-06 修回日期:2020-10-11 出版日期:2021-03-10 发布日期:2021-01-15
通讯作者: 蒋丽
作者简介:蒋丽(1990-),女,重庆合川人,助教,硕士,主要研究方向:计算机视觉、智能信息处理;黄仕建(1983-),男,四川仁寿人,副教授,博士,主要研究方向:计算机视觉、机器学习;严文娟(1976-),女,重庆人,副教授,硕士,主要研究方向:机器学习、模式识别。
基金资助:
重庆市教委科学技术研究计划项目（KJQN202001421）。

Human action recognition method based on low-rank action information and multi-scale convolutional neural network

JIANG Li, HUANG Shijian, YAN Wenjuan

School of Electronic Information Engineering, Yangtze Normal University, Chongqing 408100, China

Received:2020-07-06 Revised:2020-10-11 Online:2021-03-10 Published:2021-01-15
Supported by:
This work is partially supported by the Scientific and Technological Research Program of Chongqing Municipal Education Commission (KJQN202001421).

摘要/Abstract

摘要： 针对人体行为识别中传统行为信息获取方法需要繁琐步骤和各类假设的问题，结合卷积神经网络（CNN）在图像视频处理中的优越性能，提出了一种基于低秩行为信息（LAI）和多尺度卷积神经网络（MCNN）的人体行为识别方法。首先，对行为视频进行分段，并分别对每个视频段进行低秩学习以提取到相应的LAI，然后在时间轴上对这些LAI进行连接以获取整个视频的LAI，进而有效捕获视频中的行为信息，避免了繁琐的提取步骤和各类假设。其次，针对LAI的特点，设计了MCNN模型。该模型通过多尺度卷积核获取不同感受野下的LAI行为特征，并合理设计各卷积层、池化层及全连接层来进一步提炼特征并最终输出行为类别。将所提出的方法在KTH和HMDB51两个基准数据库上进行性能验证，同时设计和进行了三组对比实验。实验结果表明，所提方法在两个数据库上分别取得了97.33%和72.05%的识别率，与双重变换（TFT）方法和深时间嵌入网络（DTEN）方法相比，识别率分别至少提高了0.67和1.15个百分点。所提方法能进一步促进行为识别技术在安防、人机交互等领域的广泛应用。

关键词: 行为识别, 低秩学习, 行为信息, 多尺度, 卷积神经网络

Abstract: In view of the problem that traditional methods of action information acquisition in human action recognition need cumbersome steps and various assumptions, and considering the superior performance of Convolutional Neural Network (CNN) in image and video processing, a human action recognition method based on Low-rank Action Information (LAI) and Multi-scale Convolutional Neural Network (MCNN) was proposed. Firstly, the action video was divided into several segments, and the LAI of each segment was extracted by the low-rank learning of this segment, then the LAI of all segments was connected together on the time axis to obtain the LAI of the whole video, which effectively captured the action information in the video, so as to avoid cumbersome extraction steps and various assumptions. Secondly, according to the characteristics of LAI, an MCNN model was designed. In the model, the multi-scale convolution kernels were used to obtain the action characteristics of LAI under different receptive fields, and the reasonable design of each convolution layer, pooling layer and fully connected layer were utilized to further refine the characteristics and finally output the action categories. The performance of the proposed method was verified on two benchmark databases KTH and HMDB51, and three groups of comparison experiments were designed and carried out. Experimental results show that the recognition rates of the proposed method are 97.33% and 72.05% respectively on the two databases, which are at least increased by 0.67 and 1.15 percentage points respectively compared with those of the methods of Two-Fold Transformation (TFT) and Deep Temporal Embedding Network (DTEN). The proposed method can further promote the wide application of action recognition technology in security, human-computer interaction and other fields.

Key words: action recognition, low-rank learning, action information, multi-scale, Convolutional Neural Network (CNN)

中图分类号:

TP391.4

蒋丽, 黄仕建, 严文娟. 基于低秩行为信息和多尺度卷积神经网络的人体行为识别方法[J]. 计算机应用, 2021, 41(3): 721-726.

JIANG Li, HUANG Shijian, YAN Wenjuan. Human action recognition method based on low-rank action information and multi-scale convolutional neural network[J]. Journal of Computer Applications, 2021, 41(3): 721-726.

参考文献

[1] GU Y,YE X,SHENG W,et al. Multiple stream deep learning model for human action recognition[J]. Image and Vision Computing,2020,93:No. 103818.
[2] 郭明祥, 宋全军, 徐湛楠, 等. 基于三维残差稠密网络的人体行为识别算法[J]. 计算机应用,2019,39(12):3482-3489.(GUO M X,SONG Q J,XU Z N,et al. Human behavior recognition algorithm based on three-dimensional residual dense network[J]. Journal of Computer Applications,2019,39(12):3482-3489.)
[3] 杨锋, 许玉, 尹梦晓, 等. 基于深度学习的行人重识别综述[J]. 计算机应用,2020,40(5):1243-1252.(YANG F,XU Y,YIN M X,et al. Review on deep learning-based pedestrian re-identification[J]. Journal of Computer Applications,2020,40(5):1243-1252.)
[4] HAN H,LI X J. Human action recognition with sparse geometric features[J]. The Imaging Science Journal,2015,63(1):45-53.
[5] HOSHINO S, NⅡMURA K. Optical flow for real-time human detection and action recognition based on CNN classifiers[J]. Journal of Advanced Computational Intelligence and Intelligent Informatics,2019,23(4):735-742.
[6] TRIPATHI V,GANGODKAR D,MITTAL A,et al. Robust action recognition framework using segmented block and distance mean histogram of gradients approach[J]. Procedia Computer Science, 2017, 115:493-500.
[7] WANG H,KLÄSER A,SCHMID C,et al. Dense trajectories and motion boundary descriptors for action recognition[J]. International Journal of Computer Vision,2013,103(1):60-79.
[8] SIMONYAN K, ZISSERMAN A. Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,2014:568-576.
[9] WANG L, XIONG Y, WANG Z, et al. Temporal segment networks:towards good practices for deep action recognition[C]//Proceedings of the 2016 European Conference on Computer Vision, LNCS 9912. Cham:Springer,2016:20-36.
[10] DU W,WANG Y,QIAO Y. RPAN:an end-to-end recurrent poseattention network for action recognition in videos[C]//Proceedings of the 2017 IEEE Conference on Computer Visio. Piscataway:IEEE,2017:3745-3754.
[11] TRAN D, BOURDEV L, FERGUS R, et al. Learning spatiotemporal features with 3D convolutional networks[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway IEEE,2015:4489-4497.
[12] KIM Y J,LEE D G,LEE S W. Three-stream fusion network for first-person interaction recognition[J]. Pattern Recognition, 2020,103:No. 107279.
[13] 黄仕建. 视频序列中人体行为的低秩表达与识别方法研究[D]. 重庆:重庆大学,2015:39-53.(HUANG S J. Research on low-rank presentation and recognition of human actions in video sequences[D]. Chongqing:Chongqing University, 2015:39-53.)
[14] SCHULDT C, LAPTEV I, CAPUTO B. Recognizing human actions:a local SVM approach[C]//Proceedings of the 17th International Conference on Pattern Recognition. Piscataway:IEEE,2004:32-36.
[15] KUEHNE H,JHUANG H,GARROTE E,et al. HMDB:a large video database for human motion recognition[C]//Proceedings of the 2011 IEEE International Conference on Computer Vision. Piscataway:IEEE,2011:2556-2563.
[16] GUO Z,WANG X,WANG B,et al. A novel 3D gradient LBP descriptor for action recognition[J]. IEICE Transactions on Information and Systems,2017,E100-D (6):1388-1392.
[17] NAZIR S,YOUSAF M H,VELASTIN S A. Evaluating a bag-ofvisual features approach using spatio-temporal features for action recognition[J]. Computers and Electrical Engineering,2018,72:660-669.
[18] HUAN R,XIE C,GUO F,et al. Human action recognition based on HOIRM feature fusion and AP clustering BOW[J]. PLoS One, 2019,14(7):No. 0219910.
[19] KAPOOR R,MISHRA O,TRIPATHI M M,et al. Human action recognition using descriptor based on selective finite element analysis[J]. Journal of Electrical Engineering,2019,70(6):443-453.
[20] JAOUEDI N,BOUJNAH N,BOUHLEL M S. A new hybrid deep learning model for human action recognition[J]. Journal of King Saud University-Computer and Information Sciences,2020,32(4):447-453.
[21] VISHWAKARMA D K. A two-fold transformation model for human action recognition using decisive pose[J]. Cognitive Systems Research,2020,61:1-13.
[22] DUTA I C,UIJLINGS J R R,IONESCU B,et al. Efficient human action recognition using histograms of motion gradients and VLAD with descriptor shape information[J]. Multimedia Tools and Applications,2017,76(21):22445-22472.
[23] ZHANG H, XIN M, WANG S, et al. End-to-end temporal attention extraction and human action recognition[J]. Machine Vision and Applications,2018,29(7):1127-1142.
[24] LIU Z,ZHANG X,SONG L,et al. More efficient and effective tricks for deep action recognition[J]. Cluster Computing,2019, 22(S1):819-826.
[25] WANG D,XIAO H,OU F,et al. Moving human focus inference model for action recognition[C]//Proceedings of the 45th Annual Conference of the IEEE Industrial Electronics Society. Piscataway:IEEE,2019:2554-2559.
[26] MAJD M,SAFABAKHSH R. Correlational convolutional LSTM for human action recognition[J]. Neurocomputing,2020,396:224-229.
[27] KOOHZADI M,CHARKARI N M. A context based deep temporal embedding network in action recognition[J]. Neural Processing Letters,2020,52(1):187-220.

基于低秩行为信息和多尺度卷积神经网络的人体行为识别方法

Human action recognition method based on low-rank action information and multi-scale convolutional neural network

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	王贺兵, 张春梅. 基于非对称卷积-压缩激发-次代残差网络的人脸关键点检测[J]. 计算机应用, 2021, 41(9): 2741-2747.
[2]	宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.
[3]	李康康, 张静. 基于注意力机制的多层次编码和解码的图像描述模型[J]. 计算机应用, 2021, 41(9): 2504-2509.
[4]	张永斌, 常文欣, 孙连山, 张航. 基于字典的域名生成算法生成域名的检测方法[J]. 计算机应用, 2021, 41(9): 2609-2614.
[5]	赵宏, 孔东一. 图像特征注意力与自适应注意力融合的图像内容中文描述[J]. 计算机应用, 2021, 41(9): 2496-2503.
[6]	徐江浪, 李林燕, 万新军, 胡伏原. 结合目标检测的室内场景识别方法[J]. 计算机应用, 2021, 41(9): 2720-2725.
[7]	牟长宁, 王海鹏, 周丕宇, 侯鑫行. 基于图卷积神经网络的串联质谱从头测序[J]. 计算机应用, 2021, 41(9): 2773-2779.
[8]	严海升, 马新强. 基于径向基函数的多目标回归特征构建算法[J]. 计算机应用, 2021, 41(8): 2219-2224.
[9]	曾祥银, 郑伯川, 刘丹. 基于深度卷积神经网络和聚类的左右轨道线检测[J]. 计算机应用, 2021, 41(8): 2324-2329.
[10]	曹玉红, 徐海, 刘荪傲, 王紫霄, 李宏亮. 基于深度学习的医学影像分割研究综述[J]. 计算机应用, 2021, 41(8): 2273-2287.
[11]	秦斌斌, 彭良康, 卢向明, 钱江波. 司机分心驾驶检测研究进展[J]. 计算机应用, 2021, 41(8): 2330-2337.
[12]	黄程程, 董霄霄, 李钊. 基于二维Winograd算法的深流水线5×5卷积方法[J]. 计算机应用, 2021, 41(8): 2258-2264.
[13]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.
[14]	武光利, 李雷霆, 郭振洲, 王成祥. 基于改进的双向长短期记忆网络的视频摘要生成模型[J]. 计算机应用, 2021, 41(7): 1908-1914.
[15]	吴则举, 焦翠娟, 陈亮. 基于改进Faster R-CNN的轮胎缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1939-1946.