Continuous action segmentation and recognition based on sliding window and dynamic programming

doi:10.11772/j.issn.1001-9081.2018061344

Journal of Computer Applications ›› 2019, Vol. 39 ›› Issue (2): 348-353.DOI: 10.11772/j.issn.1001-9081.2018061344

Previous Articles Next Articles

Continuous action segmentation and recognition based on sliding window and dynamic programming

YANG Shiqiang, LUO Xiaoyu, QIAO Dan, LIU Peilei, LI Dexin

School of Mechanical and Precision Instrument Engineering, Xi'an University of Technology, Xi'an Shaanxi 710048, China

Received:2018-06-27 Revised:2018-08-23 Online:2019-02-10 Published:2019-02-15
Supported by:
This work is partially supported by the National Natural Science Foundation of China (54175365).

基于滑动窗口和动态规划的连续动作分割与识别

杨世强, 罗晓宇, 乔丹, 柳培蕾, 李德信

西安理工大学机械与精密仪器工程学院, 西安 710048

通讯作者: 罗晓宇
作者简介:杨世强(1973-),男,甘肃白银人,副教授,博士,主要研究方向:智能机器人控制、行为检测与识别;罗晓宇(1990-),男,山西朔州人,硕士研究生,主要研究方向:行为检测与识别;乔丹(1994-),男,河北邯郸人,硕士研究生,主要研究方向:图像检测、动作识别;柳培蕾(1993-),男,陕西宝鸡人,硕士研究生,主要研究方向:机电系统动力学分析;李德信(1965-),男,山东烟台人,副教授,博士,主要研究方向:机械计算机辅助设计/计算机辅助制造、机械动力学。
基金资助:
国家自然科学基金资助项目（54175365）。

Abstract

Abstract: Concerning the fact that there are few researches on continuous action recognition in the field of action recognition and single algorithms have poor effect on continuous action recognition, a segmentation and recognition method of continuous actions was proposed based on single motion modeling by combining sliding window method and dynamic programming method. Firstly, the single action model was constructed based on the Deep Belief Network and Hidden Markov Model (DBN-HMM). Secondly, the logarithmic likelihood value of the trained action model and the sliding window method were used to estimate the score of the continous action, detecting the initial segmentation points. Thirdly, the dynamic programming method was used to optimize the location of the segmentation points and identify the single action. Finally, the testing experiments of continuous action segmentation and recognition were conducted with an open action database MSR Action3D. The experimental results show that the dynamic programming based on sliding window can optimize the selection of segmentation points to improve the recognition accuracy, which can be used to recognize continuous action.

Key words: Hidden Markov Model (HMM), action segmentation, action recognition, sliding window, dynamic programming

摘要： 针对现有动作识别中对连续动作识别研究较少且单一算法对连续动作识别效果较差的问题，提出在单个动作建模的基础上，采用滑动窗口法和动态规划法结合，实现连续动作的分割与识别。首先，采用深度置信网络和隐马尔可夫结合的模型DBN-HMM对单个动作建模；其次，运用所训练动作模型的对数似然值和滑动窗口法对连续动作进行评分估计，实现初始分割点的检测；然后，采用动态规划对分割点位置进行优化并对单个动作进行识别。在公开动作数据库MSR Action3D上进行连续动作分割与识别测试，结果表明基于滑动窗口的动态规划能够优化分割点的选取，进而提高识别精度，能够用于连续动作识别。

关键词: 隐马尔可夫模型, 动作分割, 动作识别, 滑动窗口, 动态规划

CLC Number:

TP391.4

YANG Shiqiang, LUO Xiaoyu, QIAO Dan, LIU Peilei, LI Dexin. Continuous action segmentation and recognition based on sliding window and dynamic programming[J]. Journal of Computer Applications, 2019, 39(2): 348-353.

杨世强, 罗晓宇, 乔丹, 柳培蕾, 李德信. 基于滑动窗口和动态规划的连续动作分割与识别[J]. 计算机应用, 2019, 39(2): 348-353.

References

[1] 胡琼,秦磊,黄庆明.基于视觉的人体动作识别综述[J].计算机学报,2013,36(12):2512-2524. (HU Q, QIN L, HUANG Q M. A survey on visual human action recognition[J]. Chinese Journal of Computers, 2013, 36(12):2512-2524.)
[2] AGGARWAL J K, RYOO M S. Human activity analysis:a review[J]. ACM Computing Surveys, 2011, 43(3):Article No. 16.
[3] KOPPULA H S, SAXENA A. Anticipating human activities using object affordances for reactive robotic response[J]. IEEE Transactions on Pattern analysis and Machine Intelligence, 2015, 38(1):1-14.
[4] ZHANG C, TIAN Y. RGB-D camera-based daily living activity recognition[J]. Journal of Computer Vision and Image Processing, 2012, 2(4):1-7.
[5] 白栋天,张磊,黄华.RGB-D视频中连续动作识别[J].中国科技论文,2016(2):168-172. (BAI D T, ZHANG L,HUANG H. Recognition continuous human actions from RGB-D videos[J]. China Science Paper, 2016(2):168-172.)
[6] DARRELL T, PENTLAND A. Space-time gestures[C]//Proceedings of the 1993 IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 1993:335-340.
[7] OKA R. Spotting method for classification of real world data[J]. Computer Journal, 1998, 41(8):559-565.
[8] GONG D, MEDIONI G, ZHAO X. Structured time series analysis for human action segmentation and recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(7):1414-1427.
[9] ZHU G, ZHANG L, SHEN P, et al. An online continuous human action recognition algorithm based on the Kinect sensor[J]. Sensors, 2016, 16(2):161-179.
[10] LEI J, LI G, ZHANG J, et al. Continuous action segmentation and recognition using hybrid convolutional neural network-hidden Markov model model[J]. IET Computer Vision, 2016, 10(6):537-544.
[11] KULKARNI K, EVANGELIDIS G, CECH J, et al. Continuous action recognition based on sequence alignment[J]. International Journal of Computer Vision, 2015, 112(1):90-114.
[12] EVANGELIDIS G D, SINGH G, HORAUD R. Continuous gesture recognition from articulated poses[C]//Proceedings of the 2014 European Conference on Computer Vision. Cham:Springer, 2014:595-607.
[13] SONG Y, GU Y, WANG P, et al. A Kinect based gesture recognition algorithm using GMM and HMM[C]//Proceedings of the 20136th International Conference on Biomedical Engineering and Informatics. Piscataway, NJ:IEEE, 2013:750-754.
[14] VITERBI A J. Error bounds for convolutional codes and an asymptotically optimum decoding algorithm[J]. IEEE Transactions on Information Theory, 1967, 13(2):260-269.
[15] TAYLOR G W, HINTON G E, ROWEIS S. Modeling human motion using binary latent variables[C]//Proceedings of the 19th International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 2007:1345-1352.
[16] HINTON G E, SIMON O, TEH Y W, et al. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2014, 18(7):1527-1554.
[17] LI W, ZHANG Z, LIU Z. Action recognition based on a bag of 3D points[C]//Proceedings of the 2010 IEEE Computer Vision and Pattern Recognition Workshops. Washington, DC:IEEE Computer Society, 2010:9-14.

Continuous action segmentation and recognition based on sliding window and dynamic programming

基于滑动窗口和动态规划的连续动作分割与识别

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	JING Xinghong, SUN Guodong, HE Shibiao, LIAO Yong. Time-varying channel estimation method based on sliding window filtering and polynomial fitting [J]. Journal of Computer Applications, 2021, 41(9): 2699-2704.
[2]	LI Yangzhi, YUAN Jiazheng, LIU Hongzhe. Human skeleton-based action recognition algorithm based on spatiotemporal attention graph convolutional network model [J]. Journal of Computer Applications, 2021, 41(7): 1915-1921.
[3]	GUO Tianxiao, HU Qingrui, LI Jianwei, SHEN Yanfei. Fitness action recognition method based on human skeleton feature encoding [J]. Journal of Computer Applications, 2021, 41(5): 1458-1464.
[4]	JIANG Kun, LIU Zheng, ZHU Lei, LI Xiaoxing. Fixed word-aligned partition compression algorithm of inverted list based on directed acyclic graph [J]. Journal of Computer Applications, 2021, 41(3): 727-732.
[5]	JIANG Li, HUANG Shijian, YAN Wenjuan. Human action recognition method based on low-rank action information and multi-scale convolutional neural network [J]. Journal of Computer Applications, 2021, 41(3): 721-726.
[6]	HUO Weigang, WANG Huifang. Time series anomaly detection method based on autoencoder and HMM [J]. Journal of Computer Applications, 2020, 40(5): 1329-1334.
[7]	QIU Yuan, Chang Xiangmao, QIU Qian, PENG Cheng, SU Shanting. Stream data anomaly detection method based on long short-term memory network and sliding window [J]. Journal of Computer Applications, 2020, 40(5): 1335-1339.
[8]	ZENG Zhiyang, CHEN Yan, WANG Ke. Design and implementation of parallel genetic algorithm for cutting stock of circular parts [J]. Journal of Computer Applications, 2020, 40(2): 392-397.
[9]	LI Qian, YANG Wenzhu, CHEN Xiangyang, YUAN Tongtong, WANG Yuxia. Human action recognition model based on tightly coupled spatiotemporal two-stream convolution neural network [J]. Journal of Computer Applications, 2020, 40(11): 3178-3183.
[10]	LI Kunming, WANG Chaoqian, NI Weiwei, BAO Xiaohan. High-precision histogram publishing method based on differential privacy [J]. Journal of Computer Applications, 2020, 40(11): 3242-3248.
[11]	WANG Ping, PANG Wenhao. Two-stream CNN for action recognition based on video segmentation [J]. Journal of Computer Applications, 2019, 39(7): 2081-2086.
[12]	SHI Wenxu, YANG Yang, BAO Shengli. Greedy core acceleration dynamic programming algorithm for solving discounted {0-1} knapsack problem [J]. Journal of Computer Applications, 2019, 39(7): 1912-1917.
[13]	WANG Jince, DENG Yueping, SHI Ming, ZHOU Yunfei. Time series trend prediction at multiple time scales [J]. Journal of Computer Applications, 2019, 39(4): 1046-1052.
[14]	JI Xiaofei, QIN Linlin, WANG Yangyang. Human interaction recognition based on RGB and skeleton data fusion model [J]. Journal of Computer Applications, 2019, 39(11): 3349-3354.
[15]	WANG Jixiang, GUO Yi, QI Tianmei, WANG Zhihong, LI Zhen, TANG Minwei. RMB exchange rate forecast embedded with Internet public opinion intensity [J]. Journal of Computer Applications, 2019, 39(11): 3403-3408.