Action recognition method based on dense optical flow trajectory and sparse coding algorithm

doi:10.11772/j.issn.1001-9081.2016.01.0181

Abstract

Abstract: Focusing on the issue that the existing action feature extraction method achieves lower recognition rate, a novel unsupervised one for action recognition by combining Dense Optical Flow trajectory and Sparse Coding (DOF-SC) algorithm was proposed. First of all, the trajectory-centered image patches were sampled as the original features based on the extraction of Dense Optical Flow (DOF). Then, the sparse dictionary on the basis of Sparse Coding (SC) framework was trained, and the sparse feature representation of the trajectory through dictionary was got, then the code book of the trajectory by clustering with the Bag-of-Feature (BF) model was achieved, and the trajectory of every action by code book was voted, the action features by counting the number of every code book were got. Finally, the examples for action recognition was classified and predicted by Support Vector Machine (SVM) with the kernel of histogram intersection function. The accuracy of the DOF-SC algorithm is superior to the accuracy of Motion Boundary Histogram (MBH) as the action feature by 0.9% in the KTH (Kungliga Tekniska Hgskolan) database and 1.2% in the YouTube database with the trajectory sampling rate of 10%. The results prove the effectiveness of the unsupervised action feature extraction method.

Key words: action recognition, Sparse Coding (SC), Dense Optical Flow (DOF), Bag-of-Features (BF) model, Support Vector Machine (SVM)

摘要： 针对现有行为特征提取方法识别率低的问题,提出了一种融合稠密光流轨迹和稀疏编码框架的无监督行为特征提取方法(DOF-SC)。首先,在稠密光流(DOF)轨迹提取的基础上,对以轨迹为中心的原始图像块进行采样作为轨迹的原始特征;其次,对轨迹原始特征基于稀疏编码框架训练稀疏字典,得到轨迹的稀疏特征表示,利用词袋(BF)模型对稀疏特征聚类得到轨迹的码书,再根据码书对每个动作中出现的所有轨迹所属的码书类别进行投票,统计该动作中每个码书出现的次数,得到行为特征;最后,对行为特征利用基于直方图交叉核函数的支持向量机(SVM)进行训练得到行为识别模型,再利用该模型对行为进行分类预测,得到最终行为识别的结果。在对轨迹采样10%的情况下,DOF-SC算法得到的行为识别准确率在KTH数据库上高出采用运动边界直方图(MBH)作为特征的行为识别准确率的0.9%,在YouTube数据库上高出MBH作为特征的行为识别准确率的1.2%。实验数据表明了所提方法对行为识别的有效性。

关键词: 行为识别, 稀疏编码, 稠密光流, 词袋模型, 支持向量机

CLC Number:

TP181

ZHAO Xiaojian, ZENG Xiaoqin. Action recognition method based on dense optical flow trajectory and sparse coding algorithm[J]. Journal of Computer Applications, 2016, 36(1): 181-187.

赵晓健, 曾晓勤. 基于稠密光流轨迹和稀疏编码算法的行为识别方法[J]. 计算机应用, 2016, 36(1): 181-187.

References

[1] WEINLAND D, RONFARD R, BOYER E. A survey of vision-based methods for action representation, segmentation and recognition [J]. Computer vision and image understanding, 2011, 115(2): 224-241.
[2] NIEBLES J C, WANG H, LI F. Unsupervised learning of human action categories using spatial-temporal words [J]. International journal of computer vision, 2008, 79(3): 299-318.
[3] LAPTEV I. On space-time interest points [J]. International journal of computer vision, 2005, 64(2/3): 107-123.
[4] WANG H, KLASER A, SCHMID C, et al. Action recognition by dense trajectories [C]// CVPR 2011: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition. Washington, D.C.: IEEE Computer Society, 2011: 3169-3176.
[5] OLSHAUSEN B A, FIELD D J. Emergence of simple-cell receptive field properties by learning a sparse code for natural images [J]. Nature, 1996, 381(6583): 607-609.
[6] OLSHAUSEN B A, FIELD D J. Sparse coding with an overcomplete basis set: a strategy employed by V1? [J]. Vision research, 1997, 37(23): 3311-3325.
[7] LAPTEV I, MARSZALEK M, SCHMID C, et al. Learning realistic human actions from movies [C]// CVPR 2008: Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition. Washington, D.C.: IEEE Computer Society, 2008: 1-8.
[8] GERÓNIMO D, LÓPEZ A, PONSA D, et al. Haar wavelets and edge orientation histograms for on-board pedestrian detection [C]// IbPRIA'07: Proceedings of the 3rd Iberian Conference on Pattern Recognition and Image Analysis. Berlin: Springer, 2007: 418-425.
[9] LOWE D G. Object recognition from local scale-invariant features [C]// Proceedings of the Seventh IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE, 1999: 1150-1157.
[10] BELONGIE S, MALIK J, PUZICHA J. Shape matching and object recognition using shape contexts [J]. IEEE transactions on pattern analysis and machine intelligence, 2002, 24(4): 509-522.
[11] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection [C]// CVPR 2005: Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Washington, D.C.: IEEE Computer Society, 2005: 886-893.
[12] ZHU G, XU C, GAO W, et al. Action recognition in broadcast tennis video using optical flow and support vector machine [C]// ECCV'06: Proceedings of the 2006 International Conference on Computer Vision in Human-Computer Interaction. Berlin: Springer, 2006: 89-98.
[13] DALAL N, TRIGGS B, SCHMID C. Human detection using oriented histograms of flow and appearance [C]// ECCV'06: Proceedings of the 2006 International Conference on Computer Vision in Human-Computer Interaction. Berlin: Springer, 2006: 428-441.
[14] FARNEBÄCK G. Two-frame motion estimation based on polynomial expansion [C]// Proceedings of the 13th Scandinavian Conference on Image Analysis. Berlin: Springer, 2003: 363-370.
[15] SHI J, TOMASI C. Good features to track [C]// CVPR'94: Proceedings of the 1994 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 1994: 593-600.
[16] SCHULDT C, LAPTEV I, CAPUTO B. Recognizing human ac-tions: a local SVM approach [C]// ICPR 2004: Proceedings of the 17th International Conference on Pattern Recognition. Piscataway, NJ: IEEE, 2004: 32-36.
[17] DELAITRE V, LAPTEV I, SIVIC J. Recognizing human actions in still images: a study of bag-of-features and part-based representations [C]// Proceedings of the 2010 British Machine Vision Conference. Durham: British Machine Vision Association, 2010, 97: 1-11.
[18] LIU J, LUO J, SHAH M. Recognizing realistic actions from videos "in the wild" [C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2009: 1996-2003.