MapReduce Based Image Classification Approach

doi:10.11772/j.issn.1001-9081.2014.06.1600

Journal of Computer Applications ›› 2014, Vol. 34 ›› Issue (6): 1600-1603.DOI: 10.11772/j.issn.1001-9081.2014.06.1600

Previous Articles Next Articles

MapReduce Based Image Classification Approach

WEI Han,ZHANG Xueqing,CHEN Yang

The 54th Research Institute, China Electronics Technology Group Corporation, Shijiazhuang Hebei 050081,China

Received:2013-12-03 Revised:2014-02-19 Online:2014-07-02 Published:2014-06-01
Contact: WEI Han

基于MapReduce的图像分类方法

韩伟,张学庆,陈旸

中国电子科技集团公司第五十四研究所, 石家庄 050081

通讯作者: 韩伟
作者简介:韩伟(1977-),男,山东黄岛人,博士,主要研究方向:云计算、大数据分析;张学庆(1968-),男,河北定兴人,研究员,主要研究方向:航天地面应用、云计算;陈旸(1971-),男,河北石家庄人,高级工程师,主要研究方向:综合电子信息系统。
基金资助:
河北省高层次人才资助项目

Abstract

Abstract:

Many existing image classification algorithms cannot be used for big image data. A new approach was proposed to accelerate big image classification based on MapReduce. The whole image classification process was reconstructed to fit the MapReduce programming model. First, the Scale Invariant Feature Transform (SIFT) feature was extracted by MapReduce, then it was converted to sparse vector using sparse coding to get the sparse feature of the image. The MapReduce was also used to distributed training of random forest, and on the basis of it, the big image classification was achieved parallel. The MapReduce based algorithm was evaluated on a Hadoop cluster. The experimental results show that the proposed approach can classify images simultaneously on Hadoop cluster with a good speedup rate.

摘要：

针对现有的方法不能有效用于图像大数据分类的问题,提出了一种基于MapReduce编程模型的图像分类方法,在分类的全过程利用MapReduce机制加速分类过程。首先,利用MapReduce机制实现对图像尺度不变特征变换(SIFT)特征的分布式提取,并通过稀疏编码将其转换为稀疏向量,生成图像的稀疏特征;然后,利用MapReduce机制实现对随机森林的分布式训练;在此基础上,利用MapReduce机制对图像集实现基于随机森林方法的并行分类。通过在Hadoop平台的实验结果表明,该方法能够充分利用MapReduce框架的分布式特性,对大规模图像数据实现快速准确分类。

CLC Number:

WEI Han ZHANG Xueqing CHEN Yang. MapReduce Based Image Classification Approach[J]. Journal of Computer Applications, 2014, 34(6): 1600-1603.

韩伟张学庆陈旸. 基于MapReduce的图像分类方法[J]. 计算机应用, 2014, 34(6): 1600-1603.

References

[1]MIKOLAJCZYK K, SCHMID C. A performance evaluation of local descriptors [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 1615-1630.
[2]CORTES C, VAPNIK V. Support-vector networks [J]. Machine Learning, 1995, 20(3): 273-297.
[3]FOODY G M, MATHUR A. A relative evaluation of multiclass image classification by support vector machines [J]. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(6): 1335-1343.
[4]BARCZAK A L C, JOHNSON M J, MESSOM C H. Empirical evaluation of a new structure for AdaBoost[C]// Proceedings of the 2008 ACM symposium on Applied computing. New York: ACM Press, 2008: 1764-1765.
[5]DEAN J, GHEMAWAT S. MapReduce: Simplified data processing on large clusters [C]// Proceedings of the 2004 OSDI. Berkeley: USENIX Association, 2004: 137-150.
[6]OLSHAUSEN B A, FIELD D J. Sparse coding with and overcomplete basis set: a strategy employed by V1? [J]. Vision Research, 1997,37(23):3311-3325.
[7]BREIMAN L. Random forests [J]. Machine Learning, 2001, 45(1): 5-32.
[8]WHITE T. Hadoop: the definitive guide [M]. 3rd ed. Sebastopol: O'Reilly Media, 2012.
[9]LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[10]LEE H, BATTLE A, RAINA R, NG A Y. Efficient sparse coding algorithms [C]// Proceedings of the 2006 Advances in Neural Information Processing System. Cambridge: MIT Press, 2007: 801-808.
[11]YANG J, YU K, GONG Y, et al. Linear spatial pyramid matching using sparse coding for image classification [C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2009:1794-1801.
[12]MAIRAL J, BACH F, PONCE J, et al. Online learning for matrix factorization and sparse coding [J]. Journal of Machine Learning Research, 2010, 11(1): 19-60.
[13]EFRON B, HASTIE T, JOHNSTONE I, et al. Least angle regression[J]. Annals of Statistics, 2004, 32(2): 407-409.
[14]CARUANA R, NICULESCU-MIZIL A. An empirical comparison of supervised learning algorithms [C]// Proceedings of the 23rd International Conference on Machine Learning. New York: ACM Press, 2006:161-168.
[15]LI F, ANDREETTO M, RANZATO M A. The caltech-101 object categories [EB/OL]. [2013-05-11].http://www.vision.caltech.edu/feifeili/Datasets.htm.
[16]LI F, FERGUS R, PERONA P. Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories [J]. Computer Vision and Image Understanding, 2007, 106(1): 59-70.

MapReduce Based Image Classification Approach

基于MapReduce的图像分类方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	. Pavement defect detection algorithm with enhanced morphological perception [J]. Journal of Computer Applications, 0, (): 0-0.
[2]	YANG Jianfeng, CHEN Bin, LI Yuxuan. Self-supervised point cloud anomaly detection method based on point cloud reconstruction [J]. Journal of Computer Applications, 0, (): 0-0.
[3]	. Binocular vision object location algorithm for robot arm grasping [J]. Journal of Computer Applications, 0, (): 0-0.
[4]	. Multi-attention contrastive learning for infrared small target detection [J]. Journal of Computer Applications, 0, (): 0-0.
[5]	LI Zhonghua, ZHONG Gengxin, FAN Ping, ZHU Hengliang. Camouflaged object detection by boundary mining and background guidance [J]. Journal of Computer Applications, 0, (): 0-0.
[6]	. Multi-view difficult airway recognition based on discriminant region guidance [J]. Journal of Computer Applications, 0, (): 0-0.
[7]	. Privacy preserving localization of surveillance images based on large vision models [J]. Journal of Computer Applications, 0, (): 0-0.
[8]	. Data augmentation method for abnormal passenger behavior in elevators based on dynamic graph convolutional network [J]. Journal of Computer Applications, 0, (): 0-0.
[9]	. Text-based person retrieval method based on multi-granularity shared semantic center association [J]. Journal of Computer Applications, 0, (): 0-0.
[10]	. Distribution adaptation and dynamic curriculum pseudo-labeling framework for semi-supervised fire detection [J]. Journal of Computer Applications, 0, (): 0-0.
[11]	. Image Caption Method Based on Swin-Transformer and Multi-Scale Feature Fusion [J]. Journal of Computer Applications, 0, (): 0-0.
[12]	Keyi FU, Gaocai WANG, Man WU. Few-shot object detection method based on improved region proposal network and feature aggregation [J]. Journal of Computer Applications, 2024, 44(12): 3790-3797.
[13]	Chenghao YANG, Jie HU, Hongjun WANG, Bo PENG. Incomplete multi-view clustering algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(12): 3784-3789.
[14]	Peng FANG, Fan ZHAO, Baoquan WANG, Yi WANG, Tonghai JIANG. Development， technologies and applications of blockchain 3.0 [J]. Journal of Computer Applications, 2024, 44(12): 3647-3657.
[15]	Yudong PANG, Zhixing LI, Weijie LIU, Tianhao LI, Ningning WANG. Small target detection model in overlooking scenes on tower cranes based on improved real-time detection Transformer [J]. Journal of Computer Applications, 2024, 44(12): 3922-3929.