基于MapReduce的图像分类方法

doi:10.11772/j.issn.1001-9081.2014.06.1600

计算机应用 ›› 2014, Vol. 34 ›› Issue (6): 1600-1603.DOI: 10.11772/j.issn.1001-9081.2014.06.1600

基于MapReduce的图像分类方法

韩伟,张学庆,陈旸

中国电子科技集团公司第五十四研究所, 石家庄 050081

收稿日期:2013-12-03 修回日期:2014-02-19 出版日期:2014-06-01 发布日期:2014-07-02
通讯作者: 韩伟
作者简介:韩伟(1977-),男,山东黄岛人,博士,主要研究方向:云计算、大数据分析;张学庆(1968-),男,河北定兴人,研究员,主要研究方向:航天地面应用、云计算;陈旸(1971-),男,河北石家庄人,高级工程师,主要研究方向:综合电子信息系统。
基金资助:
河北省高层次人才资助项目

MapReduce Based Image Classification Approach

WEI Han,ZHANG Xueqing,CHEN Yang

The 54th Research Institute, China Electronics Technology Group Corporation, Shijiazhuang Hebei 050081,China

Received:2013-12-03 Revised:2014-02-19 Online:2014-06-01 Published:2014-07-02
Contact: WEI Han

摘要/Abstract

摘要：

针对现有的方法不能有效用于图像大数据分类的问题,提出了一种基于MapReduce编程模型的图像分类方法,在分类的全过程利用MapReduce机制加速分类过程。首先,利用MapReduce机制实现对图像尺度不变特征变换(SIFT)特征的分布式提取,并通过稀疏编码将其转换为稀疏向量,生成图像的稀疏特征;然后,利用MapReduce机制实现对随机森林的分布式训练;在此基础上,利用MapReduce机制对图像集实现基于随机森林方法的并行分类。通过在Hadoop平台的实验结果表明,该方法能够充分利用MapReduce框架的分布式特性,对大规模图像数据实现快速准确分类。

Abstract:

Many existing image classification algorithms cannot be used for big image data. A new approach was proposed to accelerate big image classification based on MapReduce. The whole image classification process was reconstructed to fit the MapReduce programming model. First, the Scale Invariant Feature Transform (SIFT) feature was extracted by MapReduce, then it was converted to sparse vector using sparse coding to get the sparse feature of the image. The MapReduce was also used to distributed training of random forest, and on the basis of it, the big image classification was achieved parallel. The MapReduce based algorithm was evaluated on a Hadoop cluster. The experimental results show that the proposed approach can classify images simultaneously on Hadoop cluster with a good speedup rate.

中图分类号:

韩伟张学庆陈旸. 基于MapReduce的图像分类方法[J]. 计算机应用, 2014, 34(6): 1600-1603.

WEI Han ZHANG Xueqing CHEN Yang. MapReduce Based Image Classification Approach[J]. Journal of Computer Applications, 2014, 34(6): 1600-1603.

参考文献

[1]MIKOLAJCZYK K, SCHMID C. A performance evaluation of local descriptors [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 1615-1630.
[2]CORTES C, VAPNIK V. Support-vector networks [J]. Machine Learning, 1995, 20(3): 273-297.
[3]FOODY G M, MATHUR A. A relative evaluation of multiclass image classification by support vector machines [J]. IEEE Transactions on Geoscience and Remote Sensing, 2004, 42(6): 1335-1343.
[4]BARCZAK A L C, JOHNSON M J, MESSOM C H. Empirical evaluation of a new structure for AdaBoost[C]// Proceedings of the 2008 ACM symposium on Applied computing. New York: ACM Press, 2008: 1764-1765.
[5]DEAN J, GHEMAWAT S. MapReduce: Simplified data processing on large clusters [C]// Proceedings of the 2004 OSDI. Berkeley: USENIX Association, 2004: 137-150.
[6]OLSHAUSEN B A, FIELD D J. Sparse coding with and overcomplete basis set: a strategy employed by V1? [J]. Vision Research, 1997,37(23):3311-3325.
[7]BREIMAN L. Random forests [J]. Machine Learning, 2001, 45(1): 5-32.
[8]WHITE T. Hadoop: the definitive guide [M]. 3rd ed. Sebastopol: O'Reilly Media, 2012.
[9]LOWE D G. Distinctive image features from scale-invariant keypoints [J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[10]LEE H, BATTLE A, RAINA R, NG A Y. Efficient sparse coding algorithms [C]// Proceedings of the 2006 Advances in Neural Information Processing System. Cambridge: MIT Press, 2007: 801-808.
[11]YANG J, YU K, GONG Y, et al. Linear spatial pyramid matching using sparse coding for image classification [C]// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE Press, 2009:1794-1801.
[12]MAIRAL J, BACH F, PONCE J, et al. Online learning for matrix factorization and sparse coding [J]. Journal of Machine Learning Research, 2010, 11(1): 19-60.
[13]EFRON B, HASTIE T, JOHNSTONE I, et al. Least angle regression[J]. Annals of Statistics, 2004, 32(2): 407-409.
[14]CARUANA R, NICULESCU-MIZIL A. An empirical comparison of supervised learning algorithms [C]// Proceedings of the 23rd International Conference on Machine Learning. New York: ACM Press, 2006:161-168.
[15]LI F, ANDREETTO M, RANZATO M A. The caltech-101 object categories [EB/OL]. [2013-05-11].http://www.vision.caltech.edu/feifeili/Datasets.htm.
[16]LI F, FERGUS R, PERONA P. Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories [J]. Computer Vision and Image Understanding, 2007, 106(1): 59-70.

基于MapReduce的图像分类方法

MapReduce Based Image Classification Approach

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	罗圣钦陈金怡李洪均. 基于注意力机制的多尺度残差UNet实现乳腺癌灶分割[J]. 计算机应用, 0, (): 0-0.
[2]	杨鼎康黄帅王顺利翟鹏李一丹张立华. 基于对抗生成网络和网络集成的面部表情识别方法EE-GAN[J]. 计算机应用, 0, (): 0-0.
[3]	秦庭威赵鹏程秦品乐曾建朝柴锐黄永琦. 基于残差注意力机制的点云配准算法[J]. 计算机应用, 0, (): 0-0.
[4]	鲁永帅唐英杰马鑫然. 基于深度特征融合的无纺布低对比度浆丝缺陷检测方法[J]. 计算机应用, 0, (): 0-0.
[5]	裴仪瑶, 郭会明, 张丹普, 陈文博. 基于定位不确定性的鲁棒3D目标检测方法[J]. 计算机应用, 2021, 41(10): 2979-2984.
[6]	许学斌, 张佳达, 刘伟, 路龙宾, 赵雨晴. 融合空间和通道特征的高精度乳腺癌分类方法[J]. 计算机应用, 2021, 41(10): 3025-3032.
[7]	魏淳武, 赵涓涓, 唐笑先, 强彦. 基于多时期蒸馏网络的随访数据知识提取方法[J]. 计算机应用, 2021, 41(10): 2871-2878.
[8]	韩建栋, 李晓宇. 基于多尺度特征融合的行人重识别方法[J]. 计算机应用, 2021, 41(10): 2991-2996.
[9]	张凯悦, 张鸿. 基于注意力机制网络的航运监控图像识别模型[J]. 计算机应用, 2021, 41(10): 3010-3016.
[10]	李自强, 王正勇, 陈洪刚, 李林怡, 何小海. 基于外观和动作特征双预测模型的视频异常行为检测[J]. 计算机应用, 2021, 41(10): 2997-3003.
[11]	高世伟, 张长柱, 王祝萍. 基于可分离金字塔的轻量级实时语义分割算法[J]. 计算机应用, 2021, 41(10): 2937-2944.
[12]	卓禹心, 韩素雅, 张榆锋, 李支尧, 董毅峰. 基于超声谐波包络Nakagami参数图像的微波消融区域自动分割方法[J]. 计算机应用, 2021, 41(10): 3089-3096.
[13]	汪虹余, 张彧, 杨恒, 穆楠. 基于蚁群优化算法的弱光图像显著性目标检测[J]. 计算机应用, 2021, 41(10): 2970-2978.
[14]	李敬虎邢前国郑向阳李琳王丽丽. 基于深度学习的无人机影像夜光藻赤潮提取方法[J]. 计算机应用, 0, (): 0-0.
[15]	任炜白鹤翔. 基于全局与局部的多标签图像分类方法[J]. 计算机应用, 0, (): 0-0.