计算机应用 ›› 2013, Vol. 33 ›› Issue (12): 3591-3595.

• 先进计算 • 上一篇    下一篇

适于进化算法的迭代式MapReduce框架

金伟健1,王春枝2   

  1. 1. 义乌工商职业技术学院 机电信息分院,浙江 义乌 322000;
    2. 湖北工业大学 计算机学院 ,武汉 430068
  • 收稿日期:2013-06-05 修回日期:2013-08-05 出版日期:2013-12-01 发布日期:2013-12-31
  • 通讯作者: 金伟健
  • 作者简介:金伟健(1982-),男,浙江义乌人,讲师,硕士研究生,主要研究方向:计算机网络安全、云计算;
    王春枝(1963-),女,湖北武汉人,教授,博士, CCF高级会员,主要研究方向:计算机网络协同教育、计算机信息安全。
  • 基金资助:
    国家自然科学基金面上项目资助

Iteration MapReduce framework for evolution algorithm

JIN WeijianWANG Chunzhi2   

  • Received:2013-06-05 Revised:2013-08-05 Online:2013-12-31 Published:2013-12-01
  • Contact: JIN Weijian

摘要: MapReduce模块化的编程大大降低了分布式算法的实现难度,但同时也限制了它的应用范围。介绍了MapReduce的基本结构及其实现迭代算法的缺陷,并针对基于MapReduce进化算法效率低下的问题,在对MapReduce的计算框架进行研究的基础上提出了一种适用于进化算法的迭代式MapReduce计算框架。描述了迭代式MapReduce计算框架的实现需求及其具体实现,提出并证明了异常机制的可行性,且在公有的Hadoop云计算平台上对提出的框架进行了验证。实验结果表明,基于迭代式MapReduce计算框架的并行遗传算法在算法的加速比上与基于MapReduce的并行遗传算法相比有较大的提高。

关键词: 云计算, MapReduce, 进化算法, 迭代, Hadoop

Abstract: Modular programming of MapReduce greatly simplifies the implementation difficulty of distributed programming; however, its application scope is limited. In view of that MapReduce cannot be used to solve iteration algorithm, a new iteration MapReduce framework was proposed for evolutionary algorithm based on the study of MapReduce framework. The basic structure of the MapReduce was introduced, and the defects in implementing iteration algorithm were pointed out. The realization requirements and implementation of the proposed MapReduce framework were introduced, and the feasibility of abnormal mechanism was proposed and verified. At last, the new MapReduce framework was verified on Hadoop. The experimental results show that the parallel genetic algorithm based on the iteration MapReduce framework has higher speedup than that of MapReduce framework.

Key words: cloud computing, MapReduce, evolutionary algorithm, iteration, Hadoop

中图分类号: