计算机应用 ›› 2012, Vol. 32 ›› Issue (04): 1009-1012.DOI: 10.3724/SP.J.1087.2012.01009

• 先进计算 • 上一篇    下一篇

适应大规模数据处理的动态服务私有云系统

汪竹1,梅林2,李磊2,赵太银1,胡光岷1   

  1. 1. 电子科技大学 光纤传感与通信教育部重点实验室,成都 611731
    2. 川庆钻探工程有限公司 地球物理勘探公司,成都 610213
  • 收稿日期:2011-09-14 修回日期:2011-11-16 发布日期:2012-04-20 出版日期:2012-04-01
  • 通讯作者: 汪竹
  • 基金资助:
    新世纪优秀人才计划支持项目

Private cloud computing system based on dynamic service adaptable to

WANG Zhu1,MEI Lin2,LI Lei2,ZHAO Tai-yin1,HU Guang-min1   

  1. 1. Key Laboratory of Optical Fiber Sensing and Communication Ministry of Education,University of Electronic Science and Technology of China, Chengdu Sichuan 611731, China
    2. Geophysical Exploration Company, Chuanqing Drilling Engineering Company Limited, Chengdu Sichuan 610213, China
  • Received:2011-09-14 Revised:2011-11-16 Online:2012-04-20 Published:2012-04-01
  • Contact: WANG Zhu

摘要: 为适应私有云环境下数据量大、计算密集、流程复杂的计算任务需求,借鉴公有云计算的相关理论与技术,结合私有云环境的特点,提出了一种适应大规模数据处理的动态服务私有云系统实现方案。该方案使用作业文件描述计算任务,以作业逻辑结构动态构建处理工作流程;通过数据流驱动服务请求,引入MapReduce并行框架进行大规模数据处理。实验结果表明:该方案能够正确有效地处理数据量大、计算密集、流程复杂的计算任务,显著提升处理效率,具有很高的实用性。

关键词: 云计算, 私有云计算, 数据流驱动, 动态服务, 并行处理

Abstract: In order to deal with problem in private cloud environment caused by computing tasks with large amount of data, intensive computing and complex processing, an implementation of private cloud system based on dynamic service was proposed on the basis of public cloud computing and the characteristics of private cloud environment, which was able to adapt large-scale data processing. In this implementation, computing tasks were described by job files, processing workflows were constructed dynamically by job logic, service requests were driven by data streams and the large-scale data processing could be reflected more efficiently in MapReduce parallel framework. The experimental results show that this implementation offers a high practical value, can deal with computing tasks with large amount of data, intensive computing and complex processing correctly and efficiently.

Key words: cloud computing, private cloud computing, data-flow driven, dynamic service, parallel computing