计算机应用 ›› 2011, Vol. 31 ›› Issue (02): 462-465.

• 数据库与数据挖掘 • 上一篇    下一篇

计算资源共享平台中非集中式的元数据管理

蒋炎华   

  1. 玉林师范学院
  • 收稿日期:2010-07-19 修回日期:2010-08-27 发布日期:2011-02-01 出版日期:2011-02-01
  • 通讯作者: 蒋炎华
  • 基金资助:
    玉林师范学院重点项目;广西教育厅科研项目

Decentralized approach for metadata management in computing resource sharing platform

hua yanjiang   

  • Received:2010-07-19 Revised:2010-08-27 Online:2011-02-01 Published:2011-02-01
  • Contact: hua yanjiang

摘要: 提出了计算资源共享平台中的一种非集中式元数据管理方法,它利用对等网络的方式把数据和元数据分散到网络上的其他节点,而不是在后台服务器端。该方法可以在平台运行数据密集型应用时支持大容量的并行工作机读写数据,同时具有随机的访问、灵活的访问粒度、支持高负载的并行读写的特点。运用了分布式哈希表技术,把大容量元数据划分成树型结构的分段树的形式。描述了数据与元数据的读写与追加新数据的过程。测试结果表明:该方法针对3D图像渲染类数据密集型应用,在不同的工作机并行访问与读写过程中,可以获得很高的聚集带宽与平均读写带宽。

关键词: 元数据, 计算资源共享, 非集中式, 数据密集应用, 分布式哈希表

Abstract: A decentralized approach for metadata management in computing resource sharing platform was proposed, which used peer network to distribute data and metadata to other nodes on the network instead of the back server. The random access patterns, variable access grains and potential heavy concurrency were supported in this system. Scalability under heavy concurrency was also achieved, which was supported by an original metadata scheme using a distributed segment tree built on top of a Distributed Hash Table (DHT). In order to validate our approach, a serial of simulation experiments were conducted. The experimental results show the aggregated bandwidth always increases when adding metadata servers. A wide average client bandwidth was also obtained when workers generate and write output data simultaneously.

Key words: metadata, computing resource sharing, decentralized approach, data-intensive application, Distributed Hash Table (DHT)