Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (2): 547-552.DOI: 10.11772/j.issn.1001-9081.2019101732

• CCF Bigdata 2019 • Previous Articles     Next Articles

Design and implementation of cloud native massive data storage system based on Kubernetes

Fuxin LIU(), Jingwei LI, Yihong WANG, Lin LI   

  1. School of Computer Science and Technology,Wuhan University of Technology,Wuhan Hubei 430070,China
  • Received:2019-08-30 Revised:2019-10-14 Accepted:2019-10-18 Online:2019-11-18 Published:2020-02-10
  • Contact: Fuxin LIU
  • About author:LI Jingwei, born in 1999. His research interests include machine learning, data mining.
    WANG Yihong, born in 1999. His research interests include algorithm optimization and its applications.
    LI Lin, born in 1977. Ph. D., professor. Her research interests include information retrieval, recommendation system.
  • Supported by:
    the National Innovation and Entrepreneurship Training Program for Undergraduates(20181049710013)


刘福鑫(), 李劲巍, 王熠弘, 李琳   

  1. 武汉理工大学 计算机科学与技术学院,武汉 430070
  • 通讯作者: 刘福鑫
  • 作者简介:李劲巍(1999—),男,浙江松阳人,主要研究方向:机器学习、数据挖掘
  • 基金资助:


Aiming at the sharp increasing of data on the cloud caused by the development and popularization of cloud native technology as well as the bottlenecks of the technology in performance and stability, a Haystack-based storage system was proposed. With the optimization in service discovery, automatic fault tolerance and caching mechanism, the system is more suitable for cloud native business and meets the growing and high-frequent file storage and read/write requirements of the data acquisition, storage and analysis industries. The object storage model used by the system satisfies the massive file storage with high-frequency reads and writes. A simple and unified application interface is provided for business using the storage system, a file caching strategy is applied to improve the resource utilization, and the rich automated tool chain of Kubernetes is adopted to make this storage system easier to deploy, easier to expand, and more stable than other storage systems. Experimental results indicate that the proposed storage system has a certain performance and stability improvement compared with the current mainstream object storage and file systems in the situation of large-scale fragmented data storage with more reads than writes.

Key words: file system, object storage, cloud computing, container orchestration, cloud native business



关键词: 文件系统, 对象存储, 云计算, 容器编排, 云原生业务

CLC Number: