内存云分级存储架构下的数据迁移模型

doi:10.11772/j.issn.1001-9081.2015.12.3392

计算机应用 ›› 2015, Vol. 35 ›› Issue (12): 3392-3397.DOI: 10.11772/j.issn.1001-9081.2015.12.3392

内存云分级存储架构下的数据迁移模型

郭刚¹, 于炯^1,2, 鲁亮², 英昌甜², 尹路通¹

1. 新疆大学软件学院, 乌鲁木齐 830008;
2. 新疆大学信息科学与工程学院, 乌鲁木齐 830046

收稿日期:2015-06-08 修回日期:2015-08-10 出版日期:2015-12-10 发布日期:2015-12-10
通讯作者: 于炯(1964-),男,北京人,教授,博士生导师,博士,CCF高级会员,主要研究方向:网络安全、网格与分布式计算
作者简介:郭刚(1990-),男,山东枣庄人,硕士研究生,CCF会员,主要研究方向:云计算、数据迁移;鲁亮(1990-),男,新疆乌鲁木齐人,博士研究生,CCF会员,主要研究方向:云计算、分布式计算;英昌甜(1989-),女,新疆乌鲁木齐人,博士研究生,CCF会员,主要研究方向:云计算、存储节能;尹路通(1992-),男,河南驻马店人,硕士研究生,主要研究方向:大数据、数据挖掘、推荐系统。
基金资助:
国家自然科学基金资助项目(61462079,61363083,61262088)。

Data migration model based on RAMCloud hierarchical storage architecture

GUO Gang¹, YU Jiong^1,2, LU Liang², YING Changtian², YIN Lutong¹

1. School of Software, Xinjiang University, Urumqi Xinjiang 830008, China;
2. School of Information Science and Engineering, Xinjiang University, Urumqi Xinjiang 830046, China

Received:2015-06-08 Revised:2015-08-10 Online:2015-12-10 Published:2015-12-10

摘要/Abstract

摘要： 为了实现在线海量数据的高效存储与访问,在内存云分级存储架构下,提出一种基于数据重要性的迁移模型(MMDS)。首先,通过数据本身的大小、时间重要性、用户访问总量等因素对数据本身的重要性进行计算;其次,采用推荐系统中相似用户和PageRank算法中的重要性排名思想对数据的潜在价值进行评估,数据重要性和潜在价值共同决定了数据的重要程度;然后基于数据的重要性,设计了数据迁移机制。实验结果表明:该模型能够识别出数据的重要程度并分级放置数据,相比最近最少使用(LRU)、最近最不常用(LFU)、基于价值评估的数据迁移(MSDV)等算法,提高了存储系统的数据访问命中率。该模型能够缓解部分存储压力,数据访问性能也有了一定的提高。

关键词: 分级存储, 内存云, 数据重要性, 数据迁移, 潜在价值

Abstract: In order to achieve the efficient storage and access to the huge amounts of data online, under the hierarchical storage architecture of memory cloud, a model of Migration Model based on Data Significance (MMDS) was proposed. Firstly, the importance of data itself was calculated based on factors of the size of the data itself, the importance of time, the total amount of user access, and so on. Secondly, the potential value of the data was evaluated by adopting users' similarity and the importance ranking of the PageRank algorithm in the recommendation system. The importance of the data was determined by the importance of data itself and its potential value together. Then, data migration mechanism was designed based on the importance of data, The experimental results show that, the proposed model can identify the importance of the data and place the data in a hierarchical way and improved the data access hit rate from the storage system compared with the algorithms of Least Recently Used (LRU), Least Frequently Used (LFU), Migration Strategy based on Data Value (MSDV). The proposed model can alleviate the part pressure of storage and has improved the data access performance.

Key words: Hierarchical Storage Management (HSM), RAMCloud, data importance, data migration, potential value

中图分类号:

TP393.02

郭刚, 于炯, 鲁亮, 英昌甜, 尹路通. 内存云分级存储架构下的数据迁移模型[J]. 计算机应用, 2015, 35(12): 3392-3397.

GUO Gang, YU Jiong, LU Liang, YING Changtian, YIN Lutong. Data migration model based on RAMCloud hierarchical storage architecture[J]. Journal of Computer Applications, 2015, 35(12): 3392-3397.

参考文献

[1] DEAN J, GHEMAWAT S. MapReduce:simplified data processing on large clusters[J]. Communications of the ACM, 2008, 51(1):107-113.
[2] ZHAO X, LI Z, ZENG L. A hierarchical storage strategy based on block-level data valuation[C]//Proceedings of the 20084th International Conference on Networked Computing and Advanced Information Management. Washington, DC:IEEE Computer Society, 2008, 1:36-41.
[3] OUSTERHOUT J, AGRAWAL P, ERICKSON D,et al. The case for RAMClouds:scalable high-performance storage entirely in DRAM[J]. ACM SIGOPS Operating Systems Review, 2010, 43(4):92-105.
[4] YING C,YU J,LU L, et al. Optimal storing strategy based on small files in RAMCloud[J]. Journal of Computer Applications, 2014, 34(11):3104-3108. (英昌甜,于炯,鲁亮,等.基于小文件的内存云存储优化策略[J].计算机应用,2014,34(11):3104-3108.)
[5] FENG Y, ZHANG Y. Research and application of data migration in storage area network performance optimization[J]. Computer Engineering, 2005, 31(7):43-45. (冯泳,张延园.数据迁移在SAN中性能优化的研究和应用[J].计算机工程,2005,31(7):43-45.)
[6] GIBSON J T. An improved long-term file usage prediction algorithm[C]//Proceedings of the 25th International Computer Management Group Conference. Turnersville:Computer Management Group, 1999:639-448.
[7] JEONG J, DUBOIS M. Cost-sensitive cache replacement algorithms[C]//Proceedings of the 9th International Symposium on High Performance Computer Architecture. Washington, DC:IEEE Computer Society, 2003:327-337.
[8] SMITHA A L N R. LRU-RED:an active queue management scheme to contain high bandwidth flows at congested routers[C]//GLOBECOM'01:Proceedings of the 2001 Global Telecommunications Conference. Piscataway:IEEE, 2001, 4:2311-2315.
[9] REED B, LONG D D E. Analysis of caching algorithms for distributed file systems[J]. ACM SIGOPS Operating Systems Review, 1996, 30(3):12-21.
[10] ARI I, GOTTWALS M, HENZE D. SANBoost:automated SAN-level caching in storage area networks[C]//Proceedings of the 13th International Conference on Autonomic Computing. Washington, DC:IEEE Computer Society, 2004:164-171.
[11] AO L, YU D, SHU J, et al. A tiered storage system for massive data:TH-TS[J]. Journal of Computer Research and Development, 2011, 48(6):1089-1100. (敖莉,于得水,舒继武,等.一种海量数据分级存储系统TH-TS[J].计算机研究与发展,2011,48(6):1089-1100.)
[12] BIAN G, WANG Y, SHAO B, et al. Data migration strategy research of huge amounts of data based on value evaluation[J]. Journal of Xi'an University of Architecture and Technology:Natural Science Edition, 2012, 44(3):441-445. (边根庆,王艳云,邵必林,等.基于价值评估的海量数据迁移策略研究[J].西安建筑科技大学学报:自然科学版,2012,44(3):441-445.)
[13] HUANG D, DU Y, HE Q. migration algorithm for big marine data in hybrid cloud storage[J]. Journal of Computer Research and Development, 2014, 51(1):199-205. (黄冬梅,杜艳玲,贺琪.混合云存储中海洋大数据迁移算法的研究[J].计算机研究与发展,2014,51(1):199-205.)
[14] LU L, YU J, YING C,et al. Energy-efficient strategy for disk in RAMCloud[J]. Journal of Computer Applications, 2014, 34(9):2518-2522. (鲁亮,于炯,英昌甜,等.内存云架构的磁盘节能策略[J].计算机应用,2014,34(9):2518-2522.)
[15] LUO Y, LUO S, GUAN J, et al.A RAMCloud storage system based on HDFS:architecture, implementation and evaluation[J]. Journal of Systems & Software, 2013, 86(3):744-750.
[16] ROSENBLUM M, OUSTERHOUT J K. The design and implementation of a log-structured file system[J]. ACM SIGOPS Operating Systems Review, 1991, 25(5):1-15.

内存云分级存储架构下的数据迁移模型

Data migration model based on RAMCloud hierarchical storage architecture

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

[1]	梁双, 周丽华, 杨培忠. 基于聚类分析分库策略的社交网络数据库查询性能与数据迁移[J]. 计算机应用, 2017, 37(3): 673-679.
[2]	王跃飞, 于炯, 鲁亮. 面向内存云的协调器选举策略[J]. 计算机应用, 2016, 36(9): 2402-2408.
[3]	褚征, 于炯, 鲁亮, 英昌甜, 卞琛, 王跃飞. 基于内存云的大块数据对象并行存取策略[J]. 计算机应用, 2016, 36(6): 1526-1532.
[4]	王跃飞, 于炯, 鲁亮. 面向内存云的数据块索引方法[J]. 计算机应用, 2016, 36(5): 1222-1227.
[5]	王雪飞. MP_WSMN数据融合权重μ-ξ的高QoS实现[J]. 计算机应用, 2010, 30(06): 1469-1471.
[6]	胡晓鹏，李晓航，李岗. 一种基于XML映射规则的数据迁移方法设计和实现[J]. 计算机应用, 2005, 25(08): 1849-1852.