Performance optimization of distributed file system based on new type storage devices

doi:10.11772/j.issn.1001-9081.2020050632

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (12): 3594-3603.DOI: 10.11772/j.issn.1001-9081.2020050632

• Computer software technology • Previous Articles Next Articles

Performance optimization of distributed file system based on new type storage devices

DONG Cong^1,2, ZHANG Xiao^2,3, CHENG Wendi^2,3, SHI Jia^1,2

1. School of Software, Northwestern Polytechnical University, Xi'an Shaanxi 710129, China;
2. Key Laboratory of Big Data Storage and Management, Ministry of Industry and Information Technology(Northwestern Polytechnical University), Xi'an Shaanxi 710129, China;
3. College of Computer Science, Northwestern Polytechnical University, Xi'an Shaanxi 710129, China

Received:2020-05-13 Revised:2020-06-19 Online:2020-12-10 Published:2020-08-14
Supported by:
This work is partially supported by the National Key Research and Development Program of China （2018YFB1004400）， the Beijing Natural Science Foundation-Haidian Original Innovation Union Foundation （L192027）.

基于新型存储器件的分布式文件系统性能优化

董聪^1,2, 张晓^2,3, 程文迪^2,3, 石佳^1,2

1. 西北工业大学软件学院, 西安 710129;
2. 大数据存储与管理工业和信息化部重点实验室(西北工业大学), 西安 710129;
3. 西北工业大学计算机学院, 西安 710129

通讯作者: 张晓(1978-),男,河南南阳人,副教授,博士,CCF高级会员,主要研究方向:计算机存储系统、云计算、大数据存储与管理。zhangxiao@nwpu.edu.cn
作者简介:董聪(1996-),女,陕西西安人,硕士研究生,主要研究方向:分布式文件系统性能优化、大数据存储与管理、新型存储器件;程文迪(1995-),女,陕西咸阳人,硕士研究生,主要研究方向:分布式文件系统性能优化、非易失内存优化;石佳(1995-),男,陕西咸阳人,硕士研究生,主要研究方向:分布式文件系统性能优化、非易失内存优化读写流程
基金资助:
国家重点研发计划项目（2018YFB1004400）；北京市自然科学基金-海淀原始创新联合基金资助项目（L192027）。

Abstract

Abstract: The I/O performance of new type storage devices is usually an order of magnitude higher than that of traditional Solid State Disk (SSD). However, simply replacing SSD with new type storage device will not significantly improve the performance of distributed file system. This means that the current distributed file system cannot give full play to the performance of new type storage devices. To solve the problem, the data writing process and transmission process of Hadoop Distributed File System (HDFS) were analyzed quantitatively. Through quantitative analysis of the time consumptions of different stages of HDFS writing process, the most time-consuming data transmission between nodes was found in each stage of writing data. Therefore, the corresponding optimization strategy was proposed, that is, the processes of data transmission and processing were parallelized by using asynchronous write. So that the processing stages of different data packets were parallel to each other, shortening the total processing time of data writing, thereby the write performance of HDFS was improved. Experimental results show the proposed scheme improves the HDFS write throughput by 15%-24%, and reduces the overall write execution time by 28%-36%.

Key words: distributed file system, Hadoop Distributed File System (HDFS), non-volatile memory, performance optimization, asynchronous write

摘要： 新型存储器件的I/O性能通常比传统固态驱动器（SSD）高一个数量级，然而使用新型存储器件的分布式文件系统相对于使用SSD的分布式文件系统性能并没有显著的提高，这说明目前的分布式文件系统并不能充分发挥新型存储器件的性能。针对这个问题，对Hadoop分布式文件系统（HDFS）的数据写入流程及传输过程进行了量化分析。通过量化分析HDFS数据写入过程各阶段的时间开销，发现在写入数据的各个阶段中，节点间数据传输的时间占比较大。因此提出了对应的优化方案，通过异步写入的方式并行化数据传输与处理过程，使得不同数据包的处理阶段叠加起来，减少了数据包整体的处理时间，从而提升了HDFS的写入性能。实验结果表明，所提方案将HDFS的写入吞吐量提升了15%~24%，总体的写入执行时间降低了28%~36%。

关键词: 分布式文件系统, Hadoop分布式文件系统, 非易失性存储器, 性能优化, 异步写入

CLC Number:

TP311

DONG Cong, ZHANG Xiao, CHENG Wendi, SHI Jia. Performance optimization of distributed file system based on new type storage devices[J]. Journal of Computer Applications, 2020, 40(12): 3594-3603.

董聪, 张晓, 程文迪, 石佳. 基于新型存储器件的分布式文件系统性能优化[J]. 计算机应用, 2020, 40(12): 3594-3603.

References

[1] SHVACHKO K, KUANG H, RADIA S, et al. The Hadoop distributed file system[C]//Proceedings of the IEEE 26th Symposium on Mass Storage Systems and Technologies. Piscataway:IEEE,2010:8-15.
[2] 金国栋, 卞昊穹, 陈跃国, 等. HDFS存储和优化技术研究综述[J]. 软件学报, 2020, 31(1):137-161.(JIN G D,BIAN H Q, CHEN Y G,et al. Survey on storage and optimization techniques of HDFS[J]. Journal of Software,2020,31(1):137-161.)
[3] KRISH K R,ANWAR A,BUTT A R. hatS:a heterogeneity-aware tiered storage for Hadoop[C]//Proceedings of the 201414th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing. Piscataway:IEEE,2014:502-511.
[4] GUAN Y,MA Z,LI L. HDFS optimization strategy based on hierarchical storage of hot and cold data[J]. Procedia CIRP, 2019,83:415-418.
[5] ZHENG S G,LIU H,HUANG L G,et al. HMVFS:a versioning file system on DRAM/NVM hybrid memory[J]. Journal of Parallel and Distributed Computing,2018,120:355-368.
[6] WEI Q,WANG C,CHEN C,et al. Transactional NVM cache with high performance and crash consistency[C]//Proceedings of the 2017 International Conference for High Performance Computing, Networking, Storage and Analysis. New York:ACM, 2017:Article No. 56.
[7] CHEN Y,SHU J,OU J,et al. HiNFS:a persistent memory file system with both buffering and direct-access[J]. ACM Transactions on Storage,2018,14(1):Article No. 4.
[8] FAN Z,DU D H C,VOIGT D. H-ARC:a non-volatile memory based cache policy for solid state drives[C]//Proceedings of the 30th Symposium on Mass Storage Systems and Technologies. Piscataway:IEEE,2014:1-11.
[9] ISLAM N S, WASI-UR-RAHMAN M, LU X, et al. High performance design for HDFS with byte-addressability of NVM and RDMA[C]//Proceedings of the 2016 International Conference on Supercomputing,New York:ACM,2016:Article No. 8.
[10] DONG X, XU C, XIE Y, et al. NVSim:a circuit-level performance,energy,and area model for emerging nonvolatile memory[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems,2012,31(7):994-1007.
[11] HADY F T, FOONG A, VEAL B, et al. Platform storage performance with 3D XPoint technology[J]. Proceedings of the IEEE,2017,105(9):1822-1833.
[12] SON Y,KANG H,HAN H,et al. An empirical evaluation and analysis of the performance of NVM express solid state drive[J]. Cluster Computing,2016,19(3):1541-1553.
[13] 何小庆. 从Montavista Linux 5.0看嵌入式Linux的发展[J]. 电子产品世界, 2007(4):41-46,56.(HE X Q. Embedded Linux trend:form outlook of Montavista Linux 5.0[J]. Electronic Engineering & Product World,2007(4):41-46,56.)
[14] PARK M,LEE B,KIM K T,et al. Effective management of ReRAM-based hybrid SSD for multiple node HDFS[J]. International Journal of Networked and Distributed Computing, 2015,3(3):167-176.
[15] VILLA T, SANGIOVANNI-VINCENTELLI A. NOVA:state assignment of finite state machines for optimal two-level logic implementation[J]. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems,1990,9(9):905-924.
[16] GHEMAWAT S,GOBIOFF H,LEUNG S T,et al. The Google file system[C]//Proceedings of the 200319th ACM Symposium on Operating Systems Principles. New York:ACM,2003:29-43.
[17] DITTRICH J,QUIANÉ -RUIZ J A. Efficient big data processing in Hadoop MapReduce[J]. Proceedings of the VLDB Endowment,2012,5(12):2014-2015.
[18] MOHAMMADI A,REZVANI M H. A novel optimized approach for resource reservation in cloudcomputing using producerconsumer theory of microeconomics[J]. The Journal of Supercomputing,2019,75(11):7391-7425.
[19] 李建江, 崔健, 王聃, 等. MapReduce并行编程模型研究综述[J]. 电子学报, 2011, 39(11):2635-2642.(LI J J,CUI J, WANG D,et al. Survey of MapReduce parallel programming model[J]. Acta Electronica Sinica,2011,39(11):2635-2642.)

Performance optimization of distributed file system based on new type storage devices

基于新型存储器件的分布式文件系统性能优化

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	TANG Chen, HUANG Guorui, JIN Peiquan. NVM-LH: non-volatile memory-friendly linear hash index [J]. Journal of Computer Applications, 2021, 41(3): 623-629.
[2]	GOU Zi'an, ZHANG Xiao, WU Dongnan, WANG Yanqiu. Log analysis and workload characteristic extraction in distributed storage system [J]. Journal of Computer Applications, 2020, 40(9): 2586-2593.
[3]	HE Qingjian, CAI Tao, WANG Jie, NIU Dejiao. Fast file access system for NVM storage systems [J]. Journal of Computer Applications, 2020, 40(2): 541-546.
[4]	CAI Tao, CHEN Zhipeng, NIU Dejiao, WANG Jie, ZHAN Bisheng. New NVM storage system supporting high concurrent access [J]. Journal of Computer Applications, 2019, 39(1): 51-56.
[5]	LI Shuai, SUN Lei, GUO Songhui. Interrupt path optimization method of virtual cryptographic device with reducing context switching [J]. Journal of Computer Applications, 2018, 38(7): 1946-1950.
[6]	CHEN Bo, HE Lianyue, YAN Weiwei, XU Zhaomiao, XU Jun. Portable operating system interface of UNIX compatibility technology in mass small distributed file system [J]. Journal of Computer Applications, 2018, 38(5): 1389-1392.
[7]	LI Qiang, LIU Xiaofeng. Load balancing strategy of cloud storage based on Hopfield neural network [J]. Journal of Computer Applications, 2017, 37(8): 2214-2217.
[8]	LIAO Bin, ZHANG Tao, GUO Binglei, YU Jiong, ZHANG Xuguang, LIU Yan. Performance optimization of ItemBased recommendation algorithm based on Spark [J]. Journal of Computer Applications, 2017, 37(7): 1900-1905.
[9]	YANG Shun, CHEN Zhiguang, XIAO Nong. Implementation of directory index for Pmfs [J]. Journal of Computer Applications, 2017, 37(5): 1241-1245.
[10]	XIAO Zida, ZHU Ligu, FENG Dongyu, ZHANG Di. Performance optimization of distributed database aggregation computing [J]. Journal of Computer Applications, 2017, 37(5): 1251-1256.
[11]	YANG Junjie, LIAO Zhuofan, FENG Chaochao. Survey on big data storage framework and algorithm [J]. Journal of Computer Applications, 2016, 36(9): 2465-2471.
[12]	HAO Zhifeng, CHEN Wei, CAI Ruichu, HUANG Ruihui, WEN Wen, WANG Lijuan. Performance optimization of wireless network based on canonical causal inference algorithm [J]. Journal of Computer Applications, 2016, 36(8): 2114-2120.
[13]	LIU Qing, FU Yinjin, NI Guiqiang, MEI Jianmin. Distributed deduplication storage system based on Hadoop platform [J]. Journal of Computer Applications, 2016, 36(2): 330-335.
[14]	WU Jinbo, SONG Jie, ZHANG Li, BAO Yubin. Probery: probability-based data query system for big data [J]. Journal of Computer Applications, 2016, 36(1): 8-12.
[15]	HOU Jialin, WANG Jiajun, NIE Hongyu. MapReduce performance optimization based on anomaly detection model in heterogeneous cloud environment [J]. Journal of Computer Applications, 2015, 35(9): 2476-2481.