Load balancing strategy of cloud storage based on Hopfield neural network

doi:10.11772/j.issn.1001-9081.2017.08.2214

Journal of Computer Applications ›› 2017, Vol. 37 ›› Issue (8): 2214-2217.DOI: 10.11772/j.issn.1001-9081.2017.08.2214

Previous Articles Next Articles

Load balancing strategy of cloud storage based on Hopfield neural network

LI Qiang^1,2, LIU Xiaofeng³

1. College of Finance and Economics, Taiyuan University of Technology, Taiyuan Shanxi 030024, China;
2. College of Information, Shanxi Finance and Taxation College, Taiyuan Shanxi 030024, China;
3. College of Data Science, Taiyuan University of Technology, Taiyuan Shanxi 030024, China

Received:2017-01-18 Revised:2017-03-10 Online:2017-08-10 Published:2017-08-12
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61502330),the Scientific and Technological Innovation Programs of Higher Education Institutions in Shanxi (20161131),the Project of Soft Science Plan of Shanxi Province (2016041008-5).

基于Hopfield神经网络的云存储负载均衡策略

李强^1,2, 刘晓峰³

1. 太原理工大学财经学院, 太原 030024;
2. 山西省财政税务专科学校信息学院, 太原 030024;
3. 太原理工大学大数据学院, 太原 030024

通讯作者: 刘晓峰
作者简介:李强(1980-),男,山西太原人,副教授,高级工程师,硕士,CCF会员,主要研究方向:Hopfield神经网络;刘晓峰(1979-),男,山西怀仁人,讲师,博士,主要研究方向:Hopfield神经网络。
基金资助:
国家自然科学基金资助项目（61502330）；山西省高等学校科技创新项目（20161131）；山西省软科学计划研究项目（2016041008-5）。

Abstract

Abstract: Focusing on the shortcoming of low storage efficiency and high recovery cost after copy failure of the current Hadoop, Hopfield Neural Network (HNN) was used to improve the overall performance. Firstly, the resource characteristics that affect the storage efficiency were analyzed. Secondly, the resource constraint model was established, the Hopfield energy function was designed and simplified. Finally, the average utilization rate of 8 nodes was analyzed by using the standard test case Wordcount, and the performance and resource utilization of the proposed strategy were compared with three typical algorithms including dynamic resource allocation algorithm, energy-efficient algorithm and Hadoop default storage strategy, and the comparison results showed that the average efficiency of the storage strategy based on HNN was promoted by 15.63%, 32.92% and 55.92% respectively. The results indicate that the proposed algorithm can realize the resource load balancing, help to improve the storage capacity of Hadoop, and speed up the retrieval.

Key words: cloud storage, Hadoop, Hadoop Distributed File System (HDFS), replica strategy, load balancing

摘要： 针对当前Hadoop存储效率不高，且副本故障后恢复成本较高的问题，提出一种基于Hopfield神经网络（HNN）的存储策略。为了实现系统整体性能的提升，首先分析影响存储效率的资源特征；然后建立资源约束模型，设计Hopfield能量函数，并化简该能量函数；最后，通过标准用例Wordcount测试，分析8个节点的平均利用率，并与三个常用算法包括基于资源的动态调用算法、基于能耗的算法和Hadoop默认存储策略进行性能和资源利用方面的比较。实验表明，与对比算法相比，基于HNN的存储策略在效率上分别平均提升15.63%、32.92%和55.92%。因此，该方法在应用中可以更好地实现资源负载平衡，将有助于改善Hadoop的存储能力，并可以加快检索。

关键词: 云存储, Hadoop, Hadoop分布式文件系统, 副本策略, 负载均衡

CLC Number:

TP393.027

LI Qiang, LIU Xiaofeng. Load balancing strategy of cloud storage based on Hopfield neural network[J]. Journal of Computer Applications, 2017, 37(8): 2214-2217.

李强, 刘晓峰. 基于Hopfield神经网络的云存储负载均衡策略[J]. 计算机应用, 2017, 37(8): 2214-2217.

References

[1] BORTHAKUR D. Hadoop[EB/OL].[2011-06-15]. http://lucene.apache.org/hadoop.
[2] BONVIN N, PAPAIOANNOU T G, ABERER K. A self-organized, fault-tolerant and scalable replication scheme for cloud storage[C]//SoCC' 10:Proceedings of the 1st ACM Symposium on Cloud Computing. New York:ACM, 2010:205-216.
[3] WEI Q, VEERAVALLI B, GONG B, et al. CDRM:a cost-effective dynamic replication management scheme for cloud storage cluster[C]//CLUSTER' 10:Proceedings of the 2010 IEEE International Conference on Cluster Computing. Washington, DC:IEEE Computer Society, 2010:188-196.
[4] RUAN X, XIE J, DING Z, et al. Improving MapReduce performance through data placement in heterogeneous Hadoop clusters[C]//Proceedings of the 2010 IEEE International Symposium on Parallel and Distributed Processing, Workshops and Phd Forum. Washington, DC:IEEE Computer Society, 2010:1-9.
[5] WANG Z, LI T, XIONG N, et al. A novel dynamic network data replication scheme based on historical access record and proactive deletion[J]. The Journal of Supercomputing, 2012, 62(1):227-250.
[6] 周敬利,周正达.改进的云存储系统数据分布策略[J].计算机应用,2012,32(2):309-312. (ZHOU J L, ZHOU Z D. Improved data distribution strategy for cloud storage system[J]. Journal of Computer Applications, 2012, 32(2):309-312.)
[7] 林伟伟.一种改进的Hadoop数据放置策略[J].华南理工大学学报(自然科学版),2012,40(1):152-153. (LIN W W. An improved data placement strategy for Hadoop[J]. Journal of South China University of Technology (Natural Science Edition), 2012, 40(1):153-158.)
[8] 张兴.基于Hadoop的云存储平台的研究与实现[D].成都:电子科技大学,2013:22-38. (ZHANG X. Research and Implementation of cloud storage plateform based on Hadoop[D]. Chengdu:University of Electronic Science and Technology of China, 2013:22-38.)
[9] 付雄,贡晓杰,王汝传.云存储系统中基于分簇的数据复制策略[J].计算机工程与科学,2014,36(12):2296-2304. (FU X G, GONG X J, WANG R C. A cluster based data replication strategy in cloud storage systems[J]. Computer Engineering & Science, 2014, 36(12):2296-2304.)
[10] 李强,刘晓峰,贺静.基于语音特征的情感分类[J].小型微型计算机系统,2016,37(2):385-388. (LI Q, LIU X F, HE J. Sentiment classification based on voice features[J]. Journal of Chinese Computer Systems, 2016, 37(2):385-388.)
[11] 王万良,吴启迪,徐新黎.基于Hopfield神经网络的作业车间生产调度方法[J].自动化学报,2002,28(5):838-844. (WANG W L, WU Q D, XU X L. Hopfield neural network approach for job-shop scheduling problems[J]. Acta Automatica Sinica, 2002, 28(5):838-844.)
[12] 王秀利,吴惕华.一种求解多处理器作业调度的Hopfield神经网络方法[J].系统工程与电子技术,2002,24(8):13-16. (WANG X L, WU T H. Scheduling multiprocessor job using Hopfield neural network[J]. Systems Engineering and Electronics, 2002, 24(8):13-16.)
[13] HOPFIELD J J, TANK D W. Neural computation of decision in optimization problems[J]. Biological Cybernetics, 1985, 52(3):141-152.

[1]	ZHAO Quan, TANG Xiaochun, ZHU Ziyu, MAO Anqi, LI Zhanhuai. Low-latency cluster scheduling framework for large-scale short-time tasks [J]. Journal of Computer Applications, 2021, 41(8): 2396-2405.
[2]	YANG Ling, JIANG Chunmao. Strategy of energy-aware virtual machine migration based on three-way decision [J]. Journal of Computer Applications, 2021, 41(4): 990-998.
[3]	XU Hongliang, YANG Guiqin, JIANG Zhanjun. Data center adaptive multi-path load balancing algorithm based on software defined network [J]. Journal of Computer Applications, 2021, 41(4): 1160-1164.
[4]	LI Xiuyan, LIU Mingxi, SHI Wenbo, DONG Guofang. Efficient dynamic data audit scheme for resource-constrained users [J]. Journal of Computer Applications, 2021, 41(2): 422-432.
[5]	TANG Xin, ZHOU Linna. Response obfuscation based secure deduplication method for cloud data with resistance against appending chunk attack [J]. Journal of Computer Applications, 2020, 40(4): 1085-1090.
[6]	FU Wei, GU Chenyang, GAO Qiang. Multi-user sharing ORAM scheme based on attribute encryption [J]. Journal of Computer Applications, 2020, 40(2): 497-502.
[7]	LI Cui, CHEN Qingkui. Dynamic monitoring model based on DPDK parallel communication [J]. Journal of Computer Applications, 2020, 40(2): 335-341.
[8]	DONG Cong, ZHANG Xiao, CHENG Wendi, SHI Jia. Performance optimization of distributed file system based on new type storage devices [J]. Journal of Computer Applications, 2020, 40(12): 3594-3603.
[9]	WANG Haiyong, PENG Yao, GUO Kaixuan. CP-ABE access control scheme based on proxy re-encryption in cloud storage [J]. Journal of Computer Applications, 2019, 39(9): 2611-2616.
[10]	LIU Yudong, WANG Xu'an, TU Guangsheng, WANG Han. Cloud outsourcing data secure auditing protocol throughout whole lifecycle [J]. Journal of Computer Applications, 2019, 39(7): 1954-1958.
[11]	LI Yunshu, TENG Fei, LI Tianrui. Microoperation-based parameter auto-optimization method of Hadoop [J]. Journal of Computer Applications, 2019, 39(6): 1589-1594.
[12]	LI Zhuhong, ZHAO Canming, YAN Long, ZHANG Xinming. Load balancing opportunistic routing protocol for power line communication network in smart grids [J]. Journal of Computer Applications, 2019, 39(3): 812-816.
[13]	ZHOU Jian, JIN Yu, HE Heng, LI Peng. Dynamic cloud data audit model based on nest Merkle Hash tree block chain [J]. Journal of Computer Applications, 2019, 39(12): 3575-3583.
[14]	MING Yang, HE Baokang. Attribute revocation and verifiable outsourcing supported multi-authority attribute-based encryption scheme [J]. Journal of Computer Applications, 2019, 39(12): 3556-3562.
[15]	ZHENG Zhentao, ZHAO Zhuofeng, WANG Guiling, XU Yao. Ship trajectory extraction method for port parking area identification [J]. Journal of Computer Applications, 2019, 39(1): 113-117.

Load balancing strategy of cloud storage based on Hopfield neural network

基于Hopfield神经网络的云存储负载均衡策略

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics