Blockchain storage optimization strategy based on consistent hashing

doi:10.11772/j.issn.1001-9081.2024121836

Abstract

Abstract:

In order to solve the storage challenges caused by the rapid growth of blockchain data， an improved consistent hashing algorithm was proposed to achieve blockchain storage expansion. To address the issues of uneven storage load and data skew of nodes in Hyperledger Fabric for enterprise-level applications， an improved solution based on consistent hashing algorithm was proposed， namely Hash algorithm based on Virtual Node allocation and Dynamic Weight Strategy （VNDWS）. Firstly， a virtual node allocation mechanism was adopted to dynamically assign multiple virtual nodes to each physical node， ensuring an even distribution of data across the hashing ring and reducing load imbalance. Secondly， a dynamic weight mechanism was applied to adjust weights in real-time based on performance indicators such as node storage capacity and network latency， enabling high-performance nodes to handle larger data load， and thereby optimizing data distribution and storage efficiency. Simulation experimental results showed that compared to the traditional blockchain Fabric network and the conventional consistent hashing algorithm， VNDWS reduced node storage consumption by 48.31 and 6.39 percentage points， respectively， while improving data query efficiency by 96.25% and 21.95%. VNDWS effectively reduces node storage consumption and enhances query efficiency in terms of storage expansion.

Key words: blockchain, storage expansion, consistent hashing, virtual node, dynamic weight

摘要：

为了解决区块链数据量剧增带来的存储问题，提出一种改进的一致性哈希算法，以实现区块链的存储扩展。针对Hyperledger Fabric在企业级应用中节点存储负载不均和数据倾斜的问题，基于一致性哈希算法提出了改进方案——基于虚拟节点分配与动态权重策略的哈希算法（VNDWS）。首先，采用虚拟节点分配机制为每个节点动态分配多个虚拟节点，使数据在哈希环中均匀分布，减少负载不均；其次，应用动态权重机制，基于节点存储能力和网络延迟等性能指标实时调整权重，使高性能节点承担更多数据负载，从而优化数据分配和存储效率。仿真实验结果表明，与传统区块链Fabric网络和传统一致性哈希算法相比，VNDWS的节点存储消耗分别降低了48.31和6.39个百分点，而数据查询效率分别提升了96.25%和21.95%。VNDWS在存储扩展方面能有效降低节点存储消耗并提高查询效率。

关键词: 区块链, 存储扩展, 一致性哈希, 虚拟节点, 动态权重

CLC Number:

TP391

Minghao LIU, Jianlei HONG, Chengxiang WANG, Jindong ZHAO. Blockchain storage optimization strategy based on consistent hashing[J]. Journal of Computer Applications, 2025, 45(11): 3446-3452.

刘明灏, 洪建磊, 王成翔, 赵金东. 基于一致性哈希的区块链存储优化策略[J]. 《计算机应用》唯一官方网站, 2025, 45(11): 3446-3452.

Figures/Tables 8

Fig. 1 Example of hashing ring storage structure

Fig. 2 Example of hashing ring storage structure after introducing virtual node mechanism

Fig. 3 Example of storing new data through dynamic weight mechanism

Fig. 4 Topography of small-world

Tab. 1 Blockchain initialization parameters

实验参数	取值
容量权重系数（ $α$ ）	0.6
延迟权重系数（ $β$ ）	0.4
节点存储容量大小（S_p /GB）	［1 000， 5 000］
交易数据大小（S_t /GB）	［0.1，1］
区块大小（S_b /GB）	［1，10］
区块包含交易数	10

Tab. 1 Blockchain initialization parameters

实验参数	取值
容量权重系数（ $α$ ）	0.6
延迟权重系数（ $β$ ）	0.4
节点存储容量大小（S_p /GB）	［1 000， 5 000］
交易数据大小（S_t /GB）	［0.1，1］
区块大小（S_b /GB）	［1，10］
区块包含交易数	10

Fig. 5 Comparison of node storage ratio

Fig. 6 Query request delay

Fig. 7 System reliability analysis

References 21

[1]	LASHKARI B， MUSILEK P. A comprehensive review of blockchain consensus mechanisms［J］. IEEE Access， 2021， 9： 43620-43652.
[2]	HUANG R， YANG X， AJAY P. Consensus mechanism for software-defined blockchain in internet of things［J］. Internet of Things and Cyber-Physical Systems， 2023， 3： 52-60.
[3]	UDDIN M A， STRANIERI A， GONDAL I， et al. A survey on the adoption of blockchain in IOT： challenges and solutions［J］. Blockchain： Research and Applications， 2021， 2（2）： No.100006.
[4]	SANKA A I， IRFAN M， HUANG I， et al. A survey of breakthrough in blockchain technology： adoptions， applications， challenges and future research［J］. Computer Communications， 2021， 169： 179-201.
[5]	FAN X， NIU B， LIU Z. Scalable blockchain storage systems： research progress and models［J］. Computing， 2022， 104（6）： 1497-1524.
[6]	LIANG W， FAN Y， LI K C， et al. Secure data storage and recovery in industrial blockchain network environments［J］. IEEE Transactions on Industrial Informatics， 2020， 16（10）： 6543-6552.
[7]	宋传罡，李雷孝，高昊昱.区块链系统性能优化关键方法综述［J］.计算机工程与应用，2023，59（16）：16-30.
	SONG C G， LI L X， GAO H Y. Review of key technologies for blockchain system performance optimization［J］. Computer Engineering and Applications， 2023， 59（16）： 16-30.
[8]	LUU L， NARAYANAN V， ZHENG C， et al. A secure sharding protocol for open blockchains［C］// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York： ACM， 2016： 17-30.
[9]	DABEK F. A distributed hash table［D/OL］. Cambridge： Massachusetts Institute of Technology， Department of Electrical Engineering and Computer Science， 2005 ［2024-09-21］. .
[10]	MAYMOUNKOV P， MAZIÈRES D. Kademlia： a peer-to-peer information system based on the XOR metric［C］// Proceedings of the 2002 International Workshop on Peer-to-Peer Systems， LNCS 2429. Berlin： Springer， 2002： 53-65.
[11]	STOICA I， MORRIS R， LIBEN-NOWELL D， et al. Chord： a scalable peer-to-peer lookup protocol for Internet applications［J］. IEEE/ACM Transactions on Networking， 2003， 11（1）： 17-32.
[12]	ZHAO J， ZHANG D， LIU W， et al. DHT-based blockchain dual-sharding storage extension mechanism［J］. Applied Sciences， 2022， 12（19）： No.9635.
[13]	LI M， WANG W， ZHANG J. LB-Chain： load-balanced and low-latency blockchain sharding via account migration［J］. IEEE Transactions on Parallel and Distributed Systems， 2023， 34（10）： 2797-2810.
[14]	YU L， ZHAO X-F， JIN Y， et al. Low powered blockchain consensus protocols based on consistent hash［J］. Frontiers of Information Technology and Electronic Engineering， 2019， 20（10）： 1361-1377.
[15]	宁宇豪，黄建华，顾彬，等.结合信誉跳跃一致性哈希的区块链分片协议［J］.计算机工程与应用，2024，60（16）：276-287.
	NING Y H， HUANG J H， GU B， et al. Combining reputation and jump consistent hash for blockchain sharding protocol［J］. Computer Engineering and Applications， 2024， 60（16）： 276-287.
[16]	WANG M， SUN Q. Application of consistent hash based on virtual node in traffic big data［C］// Proceedings of the 7th International Conference on Intelligent Computing and Signal Processing. Piscataway： IEEE， 2022： 1026-1029.
[17]	CHEN R， WANG L， PENG C， et al. An effective sharding consensus algorithm for blockchain systems［J］. Electronics， 2022， 11（16）： No.2597.
[18]	WANG W， YAO X， CHEN J. A map tile data access model based on the jump consistent hash algorithm［J］. ISPRS International Journal of Geo-Information， 2022， 11（12）： 608.
[19]	WANG C， CHEN Y， WANG J， et al. A dynamic weights consistent hashing load balancing method based on heuristic optimization［C］// Proceedings of the 3rd International Conference on Electronic Information Engineering and Computer Science. Piscataway： IEEE， 2023： 492-496.
[20]	曾诗钦，霍如，黄韬，等.区块链技术研究综述：原理、进展与应用［J］.通信学报，2020，41（1）：134-151.
	ZENG S Q， HUO R， HUANG T， et al. Survey of blockchain： principles， progress and applications［J］. Journal on Communications， 2020， 41（1）： 134-151.
[21]	GUGGENBERGER T， SEDLMEIR J， FRIDGEN G， et al. An in-depth investigation of the performance characteristics of Hyperledger Fabric［J］. Computers and Industrial Engineering， 2022， 173： No.108716.