Blockchain-based deduplication and data integrity audit scheme

doi:10.11772/j.issn.1001-9081.2024121835

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (11): 3453-3462.DOI: 10.11772/j.issn.1001-9081.2024121835

• The 7th CCF China Conference on Blockchain Technology • Previous Articles

Blockchain-based deduplication and data integrity audit scheme

Tingting GAO¹, Zhongyuan YAO¹(), Miao JIA¹, Xueming SI¹^,², Huanming TAN³, Yufeng ZHAN⁴

^1.The Frontier Information Technology Research Institute，Zhongyuan University of Technology，Zhengzhou Henan 450007，China
^2.Henan International Joint Laboratory of Blockchain and Data Sharing （Zhongyuan University of Technology），Zhengzhou Henan 450007，China
^3.Fujian Fulian Technology Company Limited，Fuzhou Fujian 350108，China
^4.School of Automation，Beijing Institute of Technology，Beijing 100081，China

Received:2024-12-31 Revised:2025-01-07 Accepted:2025-01-16 Online:2025-04-02 Published:2025-11-10
Contact: Zhongyuan YAO
About author:GAO Tingting， born in 2000， M. S. candidate. Her research interests include cryptology， blockchain.
JIA Miao， born in 1999， M. S. candidate. Her research interests include cryptology， blockchain.
SI Xueming， born in 1966， Ph. D.， professor. His research interests include cryptology， blockchain.
TAN Huanming， born in 1974， M. S.， associate research fellow. His research interests include cryptology， blockchain.
ZHAN Yufeng， born in 1989， Ph. D.， associate professor. His research interests include cloud-edge-device collaborative sensing and control.
Supported by:
This work is partially supported by National Key Research and Development Plan “Blockchain” Special Project(2022YFB2703200)

基于区块链的重复数据删除和数据完整性审计方案

高婷婷¹, 姚中原¹(), 贾淼¹, 斯雪明¹^,², 谭焕明³, 詹玉峰⁴

^1.中原工学院前沿信息技术研究院，郑州 450007
^2.河南省区块链与数据共享国际联合实验室（中原工学院），郑州 450007
^3.福建福链科技有限公司，福州 350108
^4.北京理工大学自动化学院，北京 100081

通讯作者: 姚中原
作者简介:高婷婷（2000—），女，甘肃平凉人，硕士研究生，主要研究方向：密码学、区块链
贾淼（1999—），女，河南南阳人，硕士研究生，主要研究方向：密码学、区块链
斯雪明（1966—），男，浙江诸暨人，教授，博士，CCF会员，主要研究方向：密码学、区块链
谭焕明（1974—），男，浙江上虞人，副研究员，硕士，CCF会员，主要研究方向：密码学、区块链
詹玉峰（1989—），男，北京人，副教授，博士，主要研究方向：云边端协同感知与控制。
基金资助:
国家重点研发计划“区块链”专项(2022YFB2703200)

Abstract

Abstract:

To address the issues of data redundancy and data integrity protection in current cloud storage systems， a blockchain-based cloud storage data deduplication and integrity audit scheme was proposed. This approach combined deduplication technology with a blockchain distributed auditing mechanism， ensuring data confidentiality and integrity while achieving deduplicated storage. Firstly， the Message-Locked Encryption （MLE） method was utilized to generate identical encrypted ciphertexts for the same data from different users. For duplicate data uploaders， ownership verification was conducted using a Verkle Tree （VT）-based Proof of Ownership （PoW） mechanism， thus enabling secure ciphertext deduplication. Secondly， based on the immutable characteristic of blockchain， an efficient data integrity auditing mechanism was designed， allowing transparent auditing and verification without compromising user data privacy. This mechanism can defend against various threats， including external attackers， malicious behaviors from Cloud Service Providers （CSPs）， and key management risks. Finally， through functional analysis and security analyses such as key security and resistance to collusion attacks， it is demonstrated that the proposed scheme has high security and practicality. Performance analysis results show that the integrity verification time is proportional to the number of challenge blocks， and when the number of challenge blocks is 1 000， the integrity verification time is approximately 85 ms. Experimental analysis results show that the scheme can provide with reliable deduplication and data integrity auditing services for the cloud storage systems.

Key words: deduplication, data integrity, blockchain, Message-Locked Encryption (MLE), data privacy

摘要：

为解决当前云存储系统中的数据冗余和数据完整性保障问题，提出一种基于区块链的云存储数据去重与完整性审计方案。该方案结合重复数据删除技术与区块链分布式审计机制，在实现去重存储的同时，可确保数据的机密性和完整性。首先，利用消息锁定加密（MLE）方法为不同用户的相同数据生成相同的加密密文，对于重复数据的上传者，采用基于沃克尔树（VT）的所有权证明（PoW）机制进行所有权验证，从而实现安全的密文去重；其次，基于区块链不可篡改的特性，设计了一种高效的数据完整性审计机制，能够在不影响用户数据隐私的前提下进行透明的审计和验证，可抵御包括外部攻击者、云服务提供商（CSP）恶意行为及密钥管理风险在内的多种威胁；最后，通过功能性分析和密钥安全、抗合谋攻击等安全性分析，证明所提方案具有较高的安全性和实用性。性能分析结果表明，完整性验证时间与挑战块数成正比，当挑战块数为1 000时，完整性验证时间约为85 ms。实验分析结果表明，该方案能为云存储系统提供可靠的去重与数据完整性审计服务。

关键词: 重复数据删除, 数据完整性, 区块链, 消息锁定加密, 数据隐私

CLC Number:

TP311.13

Tingting GAO, Zhongyuan YAO, Miao JIA, Xueming SI, Huanming TAN, Yufeng ZHAN. Blockchain-based deduplication and data integrity audit scheme[J]. Journal of Computer Applications, 2025, 45(11): 3453-3462.

高婷婷, 姚中原, 贾淼, 斯雪明, 谭焕明, 詹玉峰. 基于区块链的重复数据删除和数据完整性审计方案[J]. 《计算机应用》唯一官方网站, 2025, 45(11): 3453-3462.

Figures/Tables 16

References 27

[1]	ATENIESE G， BURNS R， CURTMOLA R， et al. Provable data possession at untrusted stores［C］// Proceedings of the 14th ACM Conference on Computer and Communications Security. New York： ACM， 2007： 598-609.
[2]	REINSEL D， GANTZ J， RYDNING J. The digitization of the world from edge to core［EB/OL］. ［2024-03-24］..
[3]	WANG L， WANG B， SONG W， et al. A key-sharing based secure deduplication scheme in cloud storage［J］. Information Sciences， 2019， 504： 48-60.
[4]	ZHANG Y， SU H， YANG M， et al. Secure deduplication based on Rabin fingerprinting over wireless sensing data in cloud computing［J］. Security and Communication Networks， 2018， 2018： No.9081814.
[5]	KUSZMAUL J. Verkle trees［EB/OL］. ［2024-09-04］..
[6]	GAO X， YU J， SHEN W T， et al. Achieving low-entropy secure cloud data auditing with file and authenticator deduplication［J］. Information Sciences， 2021， 546： 177-191.
[7]	HALEVI S， HARNIK D， PINKAS B， et al. Proofs of ownership in remote storage systems［C］// Proceedings of the 18th ACM Conference on Computer and Communications Security. New York： ACM， 2011： 491-500.
[8]	QI H， LI P， WANG C， et al. A hierarchical key management scheme for cross-domain secure deduplication［J］. Mobile Networks and Applications， 2023， 28（2）： 473-485.
[9]	TANG X， GUO C， CHOO K K R， et al. A secure and lightweight cloud data deduplication scheme with efficient access control and key management［J］. Computer Communications， 2024， 222： 209-219.
[10]	袁颖，朱洪亮，陈玉玲，等.基于数据持有性证明的完整性验证技术综述［J］.计算机工程与应用，2019，55（18）：1-7.
	YUAN Y， ZHU H L， CHEN Y L， et al. Survey of data integrity verification technology based on provable data possession［J］. Computer Engineering and Applications， 2019， 55（18）： 1-7.
[11]	ZHANG F， WEN B， YAN Y， et al. Privacy protection multi-copy provable data possession supporting data reliability［C］// Proceedings of the 2023 International Conference on Blockchain and Trustworthy Systems， CCIS 1896. Singapore： Springer， 2024： 119-132.
[12]	ZHANG K， GUO Z， WANG L， et al. Revocable certificateless provable data possession with identity privacy in cloud storage［J］. Computer Standards and Interfaces， 2024， 90： No.103848.
[13]	高婷婷，姚中原，贾淼，等.链上链下一致性保护技术综述［J］. 计算机应用，2024，44（12）：3658-3668.
	GAO T T， YAO Z Y， JIA M， et al. Overview of on-chain and off-chain consistency protection technologies［J］. Journal of Computer Applications， 2024， 44（12）： 3658-3668.
[14]	JUELS A， KALISKI B S， Jr. PORs： proofs of retrievability for large files［C］// Proceedings of the 14th ACM Conference on Computer and Communications Security. New York： ACM， 2007： 584-597.
[15]	TAN X， XIE Q， HAN L， et al. Proof of retrievability with flexible designated verification for cloud storage［J］. Computers and Security， 2023， 135： No.103486.
[16]	YUAN H， CHEN X， WANG J， et al. Blockchain-based public auditing and secure deduplication with fair arbitration［J］. Information Sciences， 2020， 541： 409-425.
[17]	XU Y， JIN C， QIN W， et al. BDACD： blockchain-based decentralized auditing supporting ciphertext deduplication［J］. Journal of Systems Architecture， 2024， 147： No.103053.
[18]	LIU S， YAO Y， TIAN G， et al. A blockchain-based compact audit-enabled deduplication in decentralized storage［J］. Computer Standards and Interfaces， 2023， 85： No.103718.
[19]	LI S， XU C， ZHANG Y， et al. Blockchain-based transparent integrity auditing and encrypted deduplication for cloud storage［J］. IEEE Transactions on Services Computing， 2023， 16（1）： 134-146.
[20]	TIAN G， HU Y， WEI J， et al. Blockchain-based secure deduplication and shared auditing in decentralized storage［J］. IEEE Transactions on Dependable and Secure Computing， 2022， 19（6）： 3941-3954.
[21]	KWON H， HAHN C， KANG K， et al. Secure deduplication with reliable and revocable key management in fog computing［J］. Peer-to-Peer Networking and Applications， 2019， 12： 850-864.
[22]	KIM W B， LEE I Y. Survey on data deduplication in cloud storage environments［J］. Journal of Information Processing Systems， 2021， 17（3）： 658-673.
[23]	伍高飞，袁紫依，孙思贤，等.云数据安全去重技术研究综述［J］.密码学报，2023，10（6）：1099-1117.
	WU G F， YUAN Z Y， SUN S X， et al. A survey on secure data deduplication techniques in cloud［J］. Journal of Cryptologic Research， 2023， 10（6）： 1099-1117.
[24]	BELLARE M， KEELVEEDHI S， RISTENPART T. Message-locked encryption and secure deduplication［C］// Proceedings of the 2013 International Conference on the Theory and Applications of Cryptographic Techniques， LNCS 7881. Berlin： Springer， 2013： 296-312.
[25]	DI PIETRO R， SORNIOTTI A. Boosting efficiency and security in proof of ownership for deduplication［C］// Proceedings of the 7th ACM Symposium on Information， Computer and Communications Security. New York： ACM， 2012： 81-82.
[26]	MIAO Y， GAI K， ZHU L， et al. Blockchain-based shared data integrity auditing and deduplication［J］. IEEE Transactions on Dependable and Secure Computing， 2024， 21（4）： 3688-3703.
[27]	LI J W， LI J， XIE D， et al. Secure auditing and deduplicating data in cloud［J］. IEEE Transactions on Computers， 2016， 65（8）： 2386-2396.

符号	含义	符号	含义
λ	安全参数	T_r	重复数据删除标签
P	系统参数	C	文件密文
G	乘法循环群	C_k	收敛密钥密文
g	G的一个生成元	N	数据块个数
q₁，q₂	2个大素数	c_i	密文块
φ（n）	欧拉函数	V_i	承诺
（e，n）	公钥	p_i	相对于承诺的证明
（d，n）	私钥	ID_DO	用户身份标识
N	数据块个数	Proof	完整性证明
chal	挑战信息	H₁	生成收敛密钥的哈希函数
		H₂	生成重复数据删除标签的哈希函数

符号	含义	符号	含义
λ	安全参数	T_r	重复数据删除标签
P	系统参数	C	文件密文
G	乘法循环群	C_k	收敛密钥密文
g	G的一个生成元	N	数据块个数
q₁，q₂	2个大素数	c_i	密文块
φ（n）	欧拉函数	V_i	承诺
（e，n）	公钥	p_i	相对于承诺的证明
（d，n）	私钥	ID_DO	用户身份标识
N	数据块个数	Proof	完整性证明
chal	挑战信息	H₁	生成收敛密钥的哈希函数
		H₂	生成重复数据删除标签的哈希函数

方案	第三方实体	存储去重	收敛加密	PoW	避免单点失效	抵制暴力攻击	抵制合谋攻击	数据完整性	密钥管理
Wang等^［3］	IS	Y	CE	MHT	N	Y	N	Y	密钥共享方法
Li等^［27］	MapReduce+密钥服务器	Y	CE	MHT	N	Y	N	Y	密钥服务器
BDACD^［17］	Blockchain	Y	CE	T-MHT	Y	Y	Y	Y	N
本文方案	Blockchain	Y	MLE	Verkle Tree	Y	Y	Y	Y	Blockchain

方案	第三方实体	存储去重	收敛加密	PoW	避免单点失效	抵制暴力攻击	抵制合谋攻击	数据完整性	密钥管理
Wang等^［3］	IS	Y	CE	MHT	N	Y	N	Y	密钥共享方法
Li等^［27］	MapReduce+密钥服务器	Y	CE	MHT	N	Y	N	Y	密钥服务器
BDACD^［17］	Blockchain	Y	CE	T-MHT	Y	Y	Y	Y	N
本文方案	Blockchain	Y	MLE	Verkle Tree	Y	Y	Y	Y	Blockchain

文件大小/MB	密钥生成时间/s	文件大小/MB	密钥生成时间/s
3	1.7	12	7.0
6	3.6	15	8.8
9	5.2

Blockchain-based deduplication and data integrity audit scheme

基于区块链的重复数据删除和数据完整性审计方案

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 16

References 27

Related Articles 15

Recommended Articles

Metrics

[1]	Sheping ZHAI, Pengju ZHU, Rui YANG, Jiayiteng LIU. Blockchain-based identity management system for internet of things [J]. Journal of Computer Applications, 2025, 45(9): 2873-2881.
[2]	Wei GAO, Lihua LIU, Bintao HE, Fang’an DENG. Research advances in blockchain consensus mechanisms and improvement algorithms [J]. Journal of Computer Applications, 2025, 45(9): 2848-2864.
[3]	Wei SHE, Tianxiang MA, Haige FENG, Zhao TIAN, Wei LIU. Blockchain covert communication method based on contract call concealment [J]. Journal of Computer Applications, 2025, 45(9): 2865-2872.
[4]	Di WANG. P-Dledger： blockchain edge node security architecture [J]. Journal of Computer Applications, 2025, 45(8): 2630-2636.
[5]	Haiyang PENG, Weixing JI, Fawang LIU. Blockchain-based data notarization model for autonomous driving simulation testing [J]. Journal of Computer Applications, 2025, 45(8): 2421-2427.
[6]	Shuo ZHANG, Guokai SUN, Yuan ZHUANG, Xiaoyu FENG, Jingzhi WANG. Dynamic detection method of eclipse attacks for blockchain node analysis [J]. Journal of Computer Applications, 2025, 45(8): 2428-2436.
[7]	Yuxuan CHEN, Haibin ZHENG, Zhenyu GUAN, Boheng SU, Yujue WANG, Zhenwei GUO. Blockchain sharding mechanism in asynchronous network based on HoneyBadgerBFT and DAG [J]. Journal of Computer Applications, 2025, 45(7): 2092-2100.
[8]	Li’e WANG, Caiyi LIN, Yongdong LI, Xingcheng FU, Xianxian LI. Digital content copyright protection and fair tracking scheme based on blockchain [J]. Journal of Computer Applications, 2025, 45(6): 1756-1765.
[9]	Xin SHAO, Zigang CHEN, Xingchun YANG, Haihua ZHU, Wenjun LUO, Long CHEN, Yousheng ZHOU. Vehicular digital evidence preservation and access control based on consortium blockchain [J]. Journal of Computer Applications, 2025, 45(6): 1902-1910.
[10]	Gaimei GAO, Miaolian DU, Chunxia LIU, Yuli YANG, Weichao DANG, Guoxia DI. Privacy protection method for consortium blockchain based on SM2 linkable ring signature [J]. Journal of Computer Applications, 2025, 45(5): 1564-1572.
[11]	Xiaojun ZHANG, Yunpu HAO, Lei LI, Chenyang LI, Ziyu ZHOU. Data tamper-proof batch auditing scheme based on industrial cloud storage systems [J]. Journal of Computer Applications, 2025, 45(3): 891-895.
[12]	Bimang SUN, Wunan WAN, Shibin ZHANG, Jinquan ZHANG. Multi-domain access control scheme in blockchain based on SM2 homomorphic encryption [J]. Journal of Computer Applications, 2025, 45(11): 3432-3439.
[13]	Mei TANG, Wunan WAN, Shibin ZHANG, Jinquan ZHANG. Survey of DDoS protection research based on blockchain [J]. Journal of Computer Applications, 2025, 45(11): 3416-3423.
[14]	Tingda SHEN, Konglin ZHU, Lin ZHANG. Semi-centralized cross-chain bridge design based on hash time lock and incentive mechanism [J]. Journal of Computer Applications, 2025, 45(11): 3440-3445.
[15]	Fei WANG, Hengdi WANG, Konglin ZHU, Lin ZHANG. Adaptive online blockchain sharding algorithm based on trusted execution environments [J]. Journal of Computer Applications, 2025, 45(11): 3424-3431.