Model integrity verification framework of deep neural network based on fragile fingerprint

doi:10.11772/j.issn.1001-9081.2023101518

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (11): 3479-3486.DOI: 10.11772/j.issn.1001-9081.2023101518

• Cyber security • Previous Articles Next Articles

Model integrity verification framework of deep neural network based on fragile fingerprint

Xiang LIN¹^,², Biao JIN¹^,², Weijing YOU¹^,², Zhiqiang YAO¹^,², Jinbo XIONG¹^,²()

^1.College of Computer and Cyber Security，Fujian Normal University，Fuzhou Fujian 350117，China
^2.Fujian Provincial Key Laboratory of Network Security and Cryptology （Fujian Normal University），Fuzhou Fujian 350007，China

Received:2023-11-07 Revised:2024-01-10 Accepted:2024-01-12 Online:2024-11-13 Published:2024-11-10
Contact: Jinbo XIONG
About author:LIN Xiang， born in 1996， M. S. candidate. His research interests include artificial intelligence security.
JIN Biao， born in 1985， Ph. D.， associate professor. His research interests include information security， privacy protection.
YOU Weijing， born in 1994， Ph. D.， associate professor. Her research interests include data security， artificial intelligence security.
YAO Zhiqiang， born in 1967， Ph. D.， professor. His research interests include information security， privacy protection.
Supported by:
National Natural Science Foundation of China(62272102);Key Program of Natural Science Foundation of Fujian Province(2023J02014);Natural Science Foundation of Fujian Province(2023J01531)

基于脆弱指纹的深度神经网络模型完整性验证框架

林翔¹^,², 金彪¹^,², 尤玮婧¹^,², 姚志强¹^,², 熊金波¹^,²()

^1.福建师范大学计算机与网络空间安全学院，福州 350117
^2.福建省网络安全与密码技术重点实验室（福建师范大学），福州 350007

通讯作者: 熊金波
作者简介:林翔（1996—），男，福建厦门人，硕士研究生，CCF会员，主要研究方向：人工智能安全
金彪（1985—），男，安徽六安人，副教授，博士，CCF会员，主要研究方向：信息安全、隐私保护
尤玮婧（1994—），女，福建三明人，副教授，博士，CCF会员，主要研究方向：数据安全、人工智能安全
姚志强（1967—），男，福建莆田人，教授，博士，CCF高级会员，主要研究方向：信息安全、隐私保护
基金资助:
国家自然科学基金资助项目(62272102);福建省自然科学基金重点项目(2023J02014);福建省自然科学基金资助项目(2023J01531)

Abstract

Abstract:

Pre-trained models are susceptible to attacks implemented by external enemies， such as model fine-tuning and pruning， which destroy their integrity. To address this issue， a fragile fingerprint framework FFWAS （Fragile Fingerprint With Adversarial Samples） for black-box models was proposed. Firstly， a model replication framework without prior knowledge was introduced， and independent model copy for each user was generated by FFWAS. Then， a black-box approach was employed to place a fragile fingerprint trigger set at the model's boundary. If the model was modified and the boundaries were changed， the trigger set would be misclassified. Finally， the integrity of the model was verified by users with the help of the fragile fingerprint trigger set on the model replicas， and if the recognition rate of the trigger set fell below the predefined threshold， it indicated that the model integrity had been compromised. The effectiveness and fragility of FFWAS were analyzed through experiments based on two publicly datasets MNIST and CIFAR-10. Experimental results demonstrate that under both model fine-tuning and pruning attacks， the fingerprint recognition rates of FFWAS significantly decrease compared to the complete model and fall below the predefined thresholds. Compared to Deep Neural Network Authentication framework （DeepAuth） based on model uniqueness and fragile signatures， FFWAS exhibits approximately 22% and 16% improvements in the similarity between the trigger set and the original samples on two datasets， indicating better stealthiness of FFWAS.

Key words: ?neural network, pre-trained model, fragile fingerprint, model integrity, black-box model

摘要：

预训练模型容易受到外部敌手实施的模型微调和模型剪枝等攻击，导致它的完整性被破坏。针对这一问题，提出一种针对黑盒模型的脆弱指纹框架FFWAS （Fragile Fingerprint With Adversarial Samples）。首先，提出一种无先验知识的模型复制框架，而FFWAS为每一位用户创建独立的模型副本；其次，利用黑盒方法在模型边界放置脆弱指纹触发集，若模型发生修改，边界发生变化，触发集将被错误分类；最后，用户借助模型副本上的脆弱指纹触发集对模型的完整性进行验证，若触发集的识别率低于预设阈值，则意味着模型完整性已被破坏。基于2种公开数据集MNIST和CIFAR-10对FFWAS的有效性和脆弱性进行实验分析，结果表明，在模型微调和剪枝攻击下，FFWAS的指纹识别率相较于完整模型均明显下降并低于设定阈值；与基于模型唯一性和脆弱签名的深度神经网络认证框架（DeepAuth）相比，FFWAS的触发集与原始样本在2个数据集上的相似性分别提高了约22%和16%，表明FFWAS具有更好的隐蔽性。

关键词: 神经网络, 预训练模型, 脆弱指纹, 模型完整性, 黑盒模型

CLC Number:

TP309.2

Xiang LIN, Biao JIN, Weijing YOU, Zhiqiang YAO, Jinbo XIONG. Model integrity verification framework of deep neural network based on fragile fingerprint[J]. Journal of Computer Applications, 2024, 44(11): 3479-3486.

林翔, 金彪, 尤玮婧, 姚志强, 熊金波. 基于脆弱指纹的深度神经网络模型完整性验证框架[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3479-3486.

Figures/Tables 14

Fig. 1 Flowchart schematic of FFWAS

Fig. 2 Process of model copy generation

Fig. 3 Process of fingerprint generation

Tab.1 Comparison of precision and FSR of original model， model copy and fingerprint model copy on two datasets

数据集	模型	模型精度	FSR
MNIST	LeNet-5	98.86	N/A
	模型副本	98.11	N/A
	指纹模型副本	98.11	100
CIFAR-10	ResNet-34	95.09	N/A
	模型副本	93.80	N/A
	指纹模型副本	93.80	100

Fig. 4 Examples of trigger set samples generated by different watermarking methods

Fig. 5 Fingerprint examples generated by FFWAS for MNIST

Fig. 6 Fingerprint examples generated by FFWAS for CIFAR-10

Fig. 7 Comparison of FSR and model precision of FFWAS under model pruning

Tab.2 Comparison of FSR and model precision of FFWAS under model fine-tuning

数据集	模型	模型精度	FSR
MNIST	指纹LeNet-5	98.11	100
MNIST	微调LeNet-5	99.02	14
CIFAR-10	指纹ResNet-34	93.80	100
CIFAR-10	微调ResNet-34	93.53	2

Fig. 8 Examples of trigger set samples generated by different schemes （MNIST）

Fig. 9 Examples of trigger set samples generated by different schemes （CIFAR-10）

Tab. 3 Comparison of average L2 distance between original samples and different trigger sets

数据集	文献［16］方案	DeepAuth^［23］	文献［11］方案	文献［27］方案	FFWAS
MNIST	3.9×10^-3	4.1×10^-4	—	—	3.2×10^-4
CIFAR-10	2.4×10^-4	6.9×10^-5	352.37	455.78	5.8×10^-5

Tab.4 Comparison of average generation time for each image of different trigger sets

数据集	文献［16］方案	DeepAuth^［23］	文献［11］方案	文献［27］方案	FFWAS
MNIST	0.08	12.62	—	—	5.29
CIFAR-10	0.13	30.48	0	26.58	17.61

Tab.5 Comparison of different DNN watermarking frameworks

方案	方法类型	触发集生成	验证方法	$Δ$ /%
文献［11］方案	鲁棒水印	黑盒	黑盒	-1.0
文献［16］方案	鲁棒指纹	白盒	黑盒	-0.7
DeepAuth^［23］	脆弱水印	白盒	黑盒	-0.3
文献［27］方案	脆弱水印	黑盒	黑盒	-0.1
FFWAS	脆弱指纹	黑盒	黑盒	=

Tab.5 Comparison of different DNN watermarking frameworks

方案	方法类型	触发集生成	验证方法	$Δ$ /%
文献［11］方案	鲁棒水印	黑盒	黑盒	-1.0
文献［16］方案	鲁棒指纹	白盒	黑盒	-0.7
DeepAuth^［23］	脆弱水印	白盒	黑盒	-0.3
文献［27］方案	脆弱水印	黑盒	黑盒	-0.1
FFWAS	脆弱指纹	黑盒	黑盒	=

References 34

1	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778.
2	AMODEI D， ANANTHANARAYANAN S， ANUBHAI R， et al. Deep Speech 2： end-to-end speech recognition in English and Mandarin［C］// Proceedings of the 33rd International Conference on Machine Learning. Cambridge： MIT Press， 2016： 173-182.
3	田蕾，葛丽娜.基于差分隐私的广告推荐算法［J］.计算机应用，2023，43（11）：3346-3350.
	TIAN L， GE L N. Advertising recommendation algorithm based on differential privacy［J］. Journal of Computer Applications， 2023， 43（11）： 3346-3350.
4	RIBEIRO M， GROLINGER K， CAPRETZ M A M. MLaaS： machine learning as a service［C］// Proceedings of the IEEE 14th International Conference on Machine Learning and Applications. Piscataway： IEEE， 2015： 896-902.
5	VAN SCHYNDEL R G， TIRKEL A Z， OSBORNE C F. A digital watermark［C］// Proceedings of 1st International Conference on Image Processing： Volume 2. Piscataway： IEEE， 1994： 86-90.
6	ZHANG X， WANG S. Fragile watermarking with error-free restoration capability［J］. IEEE Transactions on Multimedia， 2008， 10（8）： 1490-1499.
7	UCHIDA Y， NAGAI Y， SAKAZAWA S， et al. Embedding watermarks into deep neural networks［C］// Proceedings of the 2017 ACM International Conference on Multimedia Retrieval. New York： ACM， 2017： 269-277.
8	CHEN H， ROUHANI B D， FU C， et al. DeepMarks： a secure fingerprinting framework for digital rights management of deep learning models［C］// Proceedings of the 2019 International Conference on Multimedia Retrieval. New York： ACM， 2019： 105-113.
9	ZHANG J， GU Z， JANG J， et al. Protecting intellectual property of deep neural networks with watermarking［C］// Proceedings of the 2018 ACM Asia Conference on Computer and Communications Security. New York： ACM， 2018： 159-172.
10	LI Z， HU C， ZHANG Y， et al. How to prove your model belongs to you： a blind-watermark based framework to protect intellectual property of DNN［C］// Proceedings of the 35th Annual Computer Security Applications Conference. New York： ACM， 2019： 126-137.
11	ADI Y， BAUM C， CISSE M， et al. Turning your weakness into a strength： watermarking deep neural networks by backdooring［C］// Proceedings of the 27th USENIX Security Symposium. Berkeley： USENIX Association， 2018： 1615-1631.
12	JIA H， CHOQUETTE-CHOO C A， CHANDRASEKARAN V， et al. Entangled watermarks as a defense against model extraction［C］// Proceedings of the 30th USENIX Security Symposium. Berkeley： USENIX Association， 2021： 1937-1954.
13	樊雪峰，周晓谊，朱冰冰，等. 深度神经网络模型版权保护方案综述［J］. 计算机研究与发展， 2022， 59（5）： 953-977.
	FAN X F， ZHOU X Y， ZHU B B， al at. Survey of copyright protection schemes based on DNN model［J］. Journal of Computer Research and Development， 2022， 59（5）： 953-977.
14	李璇，邓天鹏，熊金波，等.基于模型后门的联邦学习水印［J］.软件学报，2024，35（7）：3454-3468.
	LI X， DENG T P， XIONG J B， et al. Federated learning watermark based on backdoor［J］. Journal of Software， 2024， 35（7）： 3454-3468.
15	WAGNER N R. Fingerprinting［C］// Proceedings of the 1983 IEEE Symposium on Security and Privacy. Piscataway： IEEE， 1983： 18.
16	ZHAO J， HU Q， LIU G， et al. AFA： adversarial fingerprinting authentication for deep neural networks［J］. Computer Communications， 2020， 150： 488-497.
17	XUE M， WU Z， HE C， et al. Active DNN IP protection： a novel user fingerprint management and DNN authorization control technique［C］// Proceedings of the IEEE 19th International Conference on Trust， Security and Privacy in Computing and Communications. Piscataway： IEEE， 2020： 975-982.
18	XUE M， HE C， WANG J， et al. One-to-N & N-to-One： two advanced backdoor attacks against deep learning models［J］. IEEE Transactions on Dependable and Secure Computing， 2022， 19（3）： 1562-1578.
19	XUE M， SUN S， ZHANG Y， et al. Active intellectual property protection for deep neural networks through stealthy backdoor and users' identities authentication［J］. Applied Intelligence， 2022， 52（14）： 16497-16511.
20	GUAN X， FENG H， ZHANG W， et al. Reversible watermarking in deep convolutional neural networks for integrity authentication［C］// Proceedings of the 28th ACM International Conference on Multimedia. New York： ACM， 2020： 2273-2280.
21	BOTTA M， CAVAGNINO D， ESPOSITO R. NeuNAC： a novel fragile watermarking algorithm for integrity protection of neural networks［J］. Information Sciences， 2021， 576： 228-241.
22	HE Z， ZHANG T， LEE R. Sensitive-sample fingerprinting of deep neural networks［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 4724-4732.
23	LAO Y， ZHAO W， YANG P， et al. DeepAuth： a DNN authentication framework by model-unique and fragile signature embedding［C］// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2022： 9595-9603.
24	MĄDRY A， MAKELOV A， SCHMIDT L， et al. Towards deep learning models resistant to adversarial attacks［EB/OL］. ［2023-08-08］. .
25	WANG S， ABUADBBA S， AGARWAL S， et al. PublicCheck： public integrity verification for services of run-time deep models［C］// Proceedings of the 2023 IEEE Symposium on Security and Privacy. Piscataway： IEEE， 2023： 1348-1365.
26	CARLINI N， WAGNER D. Towards evaluating the robustness of neural networks［C］// Proceedings of the 2017 IEEE Symposium on Security and Privacy. Piscataway： IEEE， 2017： 39-57.
27	ZHU R， WEI P， LI S， et al. Fragile neural network watermarking with trigger image set［C］// Proceedings of the 2021 International Conference on Knowledge Science， Engineering and Management， LNCS 12815. Cham： Springer， 2021： 280-293.
28	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network［EB/OL］. ［2023-10-22］. .
29	LeCUN Y， BOTTOU L， BENGIO Y， et al. Gradient-based learning applied to document recognition［J］. Proceedings of the IEEE， 1998， 86（11）： 2278-2324.
30	KRIZHEVSKY A. Learning multiple layers of features from tiny images［R/OL］. ［2023-07-19］. .
31	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. ［2022-12-22］..
32	HOOKER S， DAUPHIN Y， COURVILLE A， et al. Selective brain damage： measuring the disparate impact of model pruning［EB/OL］. ［2023-09-26］. .
33	HAN S， POOL J， TRAN J， et al. Learning both weights and connections for efficient neural network［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 1. Cambridge： MIT Press， 2015： 1135-1143.
34	PITTARAS N， MARKATOPOULOU F， MEZARIS V， et al. Comparison of fine-tuning and extension strategies for deep convolutional neural networks［C］// Proceedings of the 2017 International Conference on MultiMedia Modeling， LNCS 10132. Cham： Springer， 2017： 102-114.

[1]	Chenyang LI, Long ZHANG, Qiusheng ZHENG, Shaohua QIAN. Multivariate controllable text generation based on diffusion sequences [J]. Journal of Computer Applications, 2024, 44(8): 2414-2420.
[2]	Zhengyu ZHAO, Jing LUO, Xinhui TU. Information retrieval method based on multi-granularity semantic fusion [J]. Journal of Computer Applications, 2024, 44(6): 1775-1780.
[3]	Xinyue ZHANG, Rong LIU, Chiyu WEI, Ke FANG. Aspect-based sentiment analysis method with integrating prompt knowledge [J]. Journal of Computer Applications, 2023, 43(9): 2753-2759.
[4]	Bihui YU, Xingye CAI, Jingxuan WEI. Few-shot text classification method based on prompt learning [J]. Journal of Computer Applications, 2023, 43(9): 2735-2740.
[5]	Yuelin TIAN, Ruizhang HUANG, Lina REN. Scholar fine-grained information extraction method fused with local semantic features [J]. Journal of Computer Applications, 2023, 43(9): 2707-2714.
[6]	Xiaoyan ZHANG, Zhengyu DUAN. Cross-lingual zero-resource named entity recognition model based on sentence-level generative adversarial network [J]. Journal of Computer Applications, 2023, 43(8): 2406-2411.
[7]	Lifeng SHI, Zhengwei NI. Dialogue state tracking model based on slot correlation information extraction [J]. Journal of Computer Applications, 2023, 43(5): 1430-1437.
[8]	Ming XU, Linhao LI, Qiaoling QI, Liqin WANG. Abductive reasoning model based on attention balance list [J]. Journal of Computer Applications, 2023, 43(2): 349-355.
[9]	Mingyue WU, Dong ZHOU, Wenyu ZHAO, Wei QU. Sentence embedding optimization based on manifold learning [J]. Journal of Computer Applications, 2023, 43(10): 3062-3069.
[10]	Yaming LI, Kai XING, Hongwu DENG, Zhiyong WANG, Xuan HU. Derivative-free few-shot learning based performance optimization method of pre-trained models with convolution structure [J]. Journal of Computer Applications, 2022, 42(2): 365-374.
[11]	LIU Ruiheng, YE Xia, YUE Zengying. Review of pre-trained models for natural language processing tasks [J]. Journal of Computer Applications, 2021, 41(5): 1236-1246.
[12]	LI Huihui, YAN Kun, ZHANG Lixuan, LIU Wei, LI Zhi. Circular pointer instrument recognition system based on MobileNetV2 [J]. Journal of Computer Applications, 2021, 41(4): 1214-1220.
[13]	WANG Kun, ZHENG Yi, FANG Shuya, LIU Shouyin. Long text aspect-level sentiment analysis based on text filtering and improved BERT [J]. Journal of Computer Applications, 2020, 40(10): 2838-2844.

Model integrity verification framework of deep neural network based on fragile fingerprint

基于脆弱指纹的深度神经网络模型完整性验证框架

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 14

References 34

Related Articles 13

Recommended Articles

Metrics