Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (4): 1160-1168.DOI: 10.11772/j.issn.1001-9081.2022030337
Special Issue: 网络空间安全
• Cyber security • Previous Articles Next Articles
Received:
2022-03-21
Revised:
2022-05-13
Accepted:
2022-05-25
Online:
2023-04-11
Published:
2023-04-10
Contact:
Chunyong YIN
About author:
QU Rui, born in 1999, M. S. candidate. His research interests include differential privacy, federated learning.
通讯作者:
尹春勇
作者简介:
屈锐(1999—),男,江苏宿迁人,硕士研究生,主要研究方向:差分隐私、联邦学习。
CLC Number:
Chunyong YIN, Rui QU. Federated learning algorithm based on personalized differential privacy[J]. Journal of Computer Applications, 2023, 43(4): 1160-1168.
尹春勇, 屈锐. 基于个性化差分隐私的联邦学习算法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1160-1168.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2022030337
方法 | 优点 | 缺点 | 保护对象 | 保护场景 |
---|---|---|---|---|
同态加密[ | 准确性高,隐私保护严格 | 计算开销高,通信开销高,不保护发布模型 | 本地梯度 | 统一预算场景 |
多方安全计算[ | 准确性较高,隐私保护严格 | 通信开销高,协议复杂脆弱 | 本地梯度 | 统一预算场景 |
中心化差分隐私[ | 准确性较高,通信开销低,计算量较低 | 需要中心服务器可信 | 中心梯度 | 统一预算场景 |
本地化差分隐私[ | 通信开销低,扰动方案灵活 | 准确性低 | 本地梯度 | 统一预算场景 |
安全混洗[ | 准确性较高,隐私保护严格 | 计算开销高 | 本地梯度 | 统一预算场景 |
PDP-FL | 可以提供个性化隐私保证,准确性较高 | 需要合理的隐私分级 | 本地和中心梯度 | 个性化预算场景 |
Tab. 1 Comparison of privacy protection methods based on federated learning
方法 | 优点 | 缺点 | 保护对象 | 保护场景 |
---|---|---|---|---|
同态加密[ | 准确性高,隐私保护严格 | 计算开销高,通信开销高,不保护发布模型 | 本地梯度 | 统一预算场景 |
多方安全计算[ | 准确性较高,隐私保护严格 | 通信开销高,协议复杂脆弱 | 本地梯度 | 统一预算场景 |
中心化差分隐私[ | 准确性较高,通信开销低,计算量较低 | 需要中心服务器可信 | 中心梯度 | 统一预算场景 |
本地化差分隐私[ | 通信开销低,扰动方案灵活 | 准确性低 | 本地梯度 | 统一预算场景 |
安全混洗[ | 准确性较高,隐私保护严格 | 计算开销高 | 本地梯度 | 统一预算场景 |
PDP-FL | 可以提供个性化隐私保证,准确性较高 | 需要合理的隐私分级 | 本地和中心梯度 | 个性化预算场景 |
数据集 | 隐私保护 | 算法 | 场景 | 准确度/% | 损失 |
---|---|---|---|---|---|
MNIST | 无 | FedAvg | 无 | 95.10 | 1.2 |
GDP | GDP-FL | 无 | 94.20 | 1.5 | |
LDP | LDP-Fed | 无 | 93.70 | 1.4 | |
PDP | PDP-FL | 场景1 | 93.90 | 1.3 | |
场景2 | 1.2 | ||||
场景3 | 93.80 | 1.3 | |||
场景4 | 94.00 | 1.3 | |||
CIFAR-10 | 无 | FedAvg | 无 | 46.01 | 1.2 |
GDP | GDP-FL | 无 | 44.22 | 1.5 | |
LDP | LDP-Fed | 无 | 43.60 | 1.4 | |
PDP | PDP-FL | 场景1 | 43.55 | 1.4 | |
场景2 | 1.2 | ||||
场景3 | 43.40 | 1.4 | |||
场景4 | 44.10 | 1.3 |
Tab. 2 Comparison between FedAvg GDP-FL,LDP-Fed and PDP-FL on MNIST CIFAR-10 datasets
数据集 | 隐私保护 | 算法 | 场景 | 准确度/% | 损失 |
---|---|---|---|---|---|
MNIST | 无 | FedAvg | 无 | 95.10 | 1.2 |
GDP | GDP-FL | 无 | 94.20 | 1.5 | |
LDP | LDP-Fed | 无 | 93.70 | 1.4 | |
PDP | PDP-FL | 场景1 | 93.90 | 1.3 | |
场景2 | 1.2 | ||||
场景3 | 93.80 | 1.3 | |||
场景4 | 94.00 | 1.3 | |||
CIFAR-10 | 无 | FedAvg | 无 | 46.01 | 1.2 |
GDP | GDP-FL | 无 | 44.22 | 1.5 | |
LDP | LDP-Fed | 无 | 43.60 | 1.4 | |
PDP | PDP-FL | 场景1 | 43.55 | 1.4 | |
场景2 | 1.2 | ||||
场景3 | 43.40 | 1.4 | |||
场景4 | 44.10 | 1.3 |
1 | LÓPEZ K L, GAGNÉ C, GARDNER M A. Demand-side management using deep learning for smart charging of electric vehicles[J]. IEEE Transactions on Smart Grid, 2019, 10(3): 2683-2691. 10.1109/tsg.2018.2808247 |
2 | LIN W Y, HU Y H, TSAI C F. Machine learning in financial crisis prediction: a survey[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2012, 42(4): 421-436. 10.1109/tsmcc.2011.2170420 |
3 | CHEN S Z, YU D X, ZOU Y F, et al. Decentralized wireless federated learning with differential privacy[J]. IEEE Transactions on Industrial Informatics, 2022, 18(9):6273-6282. 10.1109/tii.2022.3145010 |
4 | BARRENO M, NELSON B, JOSEPH A D, et al. The security of machine learning[J]. Machine Learning, 2010, 81(2): 121-148. 10.1007/s10994-010-5188-5 |
5 | 张梅舒,徐雅斌. 多维数值型敏感属性数据的个性化隐私保护方法 [J]. 计算机应用, 2020, 40(2):491-496. |
ZHANG M S, XU Y B. Personalized privacy protection method for data with multiple numerical sensitive attributes[J]. Journal of Computer Applications, 2020, 40(2): 491-496. | |
6 | 刘艺璇,陈红,刘宇涵,等. 联邦学习中的隐私保护技术[J]. 软件学报, 2022, 33(3):1057-1092. |
LIU Y X, CHEN H, LIU Y H, et al. Privacy-preserving techniques in federal learning[J]. Journal of Software, 2022, 33(3): 1057-1092. | |
7 | WU N, FAROKHI F, SMITH D, et al. The value of collaboration in convex machine learning with differential privacy[C]// Proceedings of the 2020 IEEE Symposium on Security and Privacy. Piscataway: IEEE, 2020: 304-317. 10.1109/sp40000.2020.00025 |
8 | WANG B L, YAO Y S, SHAN S, et al. Neural cleanse: identifying and mitigating backdoor attacks in neural networks[C] // Proceedings of the 2019 IEEE Symposium on Security and Privacy. Piscataway: IEEE, 2019: 707-723. 10.1109/sp.2019.00031 |
9 | YUAN X Y, HE P, ZHU Q L, et al. Adversarial examples: attacks and defenses for deep learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(9): 2805-2824. 10.1109/tnnls.2018.2886017 |
10 | YUAN J W, YU S C. Privacy preserving back-propagation neural network learning made practical with cloud computing[J]. IEEE Transactions on Parallel and Distributed Systems, 2014, 25(1): 212-221. 10.1109/tpds.2013.18 |
11 | BONAWITZ K, IVANOV V, KREUTER B, et al. Practical secure aggregation for privacy-preserving machine learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2017: 1175-1191. 10.1145/3133956.3133982 |
12 | 邱鑫源,叶泽聪,崔翛龙,等. 联邦学习通信开销研究综述[J]. 计算机应用, 2022, 42(2): 333-342. 10.11772/j.issn.1001-9081.2021020232 |
QIU X Y, YE Z C, CUI X L, et al. Survey of communication overhead in federated learning[J]. Journal of Computer Applications, 2022, 42(2): 333-342. 10.11772/j.issn.1001-9081.2021020232 | |
13 | DWORK C, ROTH A. The algorithmic foundations of differential privacy[J]. Foundations and Trends® in Theoretical Computer Science, 2013, 9(3/4): 211-407. 10.1561/0400000042 |
14 | ZHANG J L, ZHAO Y C, WANG J Y, et al. FedMEC: improving efficiency of differentially private federated learning via mobile edge computing[J]. Mobile Networks and Applications, 2020, 25(6): 2421-2433. 10.1007/s11036-020-01586-4 |
15 | TRUEX S, LIU L, CHOW K H, et al. LDP-Fed: federated learning with local differential privacy[C]// Proceedings of the 3rd ACM International Workshop on Edge Systems, Analytics and Networking. New York: ACM, 2020: 61-66. 10.1145/3378679.3394533 |
16 | ZHAO Y, ZHAO J, YANG M M, et al. Local differential privacy-based federated learning for Internet of Things[J]. IEEE Internet of Things Journal, 2021, 8(11): 8836-8853. 10.1109/jiot.2020.3037194 |
17 | WU X, ZHANG Y T, SHI M Y, et al. An adaptive federated learning scheme with differential privacy preserving[J]. Future Generation Computer Systems, 2022, 127: 362-372. 10.1016/j.future.2021.09.015 |
18 | PHONG L T, AONO Y, HAYASHI T, et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(5): 1333-1345. 10.1109/tifs.2017.2787987 |
19 | SONG J C, WANG W Z, GADEKALLU T R, et al. EPPDA: an efficient privacy-preserving data aggregation federated learning scheme[J]. IEEE Transactions on Network Science and Engineering, 2022(Early Access): 1-1. 10.1109/tnse.2022.3153519 |
20 | GONG M G, FENG J L, XIE Y. Privacy-enhanced multi-party deep learning[J]. Neural Networks, 2020, 121: 484-496. 10.1016/j.neunet.2019.10.001 |
21 | McMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York: JMLR.org, 2017: 1273-1282. |
22 | ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy[C]// Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2016: 308-318. 10.1145/2976749.2978318 |
23 | WANG S W, HUANG L S, NIE Y W, et al. Local differential private data aggregation for discrete distribution estimation[J]. IEEE Transactions on Parallel and Distributed Systems, 2019, 30(9): 2046-2059. 10.1109/tpds.2019.2899097 |
24 | ARACHCHIGE P C M, BERTOK P, KHALIL I, et al. Local differential privacy for deep learning[J]. IEEE Internet of Things Journal, 2020, 7(7): 5827-5842. 10.1109/jiot.2019.2952146 |
25 | WANG N, XIAO X K, YANG Y, et al. Collecting and analyzing multidimensional data with local differential privacy[C]// Proceedings of the IEEE 35th International Conference on Data Engineering. Piscataway: IEEE, 2019: 638-649. 10.1109/icde.2019.00063 |
26 | GEYER R C, KLEIN T, NABI M. Differentially private federated learning: a client level perspective[EB/OL]. (2018-03-01) [2022-03-05].. |
27 | WEI K, LI J, DING M, et al. Federated learning with differential privacy: algorithms and performance analysis[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 3454-3469. 10.1109/tifs.2020.2988575 |
28 | WU S H, YU M Q, AHMED M A M, et al. FL-MAC-RDP: federated learning over multiple access channels with Rényi differential privacy[J]. International Journal of Theoretical Physics, 2021, 60(7): 2668-2682. 10.1007/s10773-021-04867-0 |
29 | LIU R, CAO Y, YOSHIKAWA M, et al. FedSel: federated SGD under local differential privacy with top-k dimension selection[C]// Proceedings of the 2020 International Conference on Database Systems for Advanced Applications. Cham: Springer, 2020: 485-501. 10.1007/978-3-030-59410-7_33 |
30 | 莫慧凌,郑海峰,高敏,等. 基于联邦学习的多源异构数据融合算法[J]. 计算机研究与发展, 2022, 59(2):478-487. 10.7544/issn1000-1239.20200668 |
MO H L, ZHEN H F, GAO M, et al. Multi-source heterogeneous data fusion based on federated learning[J]. Journal of Computer Research and Development, 2022, 59(2): 478-487. 10.7544/issn1000-1239.20200668 | |
31 | ZHAO L C, WANG Q, ZOU Q, et al. Privacy-preserving collaborative deep learning with unreliable participants[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 1486-1500. 10.1109/tifs.2019.2939713 |
32 | GIRGIS A, DATA D, DIGGAVI S, et al. Shuffled model of differential privacy in federated learning[C]// Proceedings of the 24th International Conference on Artificial Intelligence and Statistics. New York: JMLR.org, 2021: 2521-2529. 10.1109/jsait.2021.3056102 |
33 | ZHANG L F, ZHU T Q, XIONG P, et al. A robust game-theoretical federated learning framework with joint differential privacy[J]. IEEE Transactions on Knowledge and Data Engineering, 2022(Early Access): 1-1. 10.1109/tkde.2021.3140131 |
34 | AVENT B, KOROLOVA A, ZEBER D, et al. BLENDER: enabling local search with a hybrid differential privacy model[C]// Proceedings of the 26th USENIX Security Symposium. Berkeley: USENIX Association, 2017: 747-764. 10.29012/jpc.680 |
35 | HU R, GUO Y X, LI H N, et al. Personalized federated learning with differential privacy[J]. IEEE Internet of Things Journal, 2020, 7(10): 9530-9539. 10.1109/jiot.2020.2991416 |
36 | YANG G, WANG S W, WANG H J. Federated learning with personalized local differential privacy[C]// Proceedings of the IEEE 6th International Conference on Computer and Communication Systems. Piscataway: IEEE, 2021: 484-489. 10.1109/icccs52626.2021.9449232 |
37 | LI H R, XIONG L, JI Z L, et al. Partitioning-based mechanisms under personalized differential privacy[C]// Proceedings of the 2017 Pacific-Asia Conference on Knowledge Discovery and Data Mining, LNCS 10234. Cham: Springer, 2017: 615-627. |
38 | INAN A, GURSOY M E, SAYGIN Y. Sensitivity analysis for non-interactive differential privacy: bounds and efficient algorithms[J]. IEEE Transactions on Dependable and Secure Computing, 2020, 17(1): 194-207. 10.1109/tdsc.2017.2734664 |
39 | JORGENSEN Z, YU T, CORMODE G. Conservative or liberal? personalized differential privacy[C]// Proceedings of the IEEE 31st International Conference on Data Engineering. Piscataway: IEEE, 2015: 1023-1034. 10.1109/icde.2015.7113353 |
40 | ZHANG Y P, QU Y Y, GAO L X, et al. GPDP: game-enhanced personalized differentially private smart community[C]// Proceedings of the 2021 IEEE International Conferences on Internet of Things and IEEE Green Computing and Communications and IEEE Cyber, Physical and Social Computing and IEEE Smart Data and IEEE Congress on Cybermatics. Piscataway: IEEE, 2021: 238-243. 10.1109/ithings-greencom-cpscom-smartdata-cybermatics53846.2021.00048 |
41 | LeCUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324. 10.1109/5.726791 |
42 | KRIZHEVSKY A. Learning multiple layers of features from tiny images[R/OL]. (2009-04-08) [2022-03-05].. 10.1016/j.tics.2007.09.004 |
[1] | Zheyuan SHEN, Keke YANG, Jing LI. Personalized federated learning method based on dual stream neural network [J]. Journal of Computer Applications, 2024, 44(8): 2319-2325. |
[2] | Jie WU, Xuezhong QIAN, Wei SONG. Personalized federated learning based on similarity clustering and regularization [J]. Journal of Computer Applications, 2024, 44(11): 3345-3353. |
[3] | Xuebin CHEN, Changsheng QU. Overview of backdoor attacks and defense in federated learning [J]. Journal of Computer Applications, 2024, 44(11): 3459-3469. |
[4] | Chunyong YIN, Yongcheng ZHOU. Automatically adjusted clustered federated learning for double-ended clustering [J]. Journal of Computer Applications, 2024, 44(10): 3011-3020. |
[5] | Hui ZHOU, Yuling CHEN, Xuewei WANG, Yangwen ZHANG, Jianjiang HE. Deep shadow defense scheme of federated learning based on generative adversarial network [J]. Journal of Computer Applications, 2024, 44(1): 223-232. |
[6] | Mengjie LAN, Jianping CAI, Lan SUN. Self-regularization optimization methods for Non-IID data in federated learning [J]. Journal of Computer Applications, 2023, 43(7): 2073-2081. |
[7] | Wanzhen CHEN, En ZHANG, Leiyong QIN, Shuangxi HONG. Privacy-preserving federated learning algorithm based on blockchain in edge computing [J]. Journal of Computer Applications, 2023, 43(7): 2209-2216. |
[8] | Qian CHEN, Zheng CHAI, Zilong WANG, Jiawei CHEN. Poisoning attack detection scheme based on generative adversarial network for federated learning [J]. Journal of Computer Applications, 2023, 43(12): 3790-3798. |
[9] | Lei TIAN, Lina GE. Advertising recommendation algorithm based on differential privacy [J]. Journal of Computer Applications, 2023, 43(11): 3346-3350. |
[10] | ZHENG Sai, LI Tianrui, HUANG Wei. Federated learning algorithm for communication cost optimization [J]. Journal of Computer Applications, 2023, 43(1): 1-7. |
[11] | Zhenyu ZHANG, Guoping TAN, Siyuan ZHOU. Efficient wireless federated learning algorithm based on 1‑bit compressive sensing [J]. Journal of Computer Applications, 2022, 42(6): 1675-1682. |
[12] | Li’e WANG, Xiaocong LI, Hongyi LIU. News recommendation method with knowledge graph and differential privacy [J]. Journal of Computer Applications, 2022, 42(5): 1339-1346. |
[13] | Rui SUN, Chao LI, Wei WANG, Endong TONG, Jian WANG, Jiqiang LIU. Research progress of blockchain‑based federated learning [J]. Journal of Computer Applications, 2022, 42(11): 3413-3420. |
[14] | LI Kunming, WANG Chaoqian, NI Weiwei, BAO Xiaohan. High-precision histogram publishing method based on differential privacy [J]. Journal of Computer Applications, 2020, 40(11): 3242-3248. |
[15] | LI Yanhui, LIU Hao, YUAN Ye, WANG Guoren. Frequent sequence pattern mining with differential privacy [J]. Journal of Computer Applications, 2017, 37(2): 316-321. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||