Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (10): 3011-3020.DOI: 10.11772/j.issn.1001-9081.2023101475
• Artificial intelligence • Previous Articles Next Articles
Chunyong YIN1(), Yongcheng ZHOU2
Received:
2023-11-02
Revised:
2024-02-23
Accepted:
2024-02-26
Online:
2024-10-15
Published:
2024-10-10
Contact:
Chunyong YIN
About author:
ZHOU Yongcheng, born in 2000, M. S. candidate. His research interests include federated learning.
Supported by:
通讯作者:
尹春勇
作者简介:
尹春勇(1977—),男,山东潍坊人,教授,博士生导师,博士,主要研究方向:网络空间安全、大数据挖掘及隐私保护、人工智能及新型计算 yinchunyong@hotmail.com基金资助:
CLC Number:
Chunyong YIN, Yongcheng ZHOU. Automatically adjusted clustered federated learning for double-ended clustering[J]. Journal of Computer Applications, 2024, 44(10): 3011-3020.
尹春勇, 周永成. 双端聚类的自动调整聚类联邦学习[J]. 《计算机应用》唯一官方网站, 2024, 44(10): 3011-3020.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023101475
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 70.18 | 48.93 | 81.46 |
FeSEM | 78.24 | 55.41 | 84.80 |
WeCFL | 78.46 | 55.76 | 85.10 |
IFCA | 79.15 | 57.78 | 85.79 |
CFL | 79.14 | 55.57 | 84.82 |
AACFL | 79.20 | 56.85 | 86.13 |
Tab. 1 Accuracy evaluation of different algorithms under k=2, n=50
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 70.18 | 48.93 | 81.46 |
FeSEM | 78.24 | 55.41 | 84.80 |
WeCFL | 78.46 | 55.76 | 85.10 |
IFCA | 79.15 | 57.78 | 85.79 |
CFL | 79.14 | 55.57 | 84.82 |
AACFL | 79.20 | 56.85 | 86.13 |
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 56.19 | 42.62 | 72.82 |
FeSEM | 74.20 | 53.12 | 82.13 |
WeCFL | 74.87 | 53.68 | 82.68 |
IFCA | 78.97 | 53.20 | 85.64 |
CFL | 78.87 | 56.67 | 85.53 |
AACFL | 79.35 | 58.29 | 85.84 |
Tab. 2 Accuracy evaluation of different algorithms under k=4, n=100
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 56.19 | 42.62 | 72.82 |
FeSEM | 74.20 | 53.12 | 82.13 |
WeCFL | 74.87 | 53.68 | 82.68 |
IFCA | 78.97 | 53.20 | 85.64 |
CFL | 78.87 | 56.67 | 85.53 |
AACFL | 79.35 | 58.29 | 85.84 |
学习率 | 准确率/% | ||
---|---|---|---|
EMNIST | CIFAR-10 | Fashion-MNIST | |
0.10 | 78.28 | 57.44 | 86.76 |
0.07 | 78.12 | 57.34 | 86.06 |
0.04 | 77.26 | 56.90 | 84.46 |
0.01 | 72.89 | 50.49 | 80.46 |
Tab. 3 Accuracy evaluation of AACFL under different learning rates
学习率 | 准确率/% | ||
---|---|---|---|
EMNIST | CIFAR-10 | Fashion-MNIST | |
0.10 | 78.28 | 57.44 | 86.76 |
0.07 | 78.12 | 57.34 | 86.06 |
0.04 | 77.26 | 56.90 | 84.46 |
0.01 | 72.89 | 50.49 | 80.46 |
qκ | EMNIST | CIFAR-10 | Fashion-MNIST | |||
---|---|---|---|---|---|---|
ARI | times | ARI | times | ARI | times | |
0.4 | 0.82 | 32 | 0.79 | 12 | 0.94 | 10 |
0.6 | 0.92 | 29 | 0.89 | 16 | 0.94 | 4 |
0.8 | 0.97 | 9 | 0.87 | 20 | 0.94 | 7 |
1.0 | 0.97 | 11 | 0.97 | 5 | 0.97 | 3 |
Tab. 4 ARI evaluation and round consumption evaluation of AACFL with different clustered client participation rates
qκ | EMNIST | CIFAR-10 | Fashion-MNIST | |||
---|---|---|---|---|---|---|
ARI | times | ARI | times | ARI | times | |
0.4 | 0.82 | 32 | 0.79 | 12 | 0.94 | 10 |
0.6 | 0.92 | 29 | 0.89 | 16 | 0.94 | 4 |
0.8 | 0.97 | 9 | 0.87 | 20 | 0.94 | 7 |
1.0 | 0.97 | 11 | 0.97 | 5 | 0.97 | 3 |
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 2 146 | 5 415 | 2 812 |
FeSEM | 2 567 | 5 897 | 3 125 |
WeCFL | 2 583 | 5 812 | 3 217 |
IFCA | 4 054 | 8 240 | 4 545 |
CFL | 2 348 | 5 755 | 3 064 |
AACFL | 2 787 | 5 990 | 3 364 |
Tab. 5 Time consumption of different algorithms under k=2, n=50
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 2 146 | 5 415 | 2 812 |
FeSEM | 2 567 | 5 897 | 3 125 |
WeCFL | 2 583 | 5 812 | 3 217 |
IFCA | 4 054 | 8 240 | 4 545 |
CFL | 2 348 | 5 755 | 3 064 |
AACFL | 2 787 | 5 990 | 3 364 |
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 5 116 | 9 214 | 5 042 |
FeSEM | 5 623 | 9 845 | 5 543 |
WeCFL | 5 711 | 9 834 | 5 604 |
IFCA | 7 132 | 14 564 | 7 465 |
CFL | 5 521 | 9 664 | 5 402 |
AACFL | 5 831 | 10 168 | 5 799 |
Tab. 6 Time consumption of different algorithms under k=4, n=100
算法 | EMNIST | CIFAR-10 | Fashion-MNIST |
---|---|---|---|
FedAvg | 5 116 | 9 214 | 5 042 |
FeSEM | 5 623 | 9 845 | 5 543 |
WeCFL | 5 711 | 9 834 | 5 604 |
IFCA | 7 132 | 14 564 | 7 465 |
CFL | 5 521 | 9 664 | 5 402 |
AACFL | 5 831 | 10 168 | 5 799 |
1 | McMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York: JMLR.org, 2017: 1273-1282. |
2 | LI X, HUANG K X, YANG W H, et al. On the convergence of FedAvg on Non-IID data[EB/OL]. (2020-06-25) [2023-09-12].. |
3 | YANG H, FANG M, LIU J. Achieving linear speedup with partial worker participation in Non-IID federated learning[EB/OL]. (2021-05-04) [2023-09-12].. |
4 | DUAN M, LIU D, CHEN X, et al. Astraea: self-balancing federated learning for improving classification accuracy of mobile deep learning applications[C]// Proceedings of the 2019 IEEE 37th International Conference on Computer Design. Piscataway: IEEE, 2019: 246-254. |
5 | 罗长银,陈学斌,刘洋,等. 基于联邦集成算法对多源数据安全性的研究[J]. 计算机工程与科学, 2021, 43(8):1387-1397. |
LUO C Y, CHEN X B, LIU Y, et al. A federated ensemble algorithm for multi-source data security[J]. Computer Engineering and Science, 2021, 43(8):1387-1397. | |
6 | HAO M, LI H, XU G, et al. Towards efficient and privacy-preserving federated deep learning[C]// Proceedings of the 2019 IEEE International Conference on Communications. Piscataway: IEEE, 2019: 1-6. |
7 | FANG C, GUO Y, WANG N, et al. Highly efficient federated learning with strong privacy preservation in cloud computing[J]. Computers and Security, 2020, 96: No.101889. |
8 | LI Q, WEN Z, WU Z, et al. A survey on federated learning systems: vision, hype and reality for data privacy and protection[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 35(4): 3347-3366. |
9 | HARD A, RAO K, MATHEWS R, et al. Federated learning for mobile keyboard prediction[EB/OL]. (2019-02-28) [2023-09-12].. |
10 | BRISIMI T S, CHEN R, MELA T, et al. Federated learning of predictive models from federated electronic health records[J]. International Journal of Medical Informatics, 2018, 112: 59-67. |
11 | LI T, SAHU A K, TALWALKAR A, et al. Federated learning: challenges, methods, and future directions[J]. IEEE Signal Processing Magazine, 2020, 37(3): 50-60. |
12 | JIN H, BAI D, YAO D, et al. Personalized edge intelligence via federated self-knowledge distillation[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(2): 567-580. |
13 | LI H, CAI Z, WANG J, et al. FedTP: federated learning by Transformer personalization[EB/OL]. (2023-04-18) [2023-09-12].. |
14 | SATTLER F, MÜLLER K R, SAMEK W. Clustered federated learning: model-agnostic distributed multitask optimization under privacy constraints[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(8): 3710-3722. |
15 | SATTLER F, MÜLLER K R, WIEGAND T, et al. On the Byzantine robustness of clustered federated learning[C]// Proceedings of the 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2020: 8861-8865. |
16 | DENNIS D K, LI T, SMITH V. Heterogeneity for the win: one-shot federated clustering[C]// Proceedings of the 38th International Conference on Machine Learning. New York: JMLR.org, 2021: 2611-2620. |
17 | GHOSH A, CHUNG J, YIN D, et al. An efficient framework for clustered federated learning[J]. IEEE Transactions on Information Theory, 2022, 68(12): 8076-8091. |
18 | BRIGGS C, FAN Z, ANDRAS P. Federated learning with hierarchical clustering of local updates to improve training on non-IID data[C]// Proceedings of the 2020 International Joint Conference on Neural Networks. Piscataway: IEEE, 2020: 1-9. |
19 | LI C, LI G, VARSHNEY P K. Federated learning with soft clustering[J]. IEEE Internet of Things Journal, 2022, 9(10): 7773-7782. |
20 | DUAN M, LIU D, CHEN X, et al. Self-balancing federated learning with global imbalanced data in mobile systems[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(1): 59-71. |
21 | LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[EB/OL]. [2023-09-12].. |
22 | MOTHUKURI V, PARIZI R M, POURIYEH S, et al. A survey on security and privacy of federated learning[J]. Future Generation Computer Systems, 2021, 115: 619-640. |
23 | ZHAO Y, LI M, LAI L, et al. Federated learning with non-IID data[EB/OL]. (2022-07-21) [2023-09-12].. |
24 | LU R, ZHANG W, WANG Y, et al. Auction-based cluster federated learning in mobile edge computing systems[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(4): 1145-1158. |
25 | DUAN M, LIU D, JI X, et al. Flexible clustered federated learning for client-level data distribution shift[J]. IEEE Transactions on Parallel and Distributed Systems, 2022, 33(11): 2661-2674. |
26 | ZHANG Y, LIU D, DUAN M, et al. FedMDS: an efficient model discrepancy-aware semi-asynchronous clustered federated learning framework[J]. IEEE Transactions on Parallel and Distributed Systems, 2023, 34(3): 1007-1019. |
27 | LLOYD S. Least squares quantization in PCM[J]. IEEE Transactions on Information Theory, 1982, 28(2): 129-137. |
28 | RUAN Y, JOE-WONG C. FedSoft: soft clustered federated learning with proximal local updating[C]// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2022: 8124-8131. |
29 | TIAN P, LIAO W X, YU W, et al. WSCC: a weight-similarity-based client clustering approach for non-IID federated learning[J]. IEEE Internet of Things Journal, 2022, 9(20): 20243-20256. |
30 | LONG G, XIE M, SHEN T, et al. Multi-center federated learning: clients clustering for better personalization[J]. World Wide Web, 2023, 26: 481-500. |
31 | AGRAWAL S, SARKAR S, ALAZAB M, et al. Genetic CFL: hyperparameter optimization in clustered federated learning[J]. Computational Intelligence and Neuroscience, 2021, 2021: No.7156420. |
32 | 鲁晨阳,邓苏,马武彬,等. 基于DBSCAN聚类的集群联邦学习方法[J]. 计算机科学, 2022, 49(6A):232-237. |
LU C Y, DENG S, MA W B, et al. Clustered federated learning methods based on DBSCAN clustering[J]. Computer Science, 2022, 49(6A):232-237. | |
33 | 常黎明,刘颜红,徐恕贞. 基于数据分布的聚类联邦学习[J]. 计算机应用研究, 2023, 40(6):1697-1701. |
CHANG L M, LIU Y H, XU S Z. Clustering federated learning based on data distribution[J]. Application Research of Computers, 2023, 40(6):1697-1701. | |
34 | STALLMANN M, WILBIK A. Towards federated clustering: a Federated Fuzzy c-Means algorithm (FFCM)[EB/OL]. (2022-01-18) [2023-09-12].. |
35 | XIE H, MA J, XIONG L, et al. Federated graph classification over non-IID graphs[C]// Proceedings of the 35th Conference on Neural Information Processing Systems. New York: ACM, 2024: 18839-18852. |
36 | COHEN G, AFSHAR S, TAPSON J, et al. EMNIST: extending MNIST to handwritten letters[C]// Proceedings of the 2017 International Joint Conference on Neural Networks. Piscataway: IEEE, 2017: 2921-2926. |
37 | KRIZHEVSKY A. Learning multiple layers of features from tiny images[D]. Toronto: University of Toronto, 2009: 1-60. |
38 | XIAO H, RASUL K, VOLLGRAF R. Fashion-MNIST: a novel image dataset for benchmarking machine learning algorithms[EB/OL]. (2017-09-15) [2023-09-12].. |
39 | MA J, LONG G, ZHOU T, et al. On the convergence of clustered federated learning[EB/OL]. (2022-06-07) [2023-09-12].. |
[1] | Xingyao YANG, Yu CHEN, Jiong YU, Zulian ZHANG, Jiaying CHEN, Dongxiao WANG. Recommendation model combining self-features and contrastive learning [J]. Journal of Computer Applications, 2024, 44(9): 2704-2710. |
[2] | Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703. |
[3] | Na WANG, Lin JIANG, Yuancheng LI, Yun ZHU. Optimization of tensor virtual machine operator fusion based on graph rewriting and fusion exploration [J]. Journal of Computer Applications, 2024, 44(9): 2802-2809. |
[4] | Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910. |
[5] | Tingjie TANG, Jiajin HUANG, Jin QIN. Session-based recommendation with graph auxiliary learning [J]. Journal of Computer Applications, 2024, 44(9): 2711-2718. |
[6] | Rui ZHANG, Pengyun ZHANG, Meirong GAO. Self-optimized dual-modal multi-channel non-deep vestibular schwannoma recognition model [J]. Journal of Computer Applications, 2024, 44(9): 2975-2982. |
[7] | Jinjin LI, Guoming SANG, Yijia ZHANG. Multi-domain fake news detection model enhanced by APK-CNN and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2674-2682. |
[8] | Guanglei YAO, Juxia XIONG, Guowu YANG. Flower pollination algorithm based on neural network optimization [J]. Journal of Computer Applications, 2024, 44(9): 2829-2837. |
[9] | Ying HUANG, Jiayu YANG, Jiahao JIN, Bangrui WAN. Siamese mixed information fusion algorithm for RGBT tracking [J]. Journal of Computer Applications, 2024, 44(9): 2878-2885. |
[10] | Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974. |
[11] | Hang YANG, Wanggen LI, Gensheng ZHANG, Zhige WANG, Xin KAI. Multi-layer information interactive fusion algorithm based on graph neural network for session-based recommendation [J]. Journal of Computer Applications, 2024, 44(9): 2719-2725. |
[12] | Yu DU, Yan ZHU. Constructing pre-trained dynamic graph neural network to predict disappearance of academic cooperation behavior [J]. Journal of Computer Applications, 2024, 44(9): 2726-2731. |
[13] | Yubo ZHAO, Liping ZHANG, Sheng YAN, Min HOU, Mao GAO. Relation extraction between discipline knowledge entities based on improved piecewise convolutional neural network and knowledge distillation [J]. Journal of Computer Applications, 2024, 44(8): 2421-2429. |
[14] | Chunxue ZHANG, Liqing QIU, Cheng’ai SUN, Caixia JING. Purchase behavior prediction model based on two-stage dynamic interest recognition [J]. Journal of Computer Applications, 2024, 44(8): 2365-2371. |
[15] | Tong CHEN, Fengyu YANG, Yu XIONG, Hong YAN, Fuxing QIU. Construction method of voiceprint library based on multi-scale frequency-channel attention fusion [J]. Journal of Computer Applications, 2024, 44(8): 2407-2413. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||