Federated learning fairness algorithm based on personalized submodel and K-means clustering

doi:10.11772/j.issn.1001-9081.2024121794

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (12): 3747-3756.DOI: 10.11772/j.issn.1001-9081.2024121794

• Artificial intelligence • Next Articles

Federated learning fairness algorithm based on personalized submodel and K-means clustering

Zhongrui JING¹^,²^,³, Xuebin CHEN¹^,²^,³, Yinlong JIAN¹^,²^,³, Qi ZHONG¹^,²^,³, Zhenbo ZHANG¹^,²^,³

^1.College of Science，North China University of Science and Technology，Tangshan Hebei 063210，China
^2.Hebei Provincial Key Laboratory of Data Science and Application （North China University of Science and Technology），Tangshan Hebei 063210，China
^3.Tangshan Key Laboratory of Data Science （North China University of Science and Technology），Tangshan Hebei 063210，China

Received:2024-12-20 Revised:2025-03-13 Accepted:2025-03-18 Online:2025-03-27 Published:2025-12-10
Contact: Xuebin CHEN
About author:JING Zhongrui， born in 2000， M. S. candidate. His research interests include data security， privacy protection.
CHEN Xuebin， born in 1970， Ph. D.， professor. His research interests include big data security， internet of things security， network security.
JIAN Yinlong， born in 2001， M. S. candidate. His research interests include data security， privacy protection.
ZHONG Qi， born in 1999， M. S. candidate. Her research interests include data security， privacy protection.
ZHANG Zhenbo， born in 1999， M. S. candidate. His research interests include data security， privacy protection.
Supported by:
National Natural Science Foundation of China(U20A20179)

基于个性化子模型和K均值聚类的联邦学习公平性算法

景忠瑞¹^,²^,³, 陈学斌¹^,²^,³, 菅银龙¹^,²^,³, 钟琪¹^,²^,³, 张镇博¹^,²^,³

^1.华北理工大学理学院，河北唐山 063210
^2.河北省数据科学与应用重点实验室（华北理工大学），河北唐山 063210
^3.唐山市数据科学重点实验室（华北理工大学），河北唐山 063210

通讯作者: 陈学斌
作者简介:景忠瑞（2000—），男，山西临汾人，硕士研究生，CCF会员，主要研究方向：数据安全、隐私保护
陈学斌（1970—），男，河北唐山人，教授，博士，CCF杰出会员，主要研究方向：大数据安全、物联网安全、网络安全
菅银龙（2001—），男，河南商丘人，硕士研究生，CCF会员，主要研究方向：数据安全、隐私保护
钟琪（1999—），女，河北张家口人，硕士研究生，CCF会员，主要研究方向：数据安全、隐私保护
张镇博（1999—），男，山东济南人，硕士研究生，CCF会员，主要研究方向：数据安全、隐私保护。
基金资助:
国家自然科学基金资助项目(U20A20179)

Abstract

Abstract:

Traditional Federated Learning （FL） does not consider collaborative fairness， leading to a mismatch between the reward obtained by the client and its actual contribution. To address this issue， a Federated learning fairness algorithm based on Personalized Submodel and K-means clustering （FedPSK） was proposed. Firstly， the neurons in the neural network were clustered according to their activation patterns， and only the importance of the cluster center neurons after clustering was evaluated. And the scores of the cluster center neurons were used to represent the scores of other neurons in the cluster， which reduced the time consumption of neuron evaluation. Then， the number of neurons and their labeling included in the client submodel were selected through hierarchical selection method， and a submodel with a complete neural network structure was constructed for each client. Finally， collaborative fairness was achieved by distributing submodels to the clients. Experimental results on different datasets show that FedPSK improves the correlation coefficient of fairness measurement by 2.70% compared with FedSAC （Federated learning framework with dynamic Submodel Allocation for Collaborative fairness）. In terms of time overhead， FedPSK reduces by at least 84.12% compared with FedSAC. It can be seen that FedPSK improves the fairness of FL algorithm， and reduces the time overhead of the algorithm execution greatly， verifying the efficiency of the proposed algorithm.

Key words: Federated Learning (FL), collaborative fairness, K-means clustering, submodel distribution, personalization

摘要：

传统联邦学习（FL）未考虑协作公平性，导致客户端获得的奖励与它的实际贡献不匹配。针对这一问题，提出一种基于个性化子模型和K均值聚类的联邦学习公平性算法（FedPSK）。首先，根据神经网络中神经元的激活模式对神经元聚类，且仅对聚类后的簇中心神经元进行重要性评估，并使用簇中心神经元的评分代表簇中其他神经元的评分，从而降低神经元评估的耗时；其次，使用层次选取方式选择客户端子模型中包含的神经元数量及编号，并为每个客户端建立具有完整神经网络结构的子模型；最后，通过为客户端下发子模型，实现协作公平性。在不同数据集上的实验结果表明，在公平度量的相关系数方面，FedPSK比FedSAC（Federated learning framework with dynamic Submodel Allocation for Collaborative fairness）提高了2.70%；在时间开销方面，FedPSK比FedSAC至少降低了84.12%。可见，FedPSK在提升FL算法公平性的同时，极大地降低了算法运行的时间开销，验证了所提算法的高效性。

关键词: 联邦学习, 协作公平性, K均值聚类, 子模型下发, 个性化

CLC Number:

TP309.2

Zhongrui JING, Xuebin CHEN, Yinlong JIAN, Qi ZHONG, Zhenbo ZHANG. Federated learning fairness algorithm based on personalized submodel and K-means clustering[J]. Journal of Computer Applications, 2025, 45(12): 3747-3756.

景忠瑞, 陈学斌, 菅银龙, 钟琪, 张镇博. 基于个性化子模型和K均值聚类的联邦学习公平性算法[J]. 《计算机应用》唯一官方网站, 2025, 45(12): 3747-3756.

Figures/Tables 9

Fig. 1 Framework of FedPSK

Fig. 2 Schematic diagram of heterogeneous client data

Fig. 3 Partial activation patterns of top-10 neurons in cluster 0

Tab. 1 Comparison of maximum accuracies and correlation coefficients of fairness measurement of different algorithms on CIFAR-10 dataset

算法	pat		dir1.0		dir2.0		dir3.0
算法	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$
FedAvg	42.12	39.46	35.35	86.99	43.86	-11.95	56.12	40.44
FedProx	41.97	40.06	35.14	87.44	43.76	-12.27	56.16	41.32
FedAVE	31.56	94.80	39.21	78.51	46.15	82.76	58.56	74.91
FedSAC	42.32	99.28	40.37	98.61	41.33	97.12	56.23	99.62
FedPSK	44.46	99.32	36.59	99.01	47.64	99.74	58.86	98.25

Tab. 1 Comparison of maximum accuracies and correlation coefficients of fairness measurement of different algorithms on CIFAR-10 dataset

算法	pat		dir1.0		dir2.0		dir3.0
算法	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$
FedAvg	42.12	39.46	35.35	86.99	43.86	-11.95	56.12	40.44
FedProx	41.97	40.06	35.14	87.44	43.76	-12.27	56.16	41.32
FedAVE	31.56	94.80	39.21	78.51	46.15	82.76	58.56	74.91
FedSAC	42.32	99.28	40.37	98.61	41.33	97.12	56.23	99.62
FedPSK	44.46	99.32	36.59	99.01	47.64	99.74	58.86	98.25

Fig. 4 Accuracy comparison of different algorithms on CIFAR-10 dataset

Tab. 2 Comparison of maximum accuracies and correlation coefficients of fairness measurement of different algorithms on Fashion-MNIST dataset

算法	pat		dir1.0		dir2.0		dir3.0
算法	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$
FedAvg	76.53	-37.25	76.54	50.74	77.03	66.52	90.69	62.17
FedProx	76.57	-37.47	76.35	50.72	76.97	67.86	90.73	61.94
FedAVE	74.76	77.58	72.90	98.88	82.73	72.62	83.23	81.96
FedSAC	75.16	98.36	77.82	99.35	79.69	99.70	91.61	99.47
FedPSK	76.00	99.33	82.78	99.39	80.07	99.73	93.02	97.81

Tab. 2 Comparison of maximum accuracies and correlation coefficients of fairness measurement of different algorithms on Fashion-MNIST dataset

算法	pat		dir1.0		dir2.0		dir3.0
算法	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$	最大准确率/%	$γ$
FedAvg	76.53	-37.25	76.54	50.74	77.03	66.52	90.69	62.17
FedProx	76.57	-37.47	76.35	50.72	76.97	67.86	90.73	61.94
FedAVE	74.76	77.58	72.90	98.88	82.73	72.62	83.23	81.96
FedSAC	75.16	98.36	77.82	99.35	79.69	99.70	91.61	99.47
FedPSK	76.00	99.33	82.78	99.39	80.07	99.73	93.02	97.81

Fig. 5 Accuracy curves of client submodel on CIFAR-10 dataset

Tab. 3 Neuron evaluation time consumption for neural networks of different sizes

算法	评估耗时/s
算法	神经网络1 （154）	神经网络2 （618）	神经网络3 （1 130）
FedPSK（ $m$ =7）	13.06	14.06	18.15
FedPSK（ $m$ =9）	15.76	16.73	19.57
FedSAC	99.24	436.24	758.24

Tab. 3 Neuron evaluation time consumption for neural networks of different sizes

算法	评估耗时/s
算法	神经网络1 （154）	神经网络2 （618）	神经网络3 （1 130）
FedPSK（ $m$ =7）	13.06	14.06	18.15
FedPSK（ $m$ =9）	15.76	16.73	19.57
FedSAC	99.24	436.24	758.24

Fig. 6 Comparison of accuracy at different hyperparameter values

References 28

[1]	McMAHAN B， MOORE E， RAMAGE D， et al. Communication-efficient learning of deep networks from decentralized data［C］// Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. New York： JMLR.org， 2017： 1273-1282.
[2]	PAN Z， LI C， YU F， et al. FedLF： layer-wise fair federated learning［C］// Proceedings of the 38th AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2024： 14527-14535.
[3]	LYU L， XU X， WANG Q， et al. Collaborative fairness in federated learning［M］// YANG Q， FAN L， YU H. Federated learning： privacy and incentive， LNCS 12500. Cham： Springer， 2020： 189-204.
[4]	WANG Z， WANG Z， LYU L， et al. FedSAC： dynamic submodel allocation for collaborative fairness in federated learning［C］// Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York： ACM， 2024： 3299-3310.
[5]	YU H， LIU Z， LIU Y， et al. A fairness-aware incentive scheme for federated learning［C］// Proceedings of the 2020 AAAI/ACM Conference on AI， Ethics， and Society. New York： ACM， 2020： 393-399.
[6]	SHI Y， YU H， LEUNG C. Towards fairness-aware federated learning［J］. IEEE Transactions on Neural Networks and Learning Systems， 2024， 35（9）： 11922-11938.
[7]	ZHANG J， LI C， ROBLES-KELLY A， et al. Hierarchically fair federated learning［EB/OL］. ［2024-11-11］..
[8]	LYU L， YU J， NANDAKUMAR K， et al. Towards fair and privacy-preserving federated deep models［J］. IEEE Transactions on Parallel and Distributed Systems， 2020， 31（11）： 2524-2541.
[9]	KANG J， XIONG Z， NIYATO D， et al. Incentive design for efficient federated learning in mobile networks： a contract theory approach［C］// Proceedings of the 2019 IEEE VTS Asia Pacific Wireless Communications Symposium. Piscataway： IEEE， 2019： 1-5.
[10]	COHEN A I. Contract theory［M］// CLAEYS G. Encyclopedia of modern political thought. Thousand Oaks， CA： CQ Press， 2013： 191-194.
[11]	SARIKAYA Y， ERCETIN O. Motivating workers in federated learning： a Stackelberg game perspective［J］. IEEE Networking Letters， 2020， 2（1）： 23-27.
[12]	FAN Z， FANG H， ZHOU Z， et al. Fair and efficient contribution valuation for vertical federated learning［EB/OL］. ［2024-10-12］..
[13]	CHENG Q， QU S， LEE J. SHARPNN： shapley value regularized tabular neural network［EB/OL］. ［2024-11-16］..
[14]	XU X， LYU L， MA X， et al. Gradient driven rewards to guarantee fairness in collaborative machine learning［C］// Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2021： 16104-16117.
[15]	TASTAN N， FARES S， AREMU T， et al. Redefining contributions： shapley-driven federated learning ［C］// Proceedings of the 33rd International Joint Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2024： 5009-5017.
[16]	WAN T， DENG X， LIAO W， et al. Enhancing fairness in federated learning： a contribution‑based differentiated model approach ［J］. International Journal of Intelligent Systems， 2023， 2023： No.6692995.
[17]	TASTAN N， HORVATH S， NANDAKUMAR K. CYCle： choosing your collaborators wisely to enhance collaborative fairness in decentralized learning［EB/OL］. ［2025-02-08］..
[18]	WANG Z， PENG Z， FAN X， et al. FedAVE： adaptive data value evaluation framework for collaborative fairness in federated learning［J］. Neurocomputing， 2024， 574： No.127227.
[19]	RODGERS J l， NICEWANDER W A. Thirteen ways to look at the correlation coefficient［J］. The American Statistician， 1988， 42（1）： 59-66.
[20]	LI T， SAHU A K， TALWALKAR A， et al. Federated learning： challenges， methods， and future directions［J］. IEEE Signal Processing Magazine， 2020， 37（3）： 50-60.
[21]	浙江君同智能科技有限责任公司. 基于神经元激活值聚类的纵向联邦学习后门防御法： 202210146719.0［P］. 2022-03-18.
	Zhejiang Juntong Intelligent Technology Company Limited. Longitudinal federated learning backdoor defense based on neuron activation value clustering： 202210146719.0 ［P］. 2023-03-18.
[22]	HARTIGAN J A， WONG M A. Algorithm AS 136： a K-means clustering algorithm［J］. Journal of the Royal Statistical Society. Series C （Applied Statistics）， 1979， 28（1）： 100-108.
[23]	MOLCHANOV P， MALLYA A， TYREE S， et al. Importance estimation for neural network pruning［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 11256-11264.
[24]	KRIZHEVSKY A. Learning multiple layers of features from tiny images［R/OL］. ［2024-11-25］..
[25]	XIAO H， RASUL K， VOLLGRAF R. Fashion-MNIST： a novel image dataset for benchmarking machine learning algorithms ［EB/OL］. ［2024-11-25］..
[26]	ZHANG J， LIU Y， HUA Y， et al. PFLlib： personalized federated learning algorithm library［J］. Journal of Machine Learning Research， 2025， 26： 1-10.
[27]	YU Y， WEI A， KARIMIREDDY S P， et al. TCT： convexifying federated learning using bootstrapped neural tangent kernels［C］// Proceedings of the 36th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2022： 30882-30897.
[28]	LI T， SAHU A K， ZAHEER M， et al. Federated optimization in heterogeneous networks ［J］. Proceedings of Machine Learning and Systems， 2020， 2： 429-450.

Federated learning fairness algorithm based on personalized submodel and K-means clustering

基于个性化子模型和K均值聚类的联邦学习公平性算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 28

Related Articles 15

Recommended Articles

Metrics

[1]	Jintao SU, Lina GE, Liguang XIAO, Jing ZOU, Zhe WANG. Detection and defense scheme for backdoor attacks in federated learning [J]. Journal of Computer Applications, 2025, 45(8): 2399-2408.
[2]	Hongyang ZHANG, Shufen ZHANG, Zheng GU. Federated learning algorithm for personalization and fairness [J]. Journal of Computer Applications, 2025, 45(7): 2123-2131.
[3]	Yiming ZHANG, Tengfei CAO. Federated learning optimization algorithm based on local drift and diversity computing power [J]. Journal of Computer Applications, 2025, 45(5): 1447-1454.
[4]	Qingli CHEN, Yuanbo GUO, Chen FANG. Clustering federated learning algorithm for heterogeneous data [J]. Journal of Computer Applications, 2025, 45(4): 1086-1094.
[5]	Ruilong CHEN, Peng YI, Tao HU, Youjun BU. Encrypted traffic classification method based on federated prototypical incremental learning [J]. Journal of Computer Applications, 2025, 45(12): 3888-3895.
[6]	Shufen ZHANG, Hongyang ZHANG, Zhiqiang REN, Xuebin CHEN. Survey of fairness in federated learning [J]. Journal of Computer Applications, 2025, 45(1): 1-14.
[7]	Zheyuan SHEN, Keke YANG, Jing LI. Personalized federated learning method based on dual stream neural network [J]. Journal of Computer Applications, 2024, 44(8): 2319-2325.
[8]	Chenghao YANG, Jie HU, Hongjun WANG, Bo PENG. Incomplete multi-view clustering algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(12): 3784-3789.
[9]	Jie WU, Xuezhong QIAN, Wei SONG. Personalized federated learning based on similarity clustering and regularization [J]. Journal of Computer Applications, 2024, 44(11): 3345-3353.
[10]	Xuebin CHEN, Changsheng QU. Overview of backdoor attacks and defense in federated learning [J]. Journal of Computer Applications, 2024, 44(11): 3459-3469.
[11]	Chunyong YIN, Yongcheng ZHOU. Automatically adjusted clustered federated learning for double-ended clustering [J]. Journal of Computer Applications, 2024, 44(10): 3011-3020.
[12]	Linhao LI, Xiaoqian ZHANG, Yao DONG, Xu WANG, Yongfeng DONG. Knowledge tracing based on personalized learning and deep refinement [J]. Journal of Computer Applications, 2024, 44(10): 3039-3046.
[13]	Hui ZHOU, Yuling CHEN, Xuewei WANG, Yangwen ZHANG, Jianjiang HE. Deep shadow defense scheme of federated learning based on generative adversarial network [J]. Journal of Computer Applications, 2024, 44(1): 223-232.
[14]	Mengjie LAN, Jianping CAI, Lan SUN. Self-regularization optimization methods for Non-IID data in federated learning [J]. Journal of Computer Applications, 2023, 43(7): 2073-2081.
[15]	Wanzhen CHEN, En ZHANG, Leiyong QIN, Shuangxi HONG. Privacy-preserving federated learning algorithm based on blockchain in edge computing [J]. Journal of Computer Applications, 2023, 43(7): 2209-2216.