Abstract:

In view of the challenges of non-Independent and Identically Distributed （non-IID） data and heterogeneous computing power faced in Federated Learning （FL） for edge computing applications， the concept of local drift variable was introduced to avoid the significant deviation in client model updates caused by non-IID data， thereby preventing unstable model convergence. By correcting the local model parameters， the local training process was separated from the global aggregation process， optimizing FL performance in non-IID data training process. Furthermore， considering the diversity of edge server computing power， a new strategy was proposed： a simplified neural network sub-model was divided from the global model for deployment on resource-constrained edge servers， while high-capacity servers utilized the complete global model. Parameters trained by the low-capacity servers were uploaded to the cloud server， with partial parameter freezing to accelerate model convergence. Integrating these two methods， a Federated learning optimization algorithm based on Local drift and Diversity computing power （FedLD） was proposed to solve the heterogeneous challenges caused by non-IID data and diversity computing power in FL for edge computing. Experimental results show that FedLD has faster convergence speed and higher accuracy compared to FedAvg， SCAFFOLD， and FedProx algorithms， compared to FedProx， when 50 clients are involved in training， FedLD improves the model accuracy by 0.39%， 3.68% and 15.24% on MNIST， CIFAR-10 and CIFAR-100 datasets， respectively. Comparative analysis with the latest FedProc algorithm reveals that FedLD has lower communication overhead. Additional experiments incorporating K-Nearest Neighbors （KNN） algorithm， Long Short-Term Memory （LSTM） model， and bidirectional Gated Recurrent Unit （GRU） model demonstrate approximately 1% accuracy improvements across all three models when integrated with FedLD.

Key words: Federated Learning (FL), edge computing, heterogeneity, non-Independent and Identically Distributed (non-IID) data, client drift, diversity computing power

摘要：

针对联邦学习（FL）在边缘计算应用中所面临的非独立同分布（non-IID）数据和异构算力挑战，为了避免non-IID数据导致客户端模型更新出现较大偏差，从而引发模型不稳定的收敛，引入本地漂移变量的概念；并通过校正本地模型参数，将本地训练过程与全局聚合过程分离，优化FL在non-IID数据训练过程中的性能。此外，鉴于边缘服务器算力的多样性，提出一种新的策略：从全局模型中划分出一个简化的神经网络子模型下发给算力受限的边缘服务器进行训练，而高算力的边缘服务器则使用整个全局模型进行训练；低算力边缘服务器训练所得的参数将上传至云服务器，通过冻结部分参数提高整个模型的拟合速度。结合以上2种方法，提出一种基于本地漂移和多样性算力的联邦学习优化算法（FedLD），旨在解决联邦学习在边缘计算应用中所面临的non-IID数据和多样性算力带来的异构挑战。实验结果表明，FedLD比FedAvg、SCAFFOLD和FedProx算法收敛更快、准确率更高，相较于FedProx，在50个客户端参与训练时，FedLD在MNIST、CIFAR-10和CIFAR-100数据集上分别将模型准确率提升了0.39%、3.68%和15.24%；与最新的FedProc算法相比，FedLD通信开销更低；在K最近邻（KNN）算法、长短期记忆（LSTM）模型和双向门控循环单元（GRU）模型上的对比实验结果也表明，结合FedLD后，这3种模型的预测精度均有约1%的提升。

关键词: 联邦学习, 边缘计算, 异构性, 非独立同分布数据, 客户端漂移, 多样性算力