关键词: 联邦学习, 非独立同分布数据, 客户端偏移, 正则化, 分布式机器学习, 隐私保护

Abstract: Federated Learning (FL) is a new distributed machine learning paradigm that breaks down data barriers while protecting data privacy, enabling clients to collaboratively train a machine learning model without sharing local data. However, how to deal with Non-independent identical distribution (Non-IID) data from different clients remains a huge challenge in federated learning. Although several studies have proposed some solutions to this problem, few of them utilized the implicit relationship between the global and local models to solve the problem simply and efficiently. To address the Non-IID issue in federated learning, novel optimization algorithms including FedSR and Dyn-FedSR were proposed in this paper:. Self-regularization penalty terms were introduced in each training round to dynamically regularize the local loss function in FedSR, which alleviated the Non-IID problem by constructing a relationship between the global and local models. The local model was close to the global model that aggregated richer knowledge in this way. The self-regularization penalty term was dynamically determined by calculating the similarity between the global and local models in Dyn-FedSR. Extensive experimental analyses on different tasks demonstrate that FedSR and Dyn-FedSR significantly outperform the state-of-the-art federated learning algorithms such as FedAvg, FedProx and SCAFFOLD in various scenarios, enabling efficient communication and higher accuracy, robustness to unbalanced data and uncertain local updates.

Key words: Federated Learning (FL), Non-IID data, client drift, regularization, distributed machine learning, privacy preserving