Abstract:

Classic Federated Learning （FL） algorithms are difficult to achieve good results in scenarios where data is highly heterogeneous. In Personalized FL （PFL）， a new solution was proposed aiming at the problem of data heterogeneity in federated learning， which is to “tailor” a dedicated model for each client. In this way， the models had good performance. However， it brought the difficulty in extending federated learning to new clients at the same time. Focusing on the challenges of performance and scalability in PFL， FedDual， a FL model with dual stream neural network structure， was proposed. By adding an encoder for analyzing the personalized characteristics of clients， this model was not only able to have the performance of personalized models， but also able to be extended to new clients easily. Experimental results show that compared to the classic Federated Averaging （FedAvg） algorithm on datasets such as MNIST and FashionMNIST， FedDual obviously improves the accuracy； on CIFAR10 dataset， FedDual improves the accuracy by more than 10 percentage points， FedDual achieves “plug and play” for new clients without decrease of the accuracy， solving the problem of difficult scalability for new clients.

Key words: Federated Learning (FL), Personalized Federated Learning (PFL), data heterogeneity, dual stream neural network, new client problem

摘要：

经典的联邦学习（FL）算法在数据高度异构的场景下难以取得较好的效果。个性化联邦学习（PFL）针对数据异构问题，提出新的解决方案，即为每个客户端“量身定做”专属模型，这样模型会拥有较好的性能；然而同时会引出难以将FL扩展到新客户端上的问题。针对PFL中的性能与扩展的难题展开研究，提出基于双流神经网络结构的联邦学习模型，简称FedDual。双流神经网络模型通过增加一个用于分析客户端个性化特征的编码器，既能拥有个性化模型的性能，又便于扩展到新客户端。实验结果表明，相较于经典联邦平均（FedAvg）算法，FedDual在MNIST和FashionMNIST等数据集上的准确率有明显提升，而在CIFAR10数据集上的准确率提升了10个百分点以上，且面对新客户端保持准确率不下降，实现了“即插即用”，解决了新客户端难以扩展的问题。

关键词: 联邦学习, 个性化联邦学习, 数据异构, 双流神经网络, 新客户端问题