Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (1): 1-7.DOI: 10.11772/j.issn.1001-9081.2021122054

Special Issue: 人工智能

• Artificial intelligence •     Next Articles

Federated learning algorithm for communication cost optimization

ZHENG Sai1,2, LI Tianrui1,2, HUANG Wei1,2   

  1. 1.School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu Sichuan 611756, China
    2.Sichuan Key Laboratory of Cloud Computing and Intelligent Technique (Southwest Jiaotong University), Chengdu Sichuan 611756, China
  • Received:2021-12-08 Revised:2022-01-24 Online:2022-03-02
  • Contact: LI Tianrui, born in 1969, Ph. D., professor. His research interests include big data, cloud computing, data mining, machine learning, granular computing, rough sets.
  • About author:ZHENG Sai, born in 1996, M. S. candidate. His research interests include federated learning, machine learning;LI Tianrui, born in 1969, Ph. D., professor. His research interests include big data, cloud computing, data mining, machine learning, granular computing, rough sets;HUANG Wei, born in 1994, Ph. D. candidate. Her research interests include federated learning, data mining;
  • Supported by:
    This work is partially supported by National Key Research and Development Program of China (2019YFB2101802), National Natural Science Foundation of China (62176221).

面向通信成本优化的联邦学习算法

郑赛1,2, 李天瑞1,2, 黄维1,2   

  1. 1.西南交通大学 计算机与人工智能学院,成都 611756
    2.云计算与智能技术四川省高校重点实验室(西南交通大学),成都 611756
  • 通讯作者: 李天瑞(1969—),男,福建莆田人,教授,博士,CCF杰出会员,主要研究方向:大数据、云计算、数据挖掘、机器学习、粒度计算、粗糙集trli@swjtu.edu.cn
  • 作者简介:郑赛(1996—),男,浙江金华人,硕士研究生,主要研究方向:联邦学习、机器学习;李天瑞(1969—),男,福建莆田人,教授,博士,CCF杰出会员,主要研究方向:大数据、云计算、数据挖掘、机器学习、粒度计算、粗糙集;黄维(1994—),女,福建莆田人,博士研究生,主要研究方向:联邦学习、机器学习、数据挖掘;
  • 基金资助:
    国家重点研发计划项目(2019YFB2101802); 国家自然科学基金资助项目(62176221)。

Abstract: Federated Learning (FL) is a machine learning setting that can protect data privacy, however, the problems of high communication cost and client heterogeneity hinder the large?scale implementation of federated learning. To solve these two problems, a federated learning algorithm for communication cost optimization was proposed. First, the generative models from the clients were received and simulated data were generated by the server. Then, the simulated data were used by the server to train the global model and send it to the clients, and the final models were obtained by the clients through fine?tuning the global model. In the proposed algorithm only one round of communication between clients and the server was needed, and the fine?tuning of the client models was used to solve the problem of client heterogeneity. When the number of clients is 20, experiments were carried out on MNIST and CIFAR?10 dataset. The results show that the proposed algorithm can reduce the amount of communication data to 1/10 of that of Federated Averaging (FedAvg) algorithm on the MNIST dataset, and can reduce the amount of communication data to 1/100 of that of Federated Averaging (FedAvg) algorithm on the CIFAR-10 dataset with the premise of ensuring accuracy.

Key words: Federated Learning (FL), optimization algorithm, communication cost, single?round communication, generative model

摘要: 联邦学习是一种能够保护数据隐私的机器学习设置,然而高昂的通信成本和客户端的异质性问题阻碍了联邦学习的规模化落地。针对这两个问题,提出一种面向通信成本优化的联邦学习算法。首先,服务器接收来自客户端的生成模型并生成模拟数据;然后,服务器利用模拟数据训练全局模型并将其发送给客户端,客户端利用全局模型进行微调后得到最终模型。所提算法仅需要客户端与服务器之间的一轮通信,并且利用微调客户端模型来解决客户端异质性问题。在客户端数量为20个时,在MNIST和CIFAR?10这两个数据集上进行了实验。结果表明,所提算法能够在保证准确率的前提下,在MNIST数据集上将通信的数据量减少至联邦平均(FedAvg)算法的1/10,在CIFAR-10数据集上将通信数据量减少至FedAvg算法的1/100。

关键词: 联邦学习, 优化算法, 通信成本, 单轮通信, 生成模型

CLC Number: