关键词: 生成对抗网络, 差分隐私, 动态梯度阈值裁剪, Rényi Accountant

Abstract: Abstract: Most of the existing methods that combine the generative adversarial network (GAN) and differential privacy used gradient perturbation to achieve privacy protection. In the process of optimization, gradient clipping technology was used to constrain the sensitivity of the optimizer to single data, and random noise was added to the clipped gradient to achieve the purpose of protecting the model. However, most methods took the clipping threshold as a fixed parameter during training. When the threshold was too large or too small, the performance of the model would be affected. To solve this problem, DGC_DPGAN with dynamic gradient threshold clipping was proposed to consider privacy protection and model performance. Combined with the pre-training technology, and in the process of optimization, the gradient F-norm average of each batch of privacy data was obtained as the dynamic gradient clipping threshold, and then the gradient was perturbed. Considering different clipping orders, CLIP_DGC_DPGAN and DGC_DPGAN were proposed, and used Rényi Accountant to calculate the privacy loss. The experimental results show that under the same privacy budget, the two dynamic gradient clipping algorithms are better than the fixed gradient threshold clipping method. On the Mnist dataset, the IS, SSIM, and CNN classification accuracy are improved by 0.32-3.92, 0. 03-0.27, and 0.06-0.28 respectively; On the Fashion-Mnist dataset, the IS, SSIM, and CNN classification accuracy are improved by 0.4-4.32, 0.01-0.44 and 0.11-0.23 respectively; At the same time, the availability of the images generated by GAN model is better.

Key words: Keywords: generative adversarial network, differential privacy, dynamic gradient clipping, Rényi Accountant