Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (12): 3798-3807.DOI: 10.11772/j.issn.1001-9081.2023121835

• Artificial intelligence • Previous Articles     Next Articles

Fast adversarial training method based on data augmentation and label noise

Yifei SONG, Yi LIU()   

  1. School of Computer Science and Technology,Guangdong University of Technology,Guangzhou Guangdong 510006,China
  • Received:2024-01-02 Revised:2024-03-14 Accepted:2024-03-18 Online:2024-03-28 Published:2024-12-10
  • Contact: Yi LIU
  • About author:SONG Yifei, born in 2000, M. S. candidate. His research interests include deep learning, adversarial examples, image processing.
  • Supported by:
    Key Technologies Research and Development Program of Guangdong Province(2021B0101200002)

基于数据增强和标签噪声的快速对抗训练方法

宋逸飞, 柳毅()   

  1. 广东工业大学 计算机学院,广州 510006
  • 通讯作者: 柳毅
  • 作者简介:宋逸飞(2000—),男,福建莆田人,硕士研究生,主要研究方向:深度学习、对抗样本、图像处理;
  • 基金资助:
    广东省重点领域研发计划项目(2021B0101200002)

Abstract:

Adversarial Training (AT) has been an effective defense approach for protecting classification models against adversarial attacks. However, high computational cost of the generation of strong adversarial samples during the training process may lead to significantly large extra training time. To overcome this limitation, Fast Adversarial Training (FAT) based on single-step attacks was explored. Previous work improves FAT from different perspectives, such as sample initialization, loss regularization, and training strategies. However, Catastrophic Overfitting (CO) will be encountered when dealing with large perturbation budgets. Therefore, an FAT method based on data augmentation and label noise was proposed. Firstly, multiple image transformations were performed to the original samples and random noise was introduced to implement data enhancement. Secondly, a small amount of label noise was injected. Thirdly, the augmented data were used to generate adversarial samples for model training. Finally, the label noise rate was adjusted adaptively according to the adversarial robustness test results. Comprehensive experimental results on CIFAR-10 and CIFAR-100 datasets show that compared to FGSM-MEP (Fast Gradient Sign Method with prior from the Momentum of all Previous Epoch) method, the proposed method improves 4.63 and 5.38 percentage points respectively on AA (AutoAttack) on the two datasets under the condition of large perturbation budget. The experimental results demonstrate that the proposed method can effectively handle the catastrophic overfitting problem under large perturbation budgets and enhance the adversarial robustness of model significantly.

Key words: deep learning, adversarial example, adversarial defense, data augmentation, label noise

摘要:

对抗训练(AT)是保护分类模型免受对抗性攻击的有效防御方法;然而,在训练过程中生成强对抗样本的高成本可能导致大量的额外训练时间。为了突破这一限制,探索基于单步攻击的快速对抗训练(FAT)。以往的工作从样本初始化、损失正则化和训练策略等不同角度改进了FAT;然而,在处理大扰动预算时会遇到灾难性过拟合(CO)。因此,提出一种基于数据增强与标签噪声的FAT方法。首先,对原始样本执行多种图像转换,并引入随机噪声进行数据增强;其次,少量标签噪声被注入;再次,使用增强的数据生成对抗样本用于模型训练;最后,根据对抗鲁棒性测试结果自适应地调整标签噪声率。在CIFAR-10和CIFAR-100数据集上的全面实验结果表明,相较于FGSM-MEP(Fast Gradient Sign Method with prior from the Momentum of all Previous Epoch)方法,所提方法在大扰动预算条件下,在2个数据集上的AA(AutoAttack)分别提升了4.63和5.38个百分点。实验结果验证了所提方法可以有效地处理大扰动预算下的CO问题,并显著增强模型的对抗鲁棒性。

关键词: 深度学习, 对抗样本, 对抗防御, 数据增强, 标签噪声

CLC Number: