Abstract:

Aiming at overfitting problem caused by memory behavior of Deep Neural Networks （DNNs） on image data with noisy labels， a meta label correction method based on predictions from shallow neural networks was proposed. In this method， with the use of weakly supervised training method， a label reweighting network was set to reweight noise data， meta learning method was employed to facilitate dynamic learning of the model to noise data， and the prediction output from both deep and shallow networks was used as the pseudo labels to train the model. At the same time， the knowledge distillation algorithm was applied to allow the deep network to guide the training of the shallow networks. In this way， the memory behavior of the model was alleviated effectively and the robustness of the model was enhanced. Experiments conducted on CIFAR10/100 and Clothing1M datasets demonstrate the superiority of the proposed method over Meta Label Correction （MLC） method. Particularly， on CIFAR10 dataset with symmetrical noise ratios of 60% and 80%， the accuracy improvements are 3.49 and 1.56 percentage points respectively. Furthermore， in ablation experiments on CIFAR100 dataset with asymmetric noise ratio of 40%， at most 5.32 percentage points accuracy improvement is achieved by the proposed method over models trained without predicted labels， confirming the feasibility and effectiveness of the proposed method.

Key words: noisy label, meta learning, label correction, label reweighting, knowledge distillation

摘要：

针对深度神经网络（DNN）对含有噪声标签的图像数据具有记忆行为而导致的过拟合问题，提出一种基于浅层神经网络预测的元标签校正方法。该方法采用弱监督训练方式，通过设置标签重加权网络对噪声数据进行加权操作，利用元学习方法使模型动态地学习噪声数据，并将模型中深层与浅层网络的预测输出作为伪标签训练模型，同时利用知识蒸馏算法使深层网络指导浅层网络训练，以有效缓解模型的记忆行为并提升模型鲁棒性。在CIFAR10/100、Clothing1M数据集上的实验结果表明，相较于元标签校正（MLC）方法，所提方法在对称噪声比例为60%与80%的CIFAR10数据集上的准确率分别提升了3.49、1.56个百分点；此外，在CIFAR100数据集的消融实验中，非对称噪声比例为40%时，所提方法比无预测标签训练的模型准确率最高提升了5.32个百分点，验证了所提方法的可行性与有效性。

关键词: 噪声标签, 元学习, 标签校正, 标签重加权, 知识蒸馏