关键词: 噪声标签, 元学习, 标签校正, 标签重加权, 知识蒸馏

Abstract: Abstract: A meta-label correction technique based on shallow network predictions is proposed to address overfitting and memorization behaviors exhibited by deep neural models when dealing with noisy labeled data. This approach employs weakly supervised training, where a label re-weighting network is introduced to re-weight noisy data, and utilizes meta-learning methods to enable the model to dynamically adapt to noisy data. The model is trained using the predictions from both deep and shallow networks as new labels, and knowledge distillation techniques are applied to have the deep network guide the training of the shallow network, so as to effectively alleviate the model's memorization behavior and enhance the model's generalizability. In experiments conducted on the CIFAR10, CIFAR100, and Clothing1M datasets, compared to other noise label learning methods, this approach demonstrates a significant improvement in accuracy.

Key words: noise label, meta learning, label correction, label reweighting, knowledge distillation