Unsupervised industrial anomaly detection based on denoising reverse distillation

doi:10.11772/j.issn.1001-9081.2024111594

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (11): 3721-3729.DOI: 10.11772/j.issn.1001-9081.2024111594

• Multimedia computing and computer simulation • Previous Articles

Unsupervised industrial anomaly detection based on denoising reverse distillation

Yuxuan LI¹^,², Bin CHEN²^,³^,⁴(), Weizhi XIAN⁴

^1.Chengdu Institute of Computer Application，Chinese Academy of Sciences，Chengdu Sichuan 610213，China
^2.University of Chinese Academy of Sciences，Beijing 100049，China
^3.International Institute for Artificial Intelligence，Harbin Institute of Technology （Shenzhen），Shenzhen Guangdong 518055，China
^4.Chongqing Research Institute，Harbin Institute of Technology，Chongqing 401151，China

Received:2024-11-11 Revised:2025-03-20 Accepted:2025-03-26 Online:2025-04-02 Published:2025-11-10
Contact: Bin CHEN
About author:LI Yuxuan， born in 1999， M. S. candidate. His research interests include anomaly detection， deep learning.
XIAN Weizhi， born in 1994， Ph. D. His research interests include computer vision， pattern recognition.
Supported by:
General Project of Natural Science Foundation of Chongqing(CSTB2024NSCQ-MSX0479);General Project of China Postdoctoral Science Foundation(2024MD754244);Special Funding Program for Postdoctoral Research in Chongqing(2023CQBSHTB3119)

基于去噪反向蒸馏的无监督工业异常检测

李雨轩¹^,², 陈斌²^,³^,⁴(), 咸伟志⁴

^1.中国科学院成都计算机应用研究所，成都 610213
^2.中国科学院大学，北京 100049
^3.哈尔滨工业大学（深圳）国际人工智能研究院，广东深圳 518055
^4.哈尔滨工业大学重庆研究院，重庆 401151

通讯作者: 陈斌
作者简介:李雨轩（1999—），男，河南商丘人，硕士研究生，主要研究方向：异常检测、深度学习
咸伟志（1994—），男，江苏南京人，博士，主要研究方向：计算机视觉、模式识别。
基金资助:
重庆市自然科学基金面上项目(CSTB2024NSCQ-MSX0479);中国博士后科学基金面上项目(2024MD754244);重庆市博士后研究特别资助项目(2023CQBSHTB3119)

Abstract

Abstract:

Anomaly detection with localization capabilities is an important application of computer vision in industrial manufacturing. Recently， anomaly detection algorithms based on Reverse Distillation （RD） demonstrate good performance for this task. However， previous RD-based approaches only apply constraints to normal data， failing to ensure student network's feature reconstruction capability when dealing with anomalies. In addition， RD-based anomaly detection algorithms fuse multi-level difference information of the network based on experience solely， leading to suboptimal anomaly localization. To further enhance performance， an unsupervised industrial anomaly detection algorithm based on denoising RD， called DeRD， was proposed. It consists of an RD network with memory bank， an Multi-Scale Feature Denoising （MSFD） module， and a segmentation network. Firstly， to strengthen constraints on anomalous data， a MSFD module based on contrastive learning and multi-task learning was designed. Combined with a memory bank mechanism， this model enabled the student network to learn more discriminative feature representations. Secondly， a self-supervised segmentation network using synthetic anomalies was trained to adaptively integrate the feature difference information between multi-level teacher and student networks， thereby significantly improving anomaly localization performance. Experimental results on industrial detection benchmark datasets demonstrate the advanced performance of DeRD. It achieves an image-level AUC （Area Under the receiver operating characteristic Curve） of 98.8%， a pixel-level AUC of 98.48%， and a pixel-level Average Precision （AP） of 73.5%， surpassing comparison algorithms.

Key words: anomaly detection, Reverse Distillation (RD), teacher-student network, contrastive learning, unsupervised learning

摘要：

具有定位功能的异常检测是计算机视觉在工业制造中的一项重要应用。近年来，基于反向蒸馏（RD）的异常检测算法在这一任务中表现出良好的性能。但先前基于RD的工作由于仅对正常数据进行约束，难以保证学生网络面对异常时的特征重建能力。此外，基于RD的异常检测算法仅凭经验融合网络的多级差异信息，无法达到最优的异常定位效果。为进一步提升性能，提出一种基于去噪反向蒸馏的无监督工业异常检测算法DeRD，由带有记忆库的RD网络、多尺度特征去噪（MSFD）模块和分割网络组成。首先，为了加强对异常数据的约束，设计一种基于对比学习与多任务学习的MSFD模块，并结合记忆库机制，使学生网络能够学习更有效的特征表示；其次，为了自适应地融合多级教师网络与学生网络的特征差异信息，训练一个使用合成异常的自监督分割网络，从而显著提高模型异常定位性能。在工业检测基准数据集上的实验结果表明，DeRD算法表现出先进的性能，它的图像级受试者工作特征曲线下面积（AUC）为98.8%，像素级AUC为98.4%，像素级平均精确率（AP）为73.5%，高于对比算法。

关键词: 异常检测, 反向蒸馏, 教师学生网络, 对比学习, 无监督学习

CLC Number:

TP391.4

Yuxuan LI, Bin CHEN, Weizhi XIAN. Unsupervised industrial anomaly detection based on denoising reverse distillation[J]. Journal of Computer Applications, 2025, 45(11): 3721-3729.

李雨轩, 陈斌, 咸伟志. 基于去噪反向蒸馏的无监督工业异常检测[J]. 《计算机应用》唯一官方网站, 2025, 45(11): 3721-3729.

Figures/Tables 9

References 24

[1]	BERGMANN P， FAUSER M， SATTLEGGER D， et al. MVTec AD — a comprehensive real-world dataset for unsupervised anomaly detection［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 9584-9592.
[2]	PERERA P， NALLAPATI R， XIANG B. OCGAN： one-class novelty detection using GANs with constrained latent representations［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 2893-2901.
[3]	SABOKROU M， KHALOOEI M， FATHY M， et al. Adversarially learned one-class classifier for novelty detection［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 3379-3388.
[4]	SCHLEGL T， SEEBÖCK P， WALDSTEIN S M， et al. f-AnoGAN： fast unsupervised anomaly detection with generative adversarial networks［J］. Medical Image Analysis， 2019， 54： 30-44.
[5]	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［J］. Communications of the ACM， 2017， 60（6）： 84-90.
[6]	ROTH K， PEMULA L， ZEPEDA J， et al. Towards total recall in industrial anomaly detection［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 14298-14308.
[7]	DEFARD T， SETKOV A， LOESCH A， et al. PaDiM： a patch distribution modeling framework for anomaly detection and localization［C］// Proceedings of the 2021 International Conference on Pattern Recognition， LNCS 12664. Cham： Springer， 2021： 475-489.
[8]	刘永江，陈斌.基于多尺度记忆库的像素级无监督工业异常检测［J］．计算机应用，2024，44（11）：3587-3594.
	LIU Y J， CHEN B. Pixel-level unsupervised industrial anomaly detection based on multi-scale memory bank［J］. Journal of Computer Applications， 2024， 44（11）： 3587-3594.
[9]	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network［EB/OL］. ［2024-08-02］..
[10]	BERGMANN P， FAUSER M， SATTLEGGER D， et al. Uninformed students： student-teacher anomaly detection with discriminative latent embeddings［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 4182-4191.
[11]	SALEHI M， SADJADI N， BASELIZADEH S， et al. Multiresolution knowledge distillation for anomaly detection［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 14897-14907.
[12]	DENG J， DONG W， SOCHER R， et al. ImageNet： a large-scale hierarchical image database［C］// Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2009： 248-255.
[13]	DENG H， LI X. Anomaly detection via reverse distillation from one-class embedding［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 9727-9736.
[14]	TIEN T D， NGUYEN A T， TRAN N H， et al. Revisiting reverse distillation for anomaly detection［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023： 24511-24520.
[15]	BERGMANN P， LÖWE S， FAUSER M， et al. Improving unsupervised defect segmentation by applying structural similarity to autoencoders［C］// Proceedings of the 14th International Joint Conference on Computer Vision， Imaging and Computer Graphics Theory and Applications — Volume 5： VISAPP. Setúbal： SciTePress， 2019： 372-380.
[16]	VASILEV A， GOLKOV V， MEISSNER M， et al. q-Space novelty detection with variational autoencoders［C］// Proceedings of the 2020 Workshop on Computational Diffusion MRI： MICCAI Workshop， MATHVISUAL. Cham： Springer， 2020： 113-124.
[17]	WANG G， HAN S， DING E， et al. Student-teacher feature pyramid matching for anomaly detection［C］// Proceedings of the 2021 British Machine Vision Conference. Durham： BMVA Press， 2021： No.1273.
[18]	BATZNER K， HECKLER L， KÖNIG R. EfficientAD： accurate visual anomaly detection at millisecond-level latencies［C］// Proceedings of the 2024 IEEE/CVF Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2024： 127-137.
[19]	ZHANG X， LI S， LI X， et al. DeSTSeg： segmentation guided denoising student-teacher for anomaly detection［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023： 3914-3923.
[20]	LI C L， SOHN K， YOON J， et al. CutPaste： self-supervised learning for anomaly detection and localization［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 9659-9669.
[21]	ZAVRTANIK V， KRISTAN M， SKOČAJ D. DRÆM — a discriminatively trained reconstruction embedding for surface anomaly detection［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 8310-8319.
[22]	ZHANG X， XU M， ZHOU X. RealNet： a feature selection network with realistic synthetic anomaly for anomaly detection［C］// Proceedings of the 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2024： 16699-16708.
[23]	ZAVRTANIK V， KRISTAN M， SKOČAJ D. DSR — a dual subspace re-projection network for surface anomaly detection［C］// Proceedings of the 2022 European Conference on Computer Vision， LNCS 13691. Cham： Springer， 2022： 539-554.
[24]	YANG M， WU P， FENG H. MemSeg： a semi-supervised method for image surface defect detection using differences and commonalities［J］. Engineering Applications of Artificial Intelligence， 2023， 119： No.105835.

类别	PatchCore^［6］	PaDiM^［7］	DRÆM^［21］	STPM^［17］	RD4AD^［13］	DestSeg^［19］	DeRD
平均	97.5	90.8	95.4	92.4	98.6	98.2	98.8
bottle	100.0	99.4	96.2	100.0	99.9	100.0	100.0
cable	98.5	85.0	81.1	74.8	97.0	98.0	97.7
capsule	95.4	94.9	93.9	94.4	98.2	95.7	97.4
carpet	99.7	88.9	89.4	92.5	98.7	99.8	99.1
grid	94.7	90.4	99.6	77.4	100.0	100.0	100.0
hazelnut	99.4	99.1	99.4	95.6	100.0	99.9	99.7
leather	100.0	84.3	100.0	95.3	100.0	100.0	100.0
metal nut	100.0	87.6	95.0	94.0	100.0	99.2	99.5
pill	95.1	98.1	90.2	93.0	96.8	95.7	92.2
screw	96.4	92.2	96.8	97.6	97.8	90.0	98.2
tile	98.2	86.5	99.8	99.0	99.3	99.8	99.8
toothbrush	100.0	100.0	99.1	100.0	99.7	99.1	100.0
transistor	90.5	94.1	91.9	97.2	96.8	99.2	100.0
wood	96.3	80.8	98.2	96.3	99.2	97.1	99.4
zipper	98.6	80.9	100.0	78.5	98.0	100.0	99.6

类别	PatchCore^［6］	PaDiM^［7］	DRÆM^［21］	STPM^［17］	RD4AD^［13］	DestSeg^［19］	DeRD
平均	97.5	90.8	95.4	92.4	98.6	98.2	98.8
bottle	100.0	99.4	96.2	100.0	99.9	100.0	100.0
cable	98.5	85.0	81.1	74.8	97.0	98.0	97.7
capsule	95.4	94.9	93.9	94.4	98.2	95.7	97.4
carpet	99.7	88.9	89.4	92.5	98.7	99.8	99.1
grid	94.7	90.4	99.6	77.4	100.0	100.0	100.0
hazelnut	99.4	99.1	99.4	95.6	100.0	99.9	99.7
leather	100.0	84.3	100.0	95.3	100.0	100.0	100.0
metal nut	100.0	87.6	95.0	94.0	100.0	99.2	99.5
pill	95.1	98.1	90.2	93.0	96.8	95.7	92.2
screw	96.4	92.2	96.8	97.6	97.8	90.0	98.2
tile	98.2	86.5	99.8	99.0	99.3	99.8	99.8
toothbrush	100.0	100.0	99.1	100.0	99.7	99.1	100.0
transistor	90.5	94.1	91.9	97.2	96.8	99.2	100.0
wood	96.3	80.8	98.2	96.3	99.2	97.1	99.4
zipper	98.6	80.9	100.0	78.5	98.0	100.0	99.6

类别	PatchCore^［6］	PaDiM^［7］	DRÆM^［21］	STPM^［17］	RD4AD^［13］	DestSeg^［19］	DeRD
平均	96.4	96.6	97.5	95.4	97.8	98.0	98.4
bottle	98.8	98.4	99.3	98.3	98.8	99.2	98.6
cable	94.6	95.4	95.4	88.4	97.0	96.3	97.8
capsule	92.1	90.6	94.0	96.9	98.6	98.0	98.2
carpet	99.0	98.9	96.2	98.8	98.9	97.5	99.2
grid	96.5	96.5	99.6	76.5	99.3	99.2	98.4
hazelnut	93.8	93.0	99.5	87.8	99.0	99.3	99.5
leather	98.0	98.1	98.9	98.6	99.4	97.5	99.8
metal nut	98.2	98.5	98.7	98.0	97.3	98.9	98.9
pill	98.8	98.8	97.6	98.7	98.2	98.9	96.4
screw	93.9	94.5	99.7	98.9	99.6	98.2	98.8
tile	97.8	97.8	99.5	98.4	95.7	99.0	98.7
toothbrush	99.4	99.2	98.1	99.4	99.1	99.3	99.1
transistor	93.3	96.6	90.0	95.5	93.0	92.8	96.7
wood	95.2	95.7	97.2	98.8	95.4	97.8	96.8
zipper	97.4	97.8	98.6	97.6	98.2	98.5	99.3

类别	PatchCore^［6］	PaDiM^［7］	DRÆM^［21］	STPM^［17］	RD4AD^［13］	DestSeg^［19］	DeRD
平均	96.4	96.6	97.5	95.4	97.8	98.0	98.4
bottle	98.8	98.4	99.3	98.3	98.8	99.2	98.6
cable	94.6	95.4	95.4	88.4	97.0	96.3	97.8
capsule	92.1	90.6	94.0	96.9	98.6	98.0	98.2
carpet	99.0	98.9	96.2	98.8	98.9	97.5	99.2
grid	96.5	96.5	99.6	76.5	99.3	99.2	98.4
hazelnut	93.8	93.0	99.5	87.8	99.0	99.3	99.5
leather	98.0	98.1	98.9	98.6	99.4	97.5	99.8
metal nut	98.2	98.5	98.7	98.0	97.3	98.9	98.9
pill	98.8	98.8	97.6	98.7	98.2	98.9	96.4
screw	93.9	94.5	99.7	98.9	99.6	98.2	98.8
tile	97.8	97.8	99.5	98.4	95.7	99.0	98.7
toothbrush	99.4	99.2	98.1	99.4	99.1	99.3	99.1
transistor	93.3	96.6	90.0	95.5	93.0	92.8	96.7
wood	95.2	95.7	97.2	98.8	95.4	97.8	96.8
zipper	97.4	97.8	98.6	97.6	98.2	98.5	99.3

类别	PatchCore^［6］	PaDiM^［7］	DRÆM^［21］	STPM^［17］	RD4AD^［13］	DestSeg^［19］	DeRD
平均	45.4	45.2	68.9	51.8	58.0	71.1	73.5
bottle	76.7	73.0	89.7	73.2	78.7	85.7	89.9
cable	34.1	34.3	62.4	32.7	52.7	56.1	60.4
capsule	42.0	33.4	42.7	70.6	45.1	53.2	54.5
carpet	46.7	49.7	63.8	47.0	57.4	70.0	80.3
grid	63.3	58.0	55.3	29.9	49.2	56.2	54.8
hazelnut	39.8	37.4	87.7	40.0	62.1	82.3	87.2
leather	41.0	45.2	69.0	54.6	48.2	68.1	73.0
metal nut	33.1	39.4	91.4	36.3	79.1	87.6	92.9
pill	56.7	60.2	45.9	66.9	78.4	79.8	83.0
screw	33.5	24.9	70.5	45.2	53.6	52.7	47.1
tile	53.2	51.7	96.9	59.6	53.2	88.5	92.6
toothbrush	47.9	40.6	53.3	49.1	51.8	70.6	57.3
transistor	57.8	71.3	51.3	70.1	54.9	64.3	75.3
wood	43.1	42.3	80.8	80.1	48.2	74.3	74.4
zipper	12.8	16.6	71.9	22.0	57.1	77.3	79.8

Unsupervised industrial anomaly detection based on denoising reverse distillation

基于去噪反向蒸馏的无监督工业异常检测

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 24

Related Articles 15

Recommended Articles

Metrics

异常定位方法	transistor	leather
stage1	70.3	95.3
stage2	85.7	97.8
stage3	88.4	96.9
stage1，2，3	82.6	97.8
Seg	96.7	98.6

[1]	Chao LIU, Yanhua YU. Knowledge-aware recommendation model combining denoising strategy and multi-view contrastive learning [J]. Journal of Computer Applications, 2025, 45(9): 2827-2837.
[2]	Zhixiong XU, Bo LI, Xiaoyong BIAN, Qiren HU. Adversarial sample embedded attention U-Net for 3D medical image segmentation [J]. Journal of Computer Applications, 2025, 45(9): 3011-3016.
[3]	Zhiyuan WANG, Tao PENG, Jie YANG. Integrating internal and external data for out-of-distribution detection training and testing [J]. Journal of Computer Applications, 2025, 45(8): 2497-2506.
[4]	Jin XIE, Surong CHU, Yan QIANG, Juanjuan ZHAO, Hua ZHANG, Yong GAO. Dual-branch distribution consistency contrastive learning model for hard negative sample identification in chest X-rays [J]. Journal of Computer Applications, 2025, 45(7): 2369-2377.
[5]	Zhenzhou WANG, Fangfang GUO, Jingfang SU, He SU, Jianchao WANG. Robustness optimization method of visual model for intelligent inspection [J]. Journal of Computer Applications, 2025, 45(7): 2361-2368.
[6]	Yulin HE, Xu LI, Yingting HE, Laizhong CUI, Zhexue HUANG. Subspace Gaussian mixture model clustering ensemble algorithm based on maximum mean discrepancy [J]. Journal of Computer Applications, 2025, 45(6): 1712-1723.
[7]	Chaoying JIANG, Qian LI, Ning LIU, Lei LIU, Lizhen CUI. Readmission prediction model based on graph contrastive learning [J]. Journal of Computer Applications, 2025, 45(6): 1784-1792.
[8]	Longbo YAN, Wentao MAO, Zhihong ZHONG, Lilin FAN. Robust unsupervised multi-task anomaly detection method for defect diagnosis of urban drainage pipe network [J]. Journal of Computer Applications, 2025, 45(6): 1833-1840.
[9]	Mingfeng YU, Yongbin QIN, Ruizhang HUANG, Yanping CHEN, Chuan LIN. Multi-label text classification method based on contrastive learning enhanced dual-attention mechanism [J]. Journal of Computer Applications, 2025, 45(6): 1732-1740.
[10]	Wenjing YAN, Ruidong WANG, Min ZUO, Qingchuan ZHANG. Recipe recommendation model based on hierarchical learning of flavor embedding heterogeneous graph [J]. Journal of Computer Applications, 2025, 45(6): 1869-1878.
[11]	Ying HUANG, Shengmei GAO, Guang CHEN, Su LIU. Low-light image enhancement network combining signal-to-noise ratio guided dual-branch structure and histogram equalization [J]. Journal of Computer Applications, 2025, 45(6): 1971-1979.
[12]	Yufei LONG, Yuchen MOU, Ye LIU. Multi-source data representation learning model based on tensorized graph convolutional network and contrastive learning [J]. Journal of Computer Applications, 2025, 45(5): 1372-1378.
[13]	Wenbin HU, Tianxiang CAI, Tianle HAN, Zhaoman ZHONG, Changxia MA. Multimodal sarcasm detection model integrating contrastive learning with sentiment analysis [J]. Journal of Computer Applications, 2025, 45(5): 1432-1438.
[14]	Wenpeng WANG, Yinchang QIN, Wenxuan SHI. Review of unsupervised deep learning methods for industrial defect detection [J]. Journal of Computer Applications, 2025, 45(5): 1658-1670.
[15]	Zihe CHEN, Bin CHEN. Unsupervised point cloud anomaly detection based on multi-representation fusion [J]. Journal of Computer Applications, 2025, 45(5): 1677-1685.