基于自编码和知识蒸馏的表面缺陷检测方法

doi:10.11772/j.issn.1001-9081.2020121974

《计算机应用》唯一官方网站 ›› 2021, Vol. 41 ›› Issue (11): 3200-3205.DOI: 10.11772/j.issn.1001-9081.2020121974

基于自编码和知识蒸馏的表面缺陷检测方法

刘太亨, 何昭水()

广东工业大学自动化学院，广州 510006

收稿日期:2020-12-15 修回日期:2021-07-28 接受日期:2021-08-03 发布日期:2021-05-11 出版日期:2021-11-10
通讯作者: 何昭水
作者简介:刘太亨（1993—），男，广东肇庆人，博士研究生，主要研究方向：缺陷检测、深度学习、推荐系统
何昭水（1978—），男，湖南郴州人，教授，博士，主要研究方向：智能信息处理、机器学习。
基金资助:
国家自然科学基金资助项目(61773127);2018年度国家“万人计划”科技创新领军人才;广东省基础与应用基础研究基金联合基金重点项目(2019B1515120036);广东省自然科学基金资助项目(2018A030313306);广州科学技术基金资助项目(201802010037);广东省重点领域研发计划项目(2019B010147001)

Surface defect detection method based on auto-encoding and knowledge distillation

Taiheng LIU, Zhaoshui HE()

School of Automation，Guangdong University of Technology，Guangzhou Guangdong 510006，China

Received:2020-12-15 Revised:2021-07-28 Accepted:2021-08-03 Online:2021-05-11 Published:2021-11-10
Contact: Zhaoshui HE
About author:LIU Taiheng， born in 1993， Ph. D. candidate. His research interests include defect detection，deep learning，recommendation system
HE Zhaoshui，born in 1978，Ph. D.，professor. His research interests include intelligent information processing，machine learning.
Supported by:
the National Natural Science Foundation of China(61773127);the Leading Talents in Scientific and Technological Innovation of the “Ten Thousand Talent Program” of China in 2018, the Key Project of Joint Foundation of the Guangdong Basic and Applied Basic Research Foundation(2019B1515120036);the Natural Science Foundation of Guangdong Province(2018A030313306);the Guangzhou Science and Technology Foundation(201802010037);the Research and Development Program of Key Areas of Guangdong Province(2019B010147001)

摘要/Abstract

摘要：

针对传统的表面缺陷检测方法只能对具有高对比度或低噪声的明显缺陷轮廓进行检测的问题，提出了一种基于自编码和知识蒸馏的表面缺陷检测方法来准确定位和分类从实际工业环境捕获的输入图像中出现的缺陷。首先，设计了一种级联自动编码器（CAE）架构用于分割和定位缺陷，其目的是将输入的原始图像转换为基于CAE的预测蒙版；其次，利用阈值模块对预测结果进行二值化以获得准确的缺陷轮廓；然后，把缺陷区域检测器提取并裁剪出来的缺陷区域视为下一个模块的输入；最后，将CAE分割结果的缺陷区域通过知识蒸馏进行类别分类。实验结果表明，与其他几种表面缺陷检测方法相比，所提出的方法综合性能最好，其缺陷检测平均准确率为97.00%。该方法能够有效地对较小的、边缘不清晰的缺陷进行分割，满足对物品表面缺陷实时分割检测的工程要求。

关键词: 自动表面检测, 自编码器, 知识蒸馏, 缺陷检测, 图像处理

Abstract:

The traditional surface defect detection methods can only detect obvious defect contours with high contrast or low noise. In order to solve the problem， a surface defect detection method based on auto-encoding and knowledge distillation was proposed to accurately locate and classify the defects that appeared in the input images captured from the actual industrial environment. Firstly， a new Cascaded Auto-Encoder （CAE） architecture was designed to segment and locate defects， whose purpose was to convert the input original image into the CAE-based prediction mask. Secondly， the threshold module was used to binarize the prediction results， thereby obtaining the accurate defect contour. Then， the defect area extracted and cropped by the defect area detector was regarded as the input of the next module. Finally， the defect areas of the CAE segmentation results were classified by knowledge distillation. Experimental results show that， compared with other surface defect detection methods， the proposed method has the best comprehensive performance， and its average accuracy of defect detection is 97.00%. The proposed method can effectively segment the smaller defects with blurred edges， and meet the engineering requirements for real-time segmentation and detection of item surface defects.

Key words: automated surface inspection, Auto-Encoder (AE), knowledge distillation, defect detection, image processing

中图分类号:

TP391.4

刘太亨, 何昭水. 基于自编码和知识蒸馏的表面缺陷检测方法[J]. 计算机应用, 2021, 41(11): 3200-3205.

Taiheng LIU, Zhaoshui HE. Surface defect detection method based on auto-encoding and knowledge distillation[J]. Journal of Computer Applications, 2021, 41(11): 3200-3205.

图/表 10

图1 基于自编码和知识蒸馏的表面缺陷检测方法流程

Fig. 1 Flow chart of surface defect detection method based on auto-encoding and knowledge distillation

图2 自动编码器（AE）网络的体系结构

Fig. 2 Architecture of Auto-Encoder （AE） network

图3 空洞卷积示意图

Fig. 3 Schematic diagram of atrous convolution

表1 AE网络中空洞卷积的参数

Tab. 1 Parameters of atrous convolution in AE network

卷积层	Atrous因子	卷积大小	卷积层	Atrous因子	卷积大小
3	2	$7 × 7$	7	4	$15 × 15$
5	2	$7 × 7$	9	4	$15 × 15$

表1 AE网络中空洞卷积的参数

Tab. 1 Parameters of atrous convolution in AE network

卷积层	Atrous因子	卷积大小	卷积层	Atrous因子	卷积大小
3	2	$7 × 7$	7	4	$15 × 15$
5	2	$7 × 7$	9	4	$15 × 15$

图4 DAGM数据集的部分样例

Fig. 4 Some samples of DAGM dataset

图5 Magnetic-tile数据集的部分样例

Fig. 5 Some samples of Magnetic-tile dataset

表2 不同方法在物品表面缺陷检测上的性能指标对比

Tab. 2 Performance index comparison of different methods in item surface defect detection

数据集	方法	Accuracy	Precision	Recall	IoU
DAGM	CrackIT	0.771 9	0.768 4	0.743 2	0.578 1
	MPS	0.823 5	0.866 6	0.900 6	0.597 3
	CrackForest	0.869 1	0.902 8	0.865 8	0.614 7
	U-Net	0.898 3	0.920 2	0.932 1	0.652 6
	Mask-RCNN	0.899 6	0.920 8	0.935 2	0.654 3
	Fast-RCNN	0.900 4	0.923 7	0.938 9	0.662 5
	FCN	0.902 3	0.925 4	0.943 3	0.665 4
	SD_AEKD	0.964 7	0.939 2	0.957 8	0.819 2
Magnetic-tile	CrackIT	0.813 5	0.791 3	0.754 2	0.674 2
	MPS	0.837 3	0.882 7	0.910 4	0.690 7
	CrackForest	0.871 6	0.911 8	0.869 8	0.708 1
	U-Net	0.908 1	0.926 9	0.928 7	0.764 9
	Mask-RCNN	0.909 8	0.927 3	0.929 6	0.768 2
	Fast-RCNN	0.911 4	0.929 4	0.935 8	0.770 6
	FCN	0.913 7	0.930 2	0.944 9	0.782 4
	SD_AEKD	0.975 2	0.940 9	0.963 1	0.901 8

图6 部分样例在DAGM数据集上的检测分类结果

Fig. 6 Detection and classification results of some samples on DAGM dataset

表3 不同方法在物品表面缺陷分类上的性能指标对比

Tab. 3 Performance index comparison of different methods in item surface defect classification

数据集	方法	Accuracy	Precision	Recall
DAGM	GLCM	0.728 6	0.704 5	0.684 7
	HOG	0.689 9	0.648 3	0.632 2
	HOG+SOBEL	0.697 6	0.656 1	0.643 5
	SD_AEKD	0.904 1	0.897 7	0.886 4
Magnetic-tile	GLCM	0.832 9	0.859 3	0.832 7
	HOG	0.781 4	0.827 5	0.814 6
	HOG+SOBEL	0.763 5	0.804 1	0.791 3
	SD_AEKD	0.942 6	0.938 6	0.902 3

图7 部分样例在Magnetic-tile数据集上的检测分类结果

Fig. 7 Detection and classification results of some examples on Magnetic-tile dataset

参考文献 27

1	XIAO Z T， LENG Y Y， GENG L， et al. Defect detection and classification of galvanized stamping parts based on fully convolution neural network ［C］// Proceedings of the 2018 9th International Conference on Graphic and Image Processing. Bellingham： SPIE， 2018： Article No.106150K. 10.1117/12.2303601
2	TSANG C S C， NGAN H Y T， PANG G K H. Fabric inspection based on the Elo rating method ［J］. Pattern Recognition， 2016， 51： 378-394. 10.1016/j.patcog.2015.09.022
3	贺振东，王耀南，刘洁，等.基于背景差分的高铁钢轨表面缺陷图像分割［J］.仪器仪表学报，2016，37（3）：640-649. 10.3969/j.issn.0254-3087.2016.03.022
	HE Z D， WANG Y N， LIU J， et al. Background differencing-based high-speed rail surface defect image segmentation ［J］. Chinese Journal of Scientific Instrument， 2016， 37（3）： 640-649. 10.3969/j.issn.0254-3087.2016.03.022
4	YAN Y P， KANEKO S， ASANO H. Accumulated and aggregated shifting of intensity for defect detection on micro 3D textured surfaces ［J］. Pattern Recognition， 2020， 98： Article No.107057. 10.1016/j.patcog.2019.107057
5	HECHT-NIELSEN R. Theory of the backpropagation neural network ［J］. Neural Networks， 1988， 1（S1）： 445. 10.1016/0893-6080(88)90469-8
6	LIN C F， WANG S D. Fuzzy support vector machines ［J］. IEEE Transactions on Neural Networks， 2002， 13（2）： 464-471. 10.1109/72.991432
7	JEON Y J， CHOI D C， YUN J P， et al. Detection of scratch defects on slab surface ［C］// Proceedings of the 2011 11th International Conference on Control， Automation and Systems. Piscataway： IEEE， 2011： 1274-1278.
8	SONG K C， YAN Y H. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects ［J］. Applied Surface Science， 2013， 285（Pt B）： 858-864. 10.1016/j.apsusc.2013.09.002
9	DING S M， LIU Z F， LI C L. AdaBoost learning for fabric defect detection based on HOG and SVM ［C］// Proceedings of the 2011 International Conference on Multimedia Technology. Piscataway： IEEE， 2011： 2903-2906. 10.1109/icmt.2011.6001937
10	CHONDRONASIOS A， POPOV I， JORDANOV I. Feature selection for surface defect classification of extruded aluminum profiles ［J］. The International Journal of Advanced Manufacturing Technology， 2016， 83（1/2/3/4）： 33-41. 10.1007/s00170-015-7514-3
11	HAO X， ZHANG G G， MA S. Deep learning ［J］. International Journal of Semantic Computing， 2016， 10（3）： 417-439. 10.1142/s1793351x16500045
12	TAO X， ZHANG D P， MA W Z， et al. Automatic metallic surface defect detection and recognition with convolutional neural networks ［J］. Applied Sciences， 2018， 8（9）： Article No.1575. 10.3390/app8091575
13	YU Z Y， WU X J， GU X D. Fully convolutional networks for surface defect inspection in industrial environment ［C］// Proceedings of the 2017 International Conference on Computer Vision Systems， LNCS10528. Cham： Springer， 2017： 417-426.
14	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network ［EB/OL］. （2015-03-09）［2020-11-22］. .
15	ROMERO A， BALLAS N， KAHOU S E， et al. FitNets： hints for thin deep nets ［EB/OL］. （2015-03-27）［2020-11-22］..
16	LI Q Q， JIN S Y， YAN J J. Mimicking very efficient network for object detection ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 7341-7349. 10.1109/cvpr.2017.776
17	WEI Y， PAN X Y， QIN H W， et al. Quantization mimic： towards very tiny CNN for object detection ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11212. Cham： Springer， 2018： 274-290.
18	TANG S T， FENG L T， SHAO W Q， et al. Learning efficient detector with semi-supervised adaptive distillation ［C］// Proceedings of the 2019 British Machine Vision Conference. Durham： BMVA Press， 2019： Article No.21.
19	LIU Y F， CHEN K， LIU C， et al. Structured knowledge distillation for semantic segmentation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 2599-2608. 10.1109/cvpr.2019.00271
20	HE T， SHEN C H， TIAN Z， et al. Knowledge adaptation for efficient semantic segmentation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 578-587. 10.1109/cvpr.2019.00067
21	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5686-5696. 10.1109/cvpr.2019.00584
22	OLIVEIRA H， CORREIA P L. CrackIT — an image processing toolbox for crack detection and characterization ［C］// Proceedings of the 2014 IEEE International Conference on Image Processing. Piscataway： IEEE， 2014： 798-802. 10.1109/icip.2014.7025160
23	AMHAZ R， CHAMBON S， IDIER J， et al. Automatic crack detection on two-dimensional pavement images： an algorithm based on minimal path selection ［J］. IEEE Transactions on Intelligent Transportation Systems， 2016， 17（10）： 2718-2729. 10.1109/tits.2015.2477675
24	SHI Y， CUI L M， QI Z Q， et al. Automatic road crack detection using random structured forests ［J］. IEEE Transactions on Intelligent Transportation Systems， 2016， 17（12）： 3434-3445. 10.1109/tits.2016.2552248
25	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation ［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS9351. Cham： Springer， 2015： 234-241.
26	HU B， WANG J H. Detection of PCB surface defects with improved faster-RCNN and feature pyramid network ［J］. IEEE Access， 2020， 8： 108335-108345. 10.1109/access.2020.3001349
27	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2980-2988. 10.1109/iccv.2017.322

[1]	张成, 万源, 强浩鹏. 基于知识蒸馏的深度无监督离散跨模态哈希[J]. 计算机应用, 2021, 41(9): 2523-2531.
[2]	黄继爽, 张华, 李永龙, 赵皓, 王皓冉, 冯春成. 基于动态特征蒸馏的水工隧洞缺陷识别方法[J]. 计算机应用, 2021, 41(8): 2358-2365.
[3]	吴则举, 焦翠娟, 陈亮. 基于改进Faster R-CNN的轮胎缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1939-1946.
[4]	张元钧, 张曦煌. 基于图卷积与长短期记忆网络的动态网络表示学习模型[J]. 计算机应用, 2021, 41(7): 1857-1864.
[5]	高钦泉, 黄炳城, 刘文哲, 童同. 基于改进CenterNet的竹条表面缺陷检测方法[J]. 计算机应用, 2021, 41(7): 1933-1938.
[6]	史杨潇, 章军, 陈鹏, 王兵. 基于轻量级网络的钢铁表面缺陷分类[J]. 计算机应用, 2021, 41(6): 1836-1841.
[7]	孙鹤立, 孙玉柱, 张晓云. 基于生成对抗网络的事件描述生成[J]. 计算机应用, 2021, 41(5): 1256-1261.
[8]	樊玮, 王慧敏, 邢艳. 基于自编码器的多视图属性网络表示学习模型[J]. 计算机应用, 2021, 41(4): 1064-1070.
[9]	谭兆一, 陈白帆. 基于坐标逆映射的增强型车辆三维全景影像[J]. 计算机应用, 2021, 41(4): 1165-1171.
[10]	楼豪杰, 郑元林, 廖开阳, 雷浩, 李佳. 基于Siamese-YOLOv4的印刷品缺陷目标检测[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3206-3212.
[11]	屠红艳, 张挺, 夏鹏飞, 杜奕. 基于改进型变分自编码器的不确定性空间信息重建方法[J]. 计算机应用, 2021, 41(10): 2959-2963.
[12]	黄晓祥, 胡咏梅, 吴丹, 任力杰. 基于变分自编码器的异常颈动脉早期识别和预测[J]. 计算机应用, 2021, 41(10): 3082-3088.
[13]	魏淳武, 赵涓涓, 唐笑先, 强彦. 基于多时期蒸馏网络的随访数据知识提取方法[J]. 计算机应用, 2021, 41(10): 2871-2878.
[14]	李自强, 王正勇, 陈洪刚, 李林怡, 何小海. 基于外观和动作特征双预测模型的视频异常行为检测[J]. 计算机应用, 2021, 41(10): 2997-3003.
[15]	张京爱, 王江涛. 基于多尺度卷积神经网络和类内mixup操作的磁瓦表面质量识别[J]. 计算机应用, 2021, 41(1): 275-279.

基于自编码和知识蒸馏的表面缺陷检测方法

Surface defect detection method based on auto-encoding and knowledge distillation

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 27

相关文章 15

编辑推荐

Metrics