Surface defect detection method based on auto-encoding and knowledge distillation

doi:10.11772/j.issn.1001-9081.2020121974

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3200-3205.DOI: 10.11772/j.issn.1001-9081.2020121974

• Artificial intelligence • Previous Articles Next Articles

Surface defect detection method based on auto-encoding and knowledge distillation

Taiheng LIU, Zhaoshui HE()

School of Automation，Guangdong University of Technology，Guangzhou Guangdong 510006，China

Received:2020-12-15 Revised:2021-07-28 Accepted:2021-08-03 Online:2021-05-11 Published:2021-11-10
Contact: Zhaoshui HE
About author:LIU Taiheng， born in 1993， Ph. D. candidate. His research interests include defect detection，deep learning，recommendation system
HE Zhaoshui，born in 1978，Ph. D.，professor. His research interests include intelligent information processing，machine learning.
Supported by:
the National Natural Science Foundation of China(61773127);the Leading Talents in Scientific and Technological Innovation of the “Ten Thousand Talent Program” of China in 2018, the Key Project of Joint Foundation of the Guangdong Basic and Applied Basic Research Foundation(2019B1515120036);the Natural Science Foundation of Guangdong Province(2018A030313306);the Guangzhou Science and Technology Foundation(201802010037);the Research and Development Program of Key Areas of Guangdong Province(2019B010147001)

基于自编码和知识蒸馏的表面缺陷检测方法

刘太亨, 何昭水()

广东工业大学自动化学院，广州 510006

通讯作者: 何昭水
作者简介:刘太亨（1993—），男，广东肇庆人，博士研究生，主要研究方向：缺陷检测、深度学习、推荐系统
何昭水（1978—），男，湖南郴州人，教授，博士，主要研究方向：智能信息处理、机器学习。
基金资助:
国家自然科学基金资助项目(61773127);2018年度国家“万人计划”科技创新领军人才;广东省基础与应用基础研究基金联合基金重点项目(2019B1515120036);广东省自然科学基金资助项目(2018A030313306);广州科学技术基金资助项目(201802010037);广东省重点领域研发计划项目(2019B010147001)

Abstract

Abstract:

The traditional surface defect detection methods can only detect obvious defect contours with high contrast or low noise. In order to solve the problem， a surface defect detection method based on auto-encoding and knowledge distillation was proposed to accurately locate and classify the defects that appeared in the input images captured from the actual industrial environment. Firstly， a new Cascaded Auto-Encoder （CAE） architecture was designed to segment and locate defects， whose purpose was to convert the input original image into the CAE-based prediction mask. Secondly， the threshold module was used to binarize the prediction results， thereby obtaining the accurate defect contour. Then， the defect area extracted and cropped by the defect area detector was regarded as the input of the next module. Finally， the defect areas of the CAE segmentation results were classified by knowledge distillation. Experimental results show that， compared with other surface defect detection methods， the proposed method has the best comprehensive performance， and its average accuracy of defect detection is 97.00%. The proposed method can effectively segment the smaller defects with blurred edges， and meet the engineering requirements for real-time segmentation and detection of item surface defects.

Key words: automated surface inspection, Auto-Encoder (AE), knowledge distillation, defect detection, image processing

摘要：

针对传统的表面缺陷检测方法只能对具有高对比度或低噪声的明显缺陷轮廓进行检测的问题，提出了一种基于自编码和知识蒸馏的表面缺陷检测方法来准确定位和分类从实际工业环境捕获的输入图像中出现的缺陷。首先，设计了一种级联自动编码器（CAE）架构用于分割和定位缺陷，其目的是将输入的原始图像转换为基于CAE的预测蒙版；其次，利用阈值模块对预测结果进行二值化以获得准确的缺陷轮廓；然后，把缺陷区域检测器提取并裁剪出来的缺陷区域视为下一个模块的输入；最后，将CAE分割结果的缺陷区域通过知识蒸馏进行类别分类。实验结果表明，与其他几种表面缺陷检测方法相比，所提出的方法综合性能最好，其缺陷检测平均准确率为97.00%。该方法能够有效地对较小的、边缘不清晰的缺陷进行分割，满足对物品表面缺陷实时分割检测的工程要求。

关键词: 自动表面检测, 自编码器, 知识蒸馏, 缺陷检测, 图像处理

CLC Number:

TP391.4

Taiheng LIU, Zhaoshui HE. Surface defect detection method based on auto-encoding and knowledge distillation[J]. Journal of Computer Applications, 2021, 41(11): 3200-3205.

刘太亨, 何昭水. 基于自编码和知识蒸馏的表面缺陷检测方法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3200-3205.

Figures/Tables 10

Fig. 1 Flow chart of surface defect detection method based on auto-encoding and knowledge distillation

Fig. 2 Architecture of Auto-Encoder （AE） network

Fig. 3 Schematic diagram of atrous convolution

Tab. 1 Parameters of atrous convolution in AE network

卷积层	Atrous因子	卷积大小	卷积层	Atrous因子	卷积大小
3	2	$7 × 7$	7	4	$15 × 15$
5	2	$7 × 7$	9	4	$15 × 15$

Tab. 1 Parameters of atrous convolution in AE network

卷积层	Atrous因子	卷积大小	卷积层	Atrous因子	卷积大小
3	2	$7 × 7$	7	4	$15 × 15$
5	2	$7 × 7$	9	4	$15 × 15$

Fig. 4 Some samples of DAGM dataset

Fig. 5 Some samples of Magnetic-tile dataset

Tab. 2 Performance index comparison of different methods in item surface defect detection

数据集	方法	Accuracy	Precision	Recall	IoU
DAGM	CrackIT	0.771 9	0.768 4	0.743 2	0.578 1
	MPS	0.823 5	0.866 6	0.900 6	0.597 3
	CrackForest	0.869 1	0.902 8	0.865 8	0.614 7
	U-Net	0.898 3	0.920 2	0.932 1	0.652 6
	Mask-RCNN	0.899 6	0.920 8	0.935 2	0.654 3
	Fast-RCNN	0.900 4	0.923 7	0.938 9	0.662 5
	FCN	0.902 3	0.925 4	0.943 3	0.665 4
	SD_AEKD	0.964 7	0.939 2	0.957 8	0.819 2
Magnetic-tile	CrackIT	0.813 5	0.791 3	0.754 2	0.674 2
	MPS	0.837 3	0.882 7	0.910 4	0.690 7
	CrackForest	0.871 6	0.911 8	0.869 8	0.708 1
	U-Net	0.908 1	0.926 9	0.928 7	0.764 9
	Mask-RCNN	0.909 8	0.927 3	0.929 6	0.768 2
	Fast-RCNN	0.911 4	0.929 4	0.935 8	0.770 6
	FCN	0.913 7	0.930 2	0.944 9	0.782 4
	SD_AEKD	0.975 2	0.940 9	0.963 1	0.901 8

Fig. 6 Detection and classification results of some samples on DAGM dataset

Tab. 3 Performance index comparison of different methods in item surface defect classification

数据集	方法	Accuracy	Precision	Recall
DAGM	GLCM	0.728 6	0.704 5	0.684 7
	HOG	0.689 9	0.648 3	0.632 2
	HOG+SOBEL	0.697 6	0.656 1	0.643 5
	SD_AEKD	0.904 1	0.897 7	0.886 4
Magnetic-tile	GLCM	0.832 9	0.859 3	0.832 7
	HOG	0.781 4	0.827 5	0.814 6
	HOG+SOBEL	0.763 5	0.804 1	0.791 3
	SD_AEKD	0.942 6	0.938 6	0.902 3

Fig. 7 Detection and classification results of some examples on Magnetic-tile dataset

References 27

1	XIAO Z T， LENG Y Y， GENG L， et al. Defect detection and classification of galvanized stamping parts based on fully convolution neural network ［C］// Proceedings of the 2018 9th International Conference on Graphic and Image Processing. Bellingham： SPIE， 2018： Article No.106150K. 10.1117/12.2303601
2	TSANG C S C， NGAN H Y T， PANG G K H. Fabric inspection based on the Elo rating method ［J］. Pattern Recognition， 2016， 51： 378-394. 10.1016/j.patcog.2015.09.022
3	贺振东，王耀南，刘洁，等.基于背景差分的高铁钢轨表面缺陷图像分割［J］.仪器仪表学报，2016，37（3）：640-649. 10.3969/j.issn.0254-3087.2016.03.022
	HE Z D， WANG Y N， LIU J， et al. Background differencing-based high-speed rail surface defect image segmentation ［J］. Chinese Journal of Scientific Instrument， 2016， 37（3）： 640-649. 10.3969/j.issn.0254-3087.2016.03.022
4	YAN Y P， KANEKO S， ASANO H. Accumulated and aggregated shifting of intensity for defect detection on micro 3D textured surfaces ［J］. Pattern Recognition， 2020， 98： Article No.107057. 10.1016/j.patcog.2019.107057
5	HECHT-NIELSEN R. Theory of the backpropagation neural network ［J］. Neural Networks， 1988， 1（S1）： 445. 10.1016/0893-6080(88)90469-8
6	LIN C F， WANG S D. Fuzzy support vector machines ［J］. IEEE Transactions on Neural Networks， 2002， 13（2）： 464-471. 10.1109/72.991432
7	JEON Y J， CHOI D C， YUN J P， et al. Detection of scratch defects on slab surface ［C］// Proceedings of the 2011 11th International Conference on Control， Automation and Systems. Piscataway： IEEE， 2011： 1274-1278.
8	SONG K C， YAN Y H. A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects ［J］. Applied Surface Science， 2013， 285（Pt B）： 858-864. 10.1016/j.apsusc.2013.09.002
9	DING S M， LIU Z F， LI C L. AdaBoost learning for fabric defect detection based on HOG and SVM ［C］// Proceedings of the 2011 International Conference on Multimedia Technology. Piscataway： IEEE， 2011： 2903-2906. 10.1109/icmt.2011.6001937
10	CHONDRONASIOS A， POPOV I， JORDANOV I. Feature selection for surface defect classification of extruded aluminum profiles ［J］. The International Journal of Advanced Manufacturing Technology， 2016， 83（1/2/3/4）： 33-41. 10.1007/s00170-015-7514-3
11	HAO X， ZHANG G G， MA S. Deep learning ［J］. International Journal of Semantic Computing， 2016， 10（3）： 417-439. 10.1142/s1793351x16500045
12	TAO X， ZHANG D P， MA W Z， et al. Automatic metallic surface defect detection and recognition with convolutional neural networks ［J］. Applied Sciences， 2018， 8（9）： Article No.1575. 10.3390/app8091575
13	YU Z Y， WU X J， GU X D. Fully convolutional networks for surface defect inspection in industrial environment ［C］// Proceedings of the 2017 International Conference on Computer Vision Systems， LNCS10528. Cham： Springer， 2017： 417-426.
14	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network ［EB/OL］. （2015-03-09）［2020-11-22］. .
15	ROMERO A， BALLAS N， KAHOU S E， et al. FitNets： hints for thin deep nets ［EB/OL］. （2015-03-27）［2020-11-22］..
16	LI Q Q， JIN S Y， YAN J J. Mimicking very efficient network for object detection ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 7341-7349. 10.1109/cvpr.2017.776
17	WEI Y， PAN X Y， QIN H W， et al. Quantization mimic： towards very tiny CNN for object detection ［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11212. Cham： Springer， 2018： 274-290.
18	TANG S T， FENG L T， SHAO W Q， et al. Learning efficient detector with semi-supervised adaptive distillation ［C］// Proceedings of the 2019 British Machine Vision Conference. Durham： BMVA Press， 2019： Article No.21.
19	LIU Y F， CHEN K， LIU C， et al. Structured knowledge distillation for semantic segmentation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 2599-2608. 10.1109/cvpr.2019.00271
20	HE T， SHEN C H， TIAN Z， et al. Knowledge adaptation for efficient semantic segmentation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 578-587. 10.1109/cvpr.2019.00067
21	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5686-5696. 10.1109/cvpr.2019.00584
22	OLIVEIRA H， CORREIA P L. CrackIT — an image processing toolbox for crack detection and characterization ［C］// Proceedings of the 2014 IEEE International Conference on Image Processing. Piscataway： IEEE， 2014： 798-802. 10.1109/icip.2014.7025160
23	AMHAZ R， CHAMBON S， IDIER J， et al. Automatic crack detection on two-dimensional pavement images： an algorithm based on minimal path selection ［J］. IEEE Transactions on Intelligent Transportation Systems， 2016， 17（10）： 2718-2729. 10.1109/tits.2015.2477675
24	SHI Y， CUI L M， QI Z Q， et al. Automatic road crack detection using random structured forests ［J］. IEEE Transactions on Intelligent Transportation Systems， 2016， 17（12）： 3434-3445. 10.1109/tits.2016.2552248
25	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation ［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS9351. Cham： Springer， 2015： 234-241.
26	HU B， WANG J H. Detection of PCB surface defects with improved faster-RCNN and feature pyramid network ［J］. IEEE Access， 2020， 8： 108335-108345. 10.1109/access.2020.3001349
27	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2980-2988. 10.1109/iccv.2017.322

[1]	ZHANG Cheng, WAN Yuan, QIANG Haopeng. Deep unsupervised discrete cross-modal hashing based on knowledge distillation [J]. Journal of Computer Applications, 2021, 41(9): 2523-2531.
[2]	HUANG Jishuang, ZHANG Hua, LI Yonglong, ZHAO Hao, WANG Haoran, FENG Chuncheng. Hydraulic tunnel defect recognition method based on dynamic feature distillation [J]. Journal of Computer Applications, 2021, 41(8): 2358-2365.
[3]	WU Zeju, JIAO Cuijuan, CHEN Liang. Tire defect detection method based on improved Faster R-CNN [J]. Journal of Computer Applications, 2021, 41(7): 1939-1946.
[4]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[5]	SHI Yangxiao, ZHANG Jun, CHEN Peng, WANG Bing. Classification of steel surface defects based on lightweight network [J]. Journal of Computer Applications, 2021, 41(6): 1836-1841.
[6]	TAN Zhaoyi, CHEN Baifan. Enhanced vehicle 3D surround view based on coordinate inverse mapping [J]. Journal of Computer Applications, 2021, 41(4): 1165-1171.
[7]	Haojie LOU, Yuanlin ZHENG, Kaiyang LIAO, Hao LEI, Jia LI. Defect target detection for printed matter based on Siamese-YOLOv4 [J]. Journal of Computer Applications, 2021, 41(11): 3206-3212.
[8]	WEI Chunwu, ZHAO Juanjuan, TANG Xiaoxian, QIANG Yan. Knowledge extraction method for follow-up data based on multi-term distillation network [J]. Journal of Computer Applications, 2021, 41(10): 2871-2878.
[9]	ZHANG Jing'ai, WANG Jiangtao. Magnetic tile surface quality recognition based on multi-scale convolution neural network and within-class mixup operation [J]. Journal of Computer Applications, 2021, 41(1): 275-279.
[10]	WANG Siyu, GAO Weixin, LI Lu. Image quality evaluation model for X-ray circumferential welds [J]. Journal of Computer Applications, 2020, 40(9): 2748-2753.
[11]	ZHANG Yonghong, LIU Hao, TIAN Wei, WANG Jiangeng. Rainfall cloud segmentation method in Tibet based on DeepLab v3 [J]. Journal of Computer Applications, 2020, 40(9): 2781-2788.
[12]	ZHANG Shuping, WU Wen, WAN Yi. Single image shadow removal method based on multistage generative adversarial network [J]. Journal of Computer Applications, 2020, 40(8): 2378-2385.
[13]	YUAN Yuan, WU Wen, WAN Yi. Single image shadow detection method based on entropy driven domain adaptive learning [J]. Journal of Computer Applications, 2020, 40(7): 2131-2136.
[14]	ZHOU Siyu, BAO Guoqi, LIU Kai. lmage vignetting correction based on constrained log-intensity entropy under low-pass filtering [J]. Journal of Computer Applications, 2020, 40(6): 1812-1817.
[15]	FU Lei, REN Dejun, WU Huayun, GAO Ming, QIU Lyu, HU Yunqi. Real-time segmentation algorithm for bubble defects of plastic bottle based on improved Fast-SCNN [J]. Journal of Computer Applications, 2020, 40(6): 1824-1829.

Surface defect detection method based on auto-encoding and knowledge distillation

基于自编码和知识蒸馏的表面缺陷检测方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 27

Related Articles 15

Recommended Articles

Metrics