Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (1): 250-258.DOI: 10.11772/j.issn.1001-9081.2021111920

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Print defect detection method based on deep comparison network

WANG Youxin1,2, CHEN Bin2,3,4   

  1. 1.Chengdu Institute of Computer Application, Chinese Academy of Sciences, Chengdu Sichuan 610213, China
    2.School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing 100049, China
    3.International Research Institute for Artificial Intelligence, Harbin Institute of Technology (Shenzhen), Shenzhen Guangdong 518055, China
    4.Chongqing Research Institute, Harbin Institute of Technology, Chongqing 401100, China
  • Received:2021-11-13 Revised:2022-03-13 Online:2022-06-17
  • Contact: CHEN Bin, born in 1970, Ph. D., research fellow. His research interests include industrial detection, deep learning.
  • About author:WANG Youxin, born in 1997, M. S. candidate. His research interests include industrial defect detection, object detection, multimodal video representation;


王佑芯1,2, 陈斌2,3,4   

  1. 1.中国科学院 成都计算机应用研究所,成都 610213
    2.中国科学院大学 计算机科学与技术学院,北京 100049
    3.哈尔滨工业大学(深圳) 国际人工智能研究院,广东 深圳 518055
    4.哈尔滨工业大学 重庆研究院,重庆 401100
  • 作者简介:王佑芯(1997—),男,江西宜春人,硕士研究生,主要研究方向:工业缺陷检测、目标检测、视频多模态表征;陈斌(1970—),男,四川广汉人,研究员,博士,CCF会员,主要研究方向:工业检测、深度学习。;

Abstract: The print defect detection methods based on traditional image processing technology have poor robustness and the object detection methods based on deep learning are not completely suitable for the detection tasks of print defects. In order to solve the problems above, the comparison ideas in template matching method were combined with the semantic features in deep learning, and a Deep Comparison Network (CoNet) used for the detection tasks of print defects was proposed. Firstly, the Deep Comparison Module (DCM) adopting Siamese structure was proposed to mine the semantic relationship between the detection image and the reference image through extracting and fusing the feature maps of them in the semantic space. Then, based on the feature pyramid structure with asymmetric dual channels, the Multi-scale Change Detection Module (MsCDM) was proposed to locate and classify print defects. On the public printed circuit board defect dataset DeepPCB and dataset of Lijin defects, the average values of mean Average Precision (mAP) of CoNet are 99.1% and 69.8% respectively, compared with the two baseline models Max-Pooling Group Pyramid Pooling (MP-GPP) and Change-Detection Single Shot Detector (CD-SSD), which are increased by 0.4, 3.5 percentage points and 0.7, 2.4 percentage points respectively, and the detection accuracy of CoNet is higher. Besides, when the resolution of input image is 640×640, the average time consumption of CoNet is 35.7 ms, showing that it can absolutely meet the real-time requirements of industrial detection tasks.

Key words: print defect detection, deep learning, Siamese convolutional neural network, feature pyramid, change detection

摘要: 基于传统图像处理技术的印刷缺陷检测方法鲁棒性差,而基于深度学习的目标检测方法则不完全适用于印刷缺陷检测任务的问题。为解决上述问题,将模板匹配方法中的对比思想与深度学习中的语义特征结合,提出用于印刷缺陷检测任务的深度对比网络(CoNet)。首先,提出基于孪生结构的深度对比模块(DCM)在语义空间提取并融合检测图像与参考图像的特征图,挖掘二者间的语义关系;然后,提出基于非对称双通路特征金字塔结构的多尺度变化检测模块(MsCDM),定位并识别印刷缺陷。在公开的印刷电路板缺陷数据集DeepPCB与立金缺陷数据集上,CoNet的平均精度均值(mAP)分别为99.1%和69.8%,与同样采用变化检测思路的最大分组金字塔池化(MP-GPP)和变化检测单次检测器(CD-SSD)相比,分别提升了0.4、3.5个百分点和0.7、2.4个百分点,CoNet的检测精度更高。此外,当输入图像分辨率为640×640时,CoNet的平均耗时为35.7 ms,可见其完全可以满足工业检测任务的实时性要求。

关键词: 印刷缺陷检测, 深度学习, 孪生卷积神经网络, 特征金字塔, 变化检测

CLC Number: