Coin surface defect detection algorithm based on deformable convolution and adaptive spatial feature fusion

doi:10.11772/j.issn.1001-9081.2021020227

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (2): 638-645.DOI: 10.11772/j.issn.1001-9081.2021020227

• Frontier and comprehensive applications • Previous Articles Next Articles

Coin surface defect detection algorithm based on deformable convolution and adaptive spatial feature fusion

Pinxue WANG¹^,², Shaobing ZHANG¹^,²^,³(), Miao CHENG¹^,²^,³, Lian HE¹^,³, Xiaoshan QIN¹^,²

^1.Chengdu Institute of Computer Application，Chinese Academy of Sciences，Chengdu Sichuan 610041，China
^2.School of Computer Science and Technology，University of Chinese Academy of Sciences，Beijing 100049，China
^3.Shenzhen CBPM-KEXIN Banking Technology Company Limited，Shenzhen Guangdong 518206，China

Received:2021-02-05 Revised:2021-04-02 Accepted:2021-04-12 Online:2022-02-11 Published:2022-02-10
Contact: Shaobing ZHANG
About author:WANG Pinxue， born in 1993， M. S. candidate. His research interests include few-shot learning， surface defect detection.
ZHANG Shaobing， born in 1979， M. S.， professor of engineering. His research interests include high-speed image processing， defect detection， deep learning.
CHENG Miao， born in 1983， M. S.， senior engineer. His research interests include artificial intelligence， machine vision.
HE Lian， born in 1983， Ph. D.， senior engineer. Her research interests include artificial intelligence， machine vision.
QIN Xiaoshan， born in 1995， M. S. candidate. His research interests include computer vision.

基于可变形卷积和自适应空间特征融合的硬币表面缺陷检测算法

王品学¹^,², 张绍兵¹^,²^,³(), 成苗¹^,²^,³, 何莲¹^,³, 秦小山¹^,²

^1.中国科学院成都计算机应用研究所, 成都 610041
^2.中国科学院大学计算机科学与技术学院, 北京 100049
^3.深圳市中钞科信金融科技有限公司, 深圳 518206

通讯作者: 张绍兵
作者简介:王品学（1993—），男，四川成都人，硕士研究生，CCF会员，主要研究方向：小样本学习、表面缺陷检测；
张绍兵（1979—），男，四川成都人，研究员级高级工程师，硕士，主要研究方向：高速图像处理、缺陷检测、深度学习；
成苗（1983—），男，四川成都人，高级工程师，硕士，主要研究方向：人工智能、机器视觉；
何莲（1983—），女，四川西充人，高级工程师，博士，主要研究方向：人工智能、机器视觉；
秦小山（1995—），男，四川资阳人，硕士研究生，主要研究方向：计算机视觉。

Abstract

Abstract:

Concerning the problem that the surface defects of the coin are small， variable in shape， easily confused with the background and difficult to be detected， an improved algorithm of coin surface defect detection named DCA-YOLO （Deformable Convolution and Adaptive space feature fusion-YOLO） was proposed. First of all， due to the different shapes of defects， three network structures with deformable convolution modules added at different positions in the backbone network were designed， and the ability to extract defects was improved through convolution learning offset and adjusting parameters. Then， the adaptive spatial feature fusion network was used to learn the weight parameters to better adapt to targets with different scales by adjusting the contribution of each pixel in the feature maps of different scales. Finally， the anchor ratio was adjusted， the category weights were dynamically adjusted， the comparison network performance was optimized， thus， a model network to add deformable convolution before upsampling for multi-scale fusion of the output features of the backbone network was proposed. Experimental results show that on the coin defect dataset， the detection mAP （mean Average Precision） of DCA-YOLO algorithm reaches 92.8%， which is close to that of Faster-RCNN （Faster Region-based Convolutional Neural Network）； compared with YOLOv3， the proposed algorithm has the detection speed basically the same with 3.3 percentage points improvement on detection mAP， and 3.2 percentage points increase on F1-score.

Key words: YOLO (You Look Only Once) v3 algorithm, coin, surface defect detection, deformable convolution, adaptive spatial feature fusion

摘要：

针对硬币表面缺陷较小、形状多变且易与背景混淆而不易检出的问题，改进YOLOv3算法并提出基于可变形卷积和自适应空间特征融合的硬币表面缺陷检测算法DCA-YOLO。首先，由于缺陷形状的多变设计了3种在主干网络中不同位置添加可变形卷积模块的网络结构，通过卷积学习偏移量和调节参数来提高缺陷的提取能力；然后，使用自适应空间特征融合网络学习权重参数来调整不同尺度特征图中各像素点的贡献度以更好地适应不同尺度的目标；最后，改进先验锚框比例，动态调节类别权重，优化并对比网络性能，从而提出在主干网络输出特征进行多尺度融合的上采样前增加可变形卷积的模型网络。实验结果表明，在硬币缺陷数据集上，DCA-YOLO算法检测平均精度均值（mAP）接近于Faster-RCNN，达到了92.8%；而相较于YOLOv3，所提算法的检测速度基本持平，在检测mAP上提高了3.3个百分点，F1分数提升了3.2个百分点。

关键词: YOLOv3算法, 硬币, 表面缺陷检测, 可变形卷积, 自适应空间特征融合

CLC Number:

TP391.41

Pinxue WANG, Shaobing ZHANG, Miao CHENG, Lian HE, Xiaoshan QIN. Coin surface defect detection algorithm based on deformable convolution and adaptive spatial feature fusion[J]. Journal of Computer Applications, 2022, 42(2): 638-645.

王品学, 张绍兵, 成苗, 何莲, 秦小山. 基于可变形卷积和自适应空间特征融合的硬币表面缺陷检测算法[J]. 《计算机应用》唯一官方网站, 2022, 42(2): 638-645.

Figures/Tables 12

Fig. 1 Defect classification and imaging performance under different light sources

Fig. 2 Geometric transformation and color transformation of image

Fig. 3 Mosaic data augmentation

Fig. 4 Overall framework of DCA-YOLO algorithm for coin surface defect detection

Fig. 5 Random sampling ability of deformable convolution

Fig. 6 Deformable convolution operation steps

Fig. 7 Adding deformable convolution in different positions

Fig. 8 Adaptive spatial feature fusion

Tab. 1 Comparison of different model detection results

模型	骨干网络	mAP/%	F1/%	帧率/（frame·s^-1）
Faster-RCNN	Resnet50	92.9	92.5	19
YOLOv3	Darket53	89.5	89.2	34
YOLOv3-PAN	Darknet53	90.4	90.4	33
YOLOv3-tiny	Darknet19-Impoved	75.4	73.9	217
DenseNet-tiny	DenseNet18	85.3	83.1	46
YOLOv3-ASFF	Darknet53	91.1	91.0	34
DCA-YOLO	Darknet53	92.8	92.4	32

Fig. 9 Comparison of detection effect among YOLOv3， DCA-YOLO and Faster-RCNN

Tab. 2 Comparison of detection results of adding deformable convolution in different positions

模型分类	插入位置	P/%	R/%	mAP/%	F1/%	帧率/（frams·s^-1）
1	D5输出后替换SPP中的3×3卷积	92.1	92.0	92.2	92.0	32
2.1	上采样前添加	92.3	92.5	92.8	92.4	32
2.2	在上采样前添加并在D3、D4后加Conv	92.2	92.6	92.9	92.4	27
3	D3、D4输出后	91.6	91.9	91.9	91.7	28

Tab. 3 Influence of Mosaic data augmentation， activation function， dynamic category weight and stretching priori anchor box on detection results

序号	Mosaic数据增强	Mish激活函数	动态类别权重	拉伸先验锚框	mAP/%	F1/%
1	√				92.2	91.9
2		√			91.1	91.2
3	√		√		92.3	92.0
4				√	91.6	91.7
5	√	√			92.3	92.1
6	√	√		√	92.5	92.1
7	√	√	√	√	92.8	92.4

References 22

1	工业互联网产业联盟.工业智能白皮书2020［EB/OL］. （2020-04-26）［2020-07-01］. .
2	张建伟，雷霖，余化鹏，等.基于形态学的硬币镜面区域缺陷检测算法研究［J］.成都大学学报（自然科学版）， 2016， 35（3）： 245-247， 259.
	ZHANG J W， LEI L， YU H P， et al. Research on defect detection algorithm based on coin mirror surface［J］. Journal of Chengdu University （Natural Science）， 2016， 35（3）： 245-247， 259.
3	KIM J M， RYOO H J. Inspection of coin surface defects using multiple eigen spaces［J］. The Journal of the Korea Contents Association， 2011， 11（3）： 18-25. 10.5392/jkca.2011.11.3.018
4	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation ［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 580-587. 10.1109/cvpr.2014.81
5	GIRSHICK R. Fast R-CNN ［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1440-1448. 10.1109/iccv.2015.169
6	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks ［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2015： 91-99.
7	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2980-2988. 10.1109/iccv.2017.322
8	LIU W， ANGUELOV D， ERHAN D， et al. SSD： single shot multiBox detector ［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS9905. Cham： Springer， 2016： 21-37.
9	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 779-788. 10.1109/cvpr.2016.91
10	金侠挺，王耀南，张辉，等.基于贝叶斯CNN和注意力网络的钢轨表面缺陷检测系统［J］.自动化学报， 2019， 45（12）： 2312-2327. 10.16383/j.aas.c190143
	JIN X T， WANG Y N， ZHANG H， et al. DeepRail： automatic visual detection system for railway surface defect using Bayesian CNN and attention network［J］. Acta Automatica Sinica， 2019， 45（12）： 2312-2327. 10.16383/j.aas.c190143
11	徐镪，朱洪锦，范洪辉，等.改进的YOLOv3网络在钢板表面缺陷检测研究［J］.计算机工程与应用， 2020， 56（16）： 265-272. 10.3778/j.issn.1002-8331.2003-0232
	XU Q， ZHU H J， FAN H H， et al. Study on detection of steel plate surface defects by improved YOLOv3 network［J］. Computer Engineering and Applications， 2020， 56（16）： 265-272. 10.3778/j.issn.1002-8331.2003-0232
12	REDMON J， FARHADI A. YOLOv3： an incremental improvement［EB/OL］. （2018-04-08）［2021-01-23］. . 10.1109/cvpr.2018.00430
13	LIU S T， HUANG D， WANG Y H. Learning spatial fusion for single-shot object detection［EB/OL］. （2019-11-25）［2021-01-23］. . 10.3390/rs11212484
14	陈静，毛莺池，陈豪，等.基于改进单点多盒检测器的大坝缺陷目标检测方法［J］.计算机应用， 2021， 41（8）： 2366-2372.
	CHEN J， MAO Y C， CHEN H， et al. Dam defect object detection method based on improved single shot multibox detector［J］. Journal of Computer Applications， 2021， 41（8）： 2366-2372.
15	DAI J F， QI H Z， XIONG Y W， et al. Deformable convolutional networks ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 764-773. 10.1109/iccv.2017.89
16	LONG X， DENG K P， WANG G Z， et al. PP-YOLO： an effective and efficient implementation of object detector［EB/OL］. （2020-08-03）［2021-01-23］. .
17	ZHU X Z， HU H， LIN S， et al. Deformable ConvNets v2： more deformable， better results ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 9300-9308. 10.1109/cvpr.2019.00953
18	LEI T， WANG R S， ZHANG Y X， et al. DefED-Net： deformable encoder-decoder network for liver and liver tumor segmentation［J］. IEEE Transactions on Radiation and Plasma Medical Sciences， 2021（Early Access）： 1-1. 10.1109/TRPMS.2021.3059780
19	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 936-944. 10.1109/cvpr.2017.106
20	LIN T Y， MAIRE M， BELONGIE S J. et al. Microsoft COCO： common objects in context［C］// Proceedings of the 2014 European Conference on Computer Vision.Piscataway： IEEE， 2014： 740-755. 10.1007/978-3-319-10602-1_48
21	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2020， 42（2）： 318-327. 10.1109/tpami.2018.2858826
22	LIU S， QI L， QIN H F， et al. Path aggregation network for instance segmentation ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 8759-8768. 10.1109/cvpr.2018.00913

[1]	Zhangjian JI, Ming ZHANG, Zilong WANG. High-precision object detection algorithm based on improved VarifocalNet [J]. Journal of Computer Applications, 2023, 43(7): 2147-2154.
[2]	Yongxiang GU, Xin LAN, Boyi FU, Xiaolin QIN. Object detection algorithm for remote sensing images based on geometric adaptation and global perception [J]. Journal of Computer Applications, 2023, 43(3): 916-922.
[3]	Guohuan HE, Jiangping ZHU. WT-U-Net++： surface defect detection network based on wavelet transform [J]. Journal of Computer Applications, 2023, 43(10): 3260-3266.
[4]	SUN Zeqiang, CHEN Bingcai, CUI Xiaobo, WANG Lei, LU Yanuo. Strip steel surface defect detection by YOLOv5 algorithm fusing frequency domain attention mechanism and decoupled head [J]. Journal of Computer Applications, 2023, 43(1): 242-249.
[5]	GUO Zihao, DONG Lele, QU Zhijian. Arthropod object detection method based on improved Faster RCNN [J]. Journal of Computer Applications, 2023, 43(1): 88-97.
[6]	Hanqing LIU, Xiaodong KANG, Fuqing ZHANG, Xiuyuan ZHAO, Jingyi YANG, Xiaotian WANG, Mengfan LI. Image detection algorithm of cerebral arterial stenosis by improved Libra region-convolutional neural network [J]. Journal of Computer Applications, 2022, 42(9): 2909-2916.
[7]	Yajiao LIU, Haitao YU, Jiang WANG, Lifeng YU, Chunhui ZHANG. Surface detection algorithm of multi-shape small defects for section steel based on deep learning [J]. Journal of Computer Applications, 2022, 42(8): 2601-2608.
[8]	Yi ZHUANG, Haitao ZHAO. Proposal-based aggregation network for single object tracking in 3D point cloud [J]. Journal of Computer Applications, 2022, 42(5): 1407-1416.
[9]	CHEN Jing, MAO Yingchi, CHEN Hao, WANG Longbao, WANG Zicheng. Dam defect object detection method based on improved single shot multibox detector [J]. Journal of Computer Applications, 2021, 41(8): 2366-2372.
[10]	SHI Yangxiao, ZHANG Jun, CHEN Peng, WANG Bing. Classification of steel surface defects based on lightweight network [J]. Journal of Computer Applications, 2021, 41(6): 1836-1841.
[11]	ZHANG Jing'ai, WANG Jiangtao. Magnetic tile surface quality recognition based on multi-scale convolution neural network and within-class mixup operation [J]. Journal of Computer Applications, 2021, 41(1): 275-279.
[12]	HUANG Dongyan, LI Lang. Optimal bitcoin transaction fee payment strategy based on queuing game [J]. Journal of Computer Applications, 2020, 40(9): 2646-2649.
[13]	BIAN Xiaoyong, JIANG Peiling, ZHAO Min, DING Sheng, ZHANG Xiaolong. Multi-branch neural network model based weakly supervised fine-grained image classification method [J]. Journal of Computer Applications, 2020, 40(5): 1295-1300.
[14]	ZHAO Yulong, NIU Baoning, LI Peng, FAN Xing. Blockchain enhanced lightweight node model [J]. Journal of Computer Applications, 2020, 40(4): 942-946.
[15]	FAN Li, ZHENG Hong, HUANG Jianhua, LI Zhongcheng, JIANG Yahui. Cooperative evolution method for blockchain mining pool based on adaptive zero-determinant strategy [J]. Journal of Computer Applications, 2019, 39(3): 918-923.

Coin surface defect detection algorithm based on deformable convolution and adaptive spatial feature fusion

基于可变形卷积和自适应空间特征融合的硬币表面缺陷检测算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 22

Related Articles 15

Recommended Articles

Metrics