Application of anisotropic non-maximum suppression in industrial target detection

doi:10.11772/j.issn.1001-9081.2021040648

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (7): 2210-2218.DOI: 10.11772/j.issn.1001-9081.2021040648

• Multimedia computing and computer simulation • Previous Articles Next Articles

Application of anisotropic non-maximum suppression in industrial target detection

Shiwen ZHANG¹^,²^,³, Chunhua DENG¹^,²^,³(), Junwen ZHANG¹^,²^,³

^1.School of Computer Science and Technology，Wuhan University of Science and Technology，Wuhan Hubei 430065，China
^2.Institute of Big Data Science and Engineering，Wuhan University of Science and Technology，Wuhan Hubei 430065，China
^3.Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System （Wuhan University of Science and Technology），Wuhan Hubei 430065，China

Received:2021-04-25 Revised:2021-06-25 Accepted:2021-07-09 Online:2022-07-15 Published:2022-07-10
Contact: Chunhua DENG
About author:ZHANG Shiwen， born in 1997， M. S. candidate. His research interests include computer vision， machine learning.
ZHANG Junwen， born in 1997， M. S. candidate. Her research interests include computer vision， machine learning.
Supported by:
National Natural Science Foundation of China(61806150)

各向异性非极大值抑制在工业目标检测中的应用

张诗文¹^,²^,³, 邓春华¹^,²^,³(), 张俊雯¹^,²^,³

^1.武汉科技大学计算机科学与技术学院, 武汉 430065
^2.武汉科技大学大数据科学与工程研究院, 武汉 430065
^3.智能信息处理与实时工业系统湖北省重点实验室(武汉科技大学), 武汉 430065

通讯作者: 邓春华
作者简介:张诗文（1997—），男，湖北建始人，硕士研究生，主要研究方向：计算机视觉、机器学习
张俊雯（1997—），女，湖北荆门人，硕士研究生，主要研究方向：计算机视觉、机器学习。
基金资助:
国家自然科学基金资助项目(61806150)

Abstract

Abstract:

In certain fixed industrial application scenarios， the tolerance of the target detection algorithms to miss detection is very low. However， while increasing the recall， some non-overlapping virtual frames are likely to be regularly generated around the target. The traditional Non-Maximum Suppression （NMS） strategy has the main function to suppress multiple repeated detection frames of the same target， and cannot solve the above problem. To this end， an anisotropic NMS method was designed by adopting different suppression strategies for different directions around the target， and was able to effectively eliminate the regular virtual frames. The target shape and the regular virtual frame in a fixed industrial scene often have a certain relevance. In order to promote the accurate execution of anisotropic NMS in different directions， a ratio Intersection over Union （IoU） loss function was designed to guide the model to fit the shape of the target. In addition， an automatic labeling dataset augmentation method was used for the regular target， which reduced the workload of manual labeling and enlarged the scale of the dataset. Experimental results show that the proposed method has significant effects on the roll groove detection dataset， and when it is applied to the YOLO （You Only Look Once） series of algorithms， the detection precision is improved without reducing the speed. At present， the algorithm has been successfully applied to the production line of a cold rolling mill that automatically grabs rolls.

Key words: anisotropic, Non-Maximum Suppression (NMS), Intersection over Union (IoU), target detection, YOLO (You Only Look Once)

摘要：

在某些固定的工业应用场景中，对目标检测算法的漏检容忍性非常低。然而，提升召回率的同时，目标周围容易规律性地产生一些无重叠的虚景框。传统的非极大值抑制（NMS）策略主要作用是抑制同一目标的多个重复检测框，无法解决上述问题。为此设计了一种各向异性NMS方法来对目标周围不同方向采取不同的抑制策略，从而有效消除规律性的虚景框。固定的工业场景中的目标形状和规律的虚景框往往具有一定关联性。为了促进各向异性NMS在不同方向的精确执行，设计了一种比例交并比（IoU）损失函数用来引导模型拟合目标的形状。此外，针对规则目标使用了一种自动标注的数据集增广方法，在降低人工标注工作量的同时扩大了数据集规模。实验结果表明，所提方法在轧辊凹槽检测数据集上的效果显著，应用于YOLO系列算法时在不降低速度的同时提升了检测精度。目前该算法已成功应用于某冷轧厂轧辊自动抓取的生产线。

关键词: 各向异性, 非极大值抑制, 交并比, 目标检测, YOLO

CLC Number:

TP391.41

Shiwen ZHANG, Chunhua DENG, Junwen ZHANG. Application of anisotropic non-maximum suppression in industrial target detection[J]. Journal of Computer Applications, 2022, 42(7): 2210-2218.

张诗文, 邓春华, 张俊雯. 各向异性非极大值抑制在工业目标检测中的应用[J]. 《计算机应用》唯一官方网站, 2022, 42(7): 2210-2218.

Figures/Tables 15

Fig. 1 NMS results under different degrees of confidence

Fig. 2 Target detection box distribution in industrial scenes

Fig. 3 Schematic diagram of ellipse rotation angle

Fig. 4 NMS suppression results in a limited area

Fig. 5 Groove dataset distribution

Fig. 6 Curve image of function νR

Fig. 7 Generation process of dataset

Fig. 8 Comparison of different ways of filling

Fig. 9 Display of some samples

Fig. 10 Slice operation in Focus structure

Tab. 1 Allocation of anchor boxes at different scale detection layers

尺度	YOLOv5默认	本文聚类
19×19	［116，90］［156，198］［373，326］	［54，200］［71，242］［102，314］
38×38	［30，61］［62，45］［59，119］	［27，102］［35，121］［45，155］
76×76	［10，13］［16，30］［33，23］	［11，42］［16，60］［22，79］

Fig. 11 Comparison of original and anisotropic NMS

Fig. 12 Comparison of DIoU and ratio IoU loss

Tab. 2 Effect of different combinations of the proposed method and original model

方法	RIoU	NMS_l	mAP/%	FPS
YOLOv5s			71.9	64.6
YOLOv5s+CIoU			72.1	63.2
YOLOv5s+RIoU	√		72.8	66.0
YOLOv5s+NMS_l		√	76.9	63.6
YOLOv5s+CIoU+NMS_l		√	77.5	65.9
YOLOv5s+RIoU+NMS_l	√	√	79.2	64.5

Tab. 3 Comparison of original YOLO series of algorithms and them adding the proposed method

方法	mAP@.5/%	mAP@.5：.95/%	FPS
YOLOv3	92.4	66.6	47.3
YOLOv3+CIoU	93.7	67.3	48.3
YOLOv4^［43］	93.3	72.2	28.2
YOLOv4+CIoU	94.2	73.3	30.8
YOLOv5s	94.2	71.9	64.6
YOLOv3+RIoU+NMS_l	94.0	68.0	55.4
YOLOv4+RIoU+NMS_l	96.7	75.4	31.3
YOLOv5s+RIoU+NMS_l	97.7	79.2	64.5

References 43

1	LIU W， ANGUELOV D， ERHAN D， et al. SSD： single shot multibox detector［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS 9905. Cham： Springer， 2016： 21-37.
2	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 779-788. 10.1109/cvpr.2016.91
3	REDMON J， FARHADI A. YOLO9000： better， faster， stronger［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6517-6525. 10.1109/cvpr.2017.690
4	REDMON J， FARHADI A. YOLOv3： an incremental improvement［EB/OL］. （2018-04-08）［2021-01-08］..
5	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 580-587. 10.1109/cvpr.2014.81
6	GIRSHICK R. Fast R-CNN［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1440-1448. 10.1109/iccv.2015.169
7	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149. 10.1109/tpami.2016.2577031
8	HE K M， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2980-2988. 10.1109/iccv.2017.322
9	LI Z M， PENG C， YU G， et al. Light-head R-CNN： in defense of two-stage object detector［EB/OL］. （2017-11-23）［2021-01-08］..
10	CAI Z W， VASCONCELOS N. Cascade R-CNN： delving into high quality object detection［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 6154-6162. 10.1109/cvpr.2018.00644
11	FU C Y， LIU W， RANGA A， et al. DSSD： deconvolutional single shot detector［EB/OL］. （2017-01-23）［2021-01-08］..
12	PENG J， SU Y. An improved algorithm for detection and pose estimation of texture-less objects［J］. Journal of Advanced Computational Intelligence and Intelligent Informatics， 2021， 25（2）： 204-212. 10.20965/jaciii.2021.p0204
13	LAVIE A， SAGAE K， JAYARAMAN S. The significance of recall in automatic metrics for MT evaluation［C］// Proceedings of the 2004 Conference of the Association for Machine Translation in the Americas， LNCS 3265/LNAI 3265. Berlin： Springer， 2004： 134-143. 10.1007/978-3-540-30194-3_16
14	JUBA B， LE H S. Precision-recall versus accuracy and the role of large data sets［C］// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2019： 4039-4048. 10.1609/aaai.v33i01.33014039
15	MUKHERJEE S. Object detection［M］// ML.NET Revealed. Berkeley： Apress， 2021： 159-170. 10.1007/978-1-4842-6543-7_10
16	RAZAKARIVONY S， JURIE F. Vehicle detection in aerial imagery： a small target detection benchmark［J］. Journal of Visual Communication and Image Representation， 2016， 34： 187-203. 10.1016/j.jvcir.2015.11.002
17	GUO Y L， BENNAMOUN M， SOHEL F， et al. An integrated framework for 3-D modeling， object detection， and pose estimation from point-clouds［J］. IEEE Transactions on Instrumentation and Measurement， 2015， 64（3）： 683-693. 10.1109/tim.2014.2358131
18	ZHUANG J F， YANG L J， LI J. An improved segmentation algorithm based on super pixel for typical industrial applications［C］// Proceedings of the 11th International Symposium on Computational Intelligence and Design. Piscataway： IEEE， 2018： 366-370. 10.1109/iscid.2018.10184
19	CATENI S， COLLA V， VANNUCCI M. A method for resampling imbalanced datasets in binary classification tasks for real-world problems［J］. Neurocomputing， 2014， 135： 32-41. 10.1016/j.neucom.2013.05.059
20	JACQUES J C S， Jr， LAPEDRIZA A， PALMERO C， et al. Person perception biases exposed： revisiting the first impressions dataset ［C］// Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision Workshops. Piscataway： IEEE， 2020： 13-21. 10.1109/wacvw52041.2021.00006
21	ROSENFELD A， THURSTON M. Edge and curve detection for visual scene analysis［J］. IEEE Transactions on Computers， 1971， C-20（5）： 562-569. 10.1109/t-c.1971.223290
22	HARRIS C， STEPHENS M. A combined corner and edge detector［C］// Proceedings of the 1988 Alvey Vision Conference. ［S.l.］： Alvety Vision Club， 1988： No.23. 10.5244/c.2.23
23	VIOLA P， JONES M. Rapid object detection using a boosted cascade of simple features［C］// Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2001： Ⅰ-511-Ⅰ-518.
24	FELZENSZWALB P F， GIRSHICK R B， McALLESTER D， et al. Object detection with discriminatively trained part-based models［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2010， 32（9）： 1627-1645. 10.1109/tpami.2009.167
25	DALAL N， TRIGGS B. Histograms of oriented gradients for human detection［C］// Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2005： 886-893.
26	GIRSHICK R， DONAHUE J， DARRELL T， et al. Rich feature hierarchies for accurate object detection and semantic segmentation［C］// Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2014： 580-587. 10.1109/cvpr.2014.81
27	BODLA N， SINGH B， CHELLAPPA R， et al. Soft-NMS — improving object detection with one line of code［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 5562-5570. 10.1109/iccv.2017.593
28	YU J H， JIANG Y N， WANG Z Y， et al. UnitBox： an advanced object detection network［C］// Proceedings of the 24th ACM International Conference on Multimedia. New York： ACM， 2016： 516-520. 10.1145/2964284.2967274
29	REZATOFIGHI H， TSOI N， GWAK J， et al. Generalized intersection over union： a metric and a loss for bounding box regression［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 658-666. 10.1109/cvpr.2019.00075
30	SONG T， SUN L Y， XIE D， et al. Small-scale pedestrian detection based on topological line localization and temporal feature aggregation［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211/LNIP 11211. Cham： Springer， 2018： 554-569. 10.1007/978-3-030-01234-2_33
31	LAW H， DENG J. CornerNet： detecting objects as paired keypoints［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11218/LNIP 11218. Cham： Springer， 2018： 765-781. 10.1007/978-3-030-01264-9_45
32	YANG Z， LIU S H， HU H， et al. RepPoints： point set representation for object detection［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 9656-9665. 10.1109/iccv.2019.00975
33	ZHU C C， HE Y H， SAVVIDES M. Feature selective anchor-free module for single-shot object detection［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 840-849. 10.1109/cvpr.2019.00093
34	LIN T Y， GOYAL P， GIRSHICK R， et al. Focal loss for dense object detection［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2999-3007. 10.1109/iccv.2017.324
35	CUI Y， JIA M L， LIN T Y， et al. Class-balanced loss based on effective number of samples［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 9260-9269. 10.1109/cvpr.2019.00949
36	LI B Y， LIU Y， WANG X G. Gradient harmonized single-stage detector［C］// Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2019： 8577-8584. 10.1609/aaai.v33i01.33018577
37	TIAN Z， SHEN C H， CHEN H， et al. FCOS： fully convolutional one-stage object detection［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 9626-9635. 10.1109/iccv.2019.00972
38	ZHENG Z H， WANG P， LIU W， et al. Distance-IoU loss： faster and better learning for bounding box regression［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 12993-13000. 10.1609/aaai.v34i07.6999
39	YANG T， ZHANG X Y， LI Z M， et al. MetaAnchor： learning to detect objects with customized anchors［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2018： 318-328. 10.1016/j.ipl.2018.03.004
40	赵媛媛，朱军，谢亚坤，等. 改进Yolo-v3的视频图像火焰实时检测算法［J］. 武汉大学学报（信息科学版）， 2021， 46（3）： 326-334.
	ZHAO Y Y， ZHU J， XIE Y K， et al. A real-time video flame detection algorithm based on improved Yolo-v3［J］. Geomatics and Information Science of Wuhan University， 2021， 46（3）： 326-334.
41	陈静，毛莺池，陈豪，等. 基于改进单点多盒检测器的大坝缺陷目标检测方法［J］. 计算机应用， 2021， 41（8）： 2366-2372.
	CHEN J， MAO Y C， CHEN H， et al. Dam defect object detection method based on improved single shot multibox detector［J］. Journal of Computer Applications， 2021， 41（8）： 2366-2372.
42	卢官有，顾正弘. 改进的YOLOv3安检包裹中危险品检测算法［J］.计算机应用与软件， 2021， 38（1）： 197-204. 10.3969/j.issn.1000-386x.2021.01.033
	LU G Y， GU Z H. A Dangerous goods detection algorithm based on improved YOLOv3［J］. Computer Applications and Software， 2021， 38（1）： 197-204. 10.3969/j.issn.1000-386x.2021.01.033
43	BOCHKOVSKIY A， WANG C Y， LIAO H Y M. YOLOv4： optimal speed and accuracy of object detection［EB/OL］. （2020-04-23）［2021-01-28］..

[1]	Zhangjian JI, Na DU. Tiny target detection based on improved VariFocalNet [J]. Journal of Computer Applications, 2024, 44(7): 2200-2207.
[2]	Yaping DENG, Yingjiang LI. Review of YOLO algorithm and its applications to object detection in autonomous driving scenes [J]. Journal of Computer Applications, 2024, 44(6): 1949-1958.
[3]	Tianhua CHEN, Jiaxuan ZHU, Jie YIN. Bird recognition algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1114-1120.
[4]	Zhanjun JIANG, Baijing WU, Long MA, Jing LIAN. Faster-RCNN water-floating garbage recognition based on multi-scale feature and polarized self-attention [J]. Journal of Computer Applications, 2024, 44(3): 938-944.
[5]	Yuliang ZHENG, Yunhua CHEN, Weijie BAI, Pinghua CHEN. Vehicle target detection by fusing event data and image frames [J]. Journal of Computer Applications, 2024, 44(3): 931-937.
[6]	Cunyi LIAO, Yi ZHENG, Weijin LIU, Huan YU, Shouyin LIU. Decoupling-fusing algorithm for multiple tasks with autonomous driving environment perception [J]. Journal of Computer Applications, 2024, 44(2): 424-431.
[7]	Yudong PANG, Zhixing LI, Weijie LIU, Tianhao LI, Ningning WANG. Small target detection model in overlooking scenes on tower cranes based on improved real-time detection Transformer [J]. Journal of Computer Applications, 2024, 44(12): 3922-3929.
[8]	Dahai LI, Bingtao LI, Zhendong WANG. Underwater target detection algorithm based on improved YOLOv8 [J]. Journal of Computer Applications, 2024, 44(11): 3610-3616.
[9]	Lin WANG, Jingliang LIU, Wuwei WANG. Small target detection method in UAV images based on fusion of dilated convolution and Transformer [J]. Journal of Computer Applications, 2024, 44(11): 3595-3602.
[10]	Meijia LIANG, Xinwu LIU, Xiaopeng HU. Small target detection algorithm for train operating environment image based on improved YOLOv3 [J]. Journal of Computer Applications, 2023, 43(8): 2611-2618.
[11]	Hui LIU, Linyu ZHANG, Fugang WANG, Rujin HE. Object detection algorithm based on attention mechanism and context information [J]. Journal of Computer Applications, 2023, 43(5): 1557-1564.
[12]	Zhouhua ZHU, Qi QI. Automatic detection and recognition of electric vehicle helmet based on improved YOLOv5s [J]. Journal of Computer Applications, 2023, 43(4): 1291-1296.
[13]	Haifeng LI, Fan ZHANG, Minnan PIAO, Huaichao WANG, Nansha LI, Zhongcheng GUI. Automatic detection of targets under airport pavement based on channel and spatial attention [J]. Journal of Computer Applications, 2023, 43(3): 930-935.
[14]	LI Kewen, YANG Jiantao, HUANG Zongchao. Improved YOLOv3 target detection based on boundary limit point features [J]. Journal of Computer Applications, 2023, 43(1): 81-87.
[15]	CHEN Haiyan, JIA Mingming, ZHAO Wenli, WANG Chanfei. Target detection of Ochotona curzoniae based on embedded Jetson TX2 [J]. Journal of Computer Applications, 2023, 43(1): 98-103.

Application of anisotropic non-maximum suppression in industrial target detection

各向异性非极大值抑制在工业目标检测中的应用

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 43

Related Articles 15

Recommended Articles

Metrics