Road abandoned object detection algorithm based on optimized instance segmentation model

doi:10.11772/j.issn.1001-9081.2021010073

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (11): 3228-3233.DOI: 10.11772/j.issn.1001-9081.2021010073

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Road abandoned object detection algorithm based on optimized instance segmentation model

Yue ZHANG¹, Liang ZHANG¹^,²(), Fei XIE¹^,², Jiale YANG¹, Rui ZHANG¹, Yijian LIU¹^,²

^1.School of Electrical and Automation Engineering，Nanjing Normal University，Nanjing Jiangsu，210023，China
^2.Nanjing Industry Institute for Advanced Intelligent Equipment，Nanjing Jiangsu，210042，China

Received:2021-01-14 Revised:2021-03-25 Accepted:2021-04-06 Online:2021-04-26 Published:2021-11-10
Contact: Liang ZHANG
About author:ZHANG Yue， born in 1995， M. S. candidate. Her research interests include deep learning， computer vision， target detection， instance segmentation
ZHANG Liang，born in 1974，M. S.，lecturer. His research interests include image processing，robot control
XIE Fei，born in 1983，Ph. D.，associate professor. His research interests include machine vision， deep learning， intelligent transportation，navigation and positioning
YANG Jiale，born in 1996，M. S. candidate. Her research interests include deep learning，computer vision
ZHANG Rui， born in 1997， M. S. candidate. His research interests include deep learning，computer vision
LIU Yijian， born in 1977， Ph. D.， associate professor. His research interests include robotics，embedded system.

基于实例分割模型优化的道路抛洒物检测算法

章悦¹, 张亮¹^,²(), 谢非¹^,², 杨嘉乐¹, 张瑞¹, 刘益剑¹^,²

^1.南京师范大学电气与自动化工程学院，南京 210023
^2.南京智能高端装备产业研究院，南京 210042

通讯作者: 张亮
作者简介:章悦（1995—），女，江苏南京人，硕士研究生，CCF 会员，主要研究方向：深度学习、计算机视觉、目标检测、实例分割
张亮（1974—），男，江苏扬州人，讲师，硕士，主要研究方向：图像处理、机器人控制
谢非（1983—），男，江苏徐州人，副教授，博士，主要研究方向：机器视觉、深度学习、智能交通、导航定位
杨嘉乐（1996—），女，江苏泰州人，硕士研究生，主要研究方向：深度学习、计算机视觉
张瑞（1997—），男，山东泰安人，硕士研究生，主要研究方向：深度学习、计算机视觉
刘益剑（1977—），男，江苏淮安人，副教授，博士，主要研究方向：机器人技术、嵌入式系统。

Abstract

Abstract:

In the field of traffic safety， the road abandoned objects easily cause traffic accidents and become potential traffic safety hazards. Focusing on the problems of low recognition rate and poor detection effect for different abandoned objects of traditional road abandoned object detection methods， a road abandoned object detection algorithm based on the optimized instance segmentation model CenterMask was proposed. Firstly， the residual network ResNet50 optimized by dilated convolution was used as the backbone neural network to extract image features and carry out the multi-scale processing. Then， the Fully Convolutional One-Stage （FCOS） target detector optimized by Distance Intersection over Union （DIoU） function was used to realize the detection and classification of road abandoned objects. Finally， the spatial attention-guided mask was used as the mask segmentation branch to realize the object shape segmentation， and the model training was realized by the transfer learning method. Experimental results show that， the detection rate of the proposed algorithm for road abandoned objects is 94.82%， and compared with the common instance segmentation algorithm Mask Region-Convolutional Neural Network （Mask R-CNN）， the proposed road abandoned object detection algorithm has the Average Precision （AP） increased by 8.10 percentage points in bounding box detection.

Key words: instance segmentation, road abandoned object, dilated convolution, Distance Intersection over Union (DIoU) function, deep learning

摘要：

在交通安全领域，道路抛洒物易引发交通事故，构成了交通安全隐患。针对传统抛洒物检测方式识别率低、对于多类抛洒物检测效果不佳等问题，提出了一种基于实例分割模型CenterMask优化的道路抛洒物检测算法。首先，使用空洞卷积优化的残差网络ResNet50作为主干神经网络来提取特征并进行多尺度处理；然后，通过距离交并比（DIoU）函数优化的全卷积单阶段（FCOS）目标检测器实现对抛洒物的检测和分类；最后，使用空间注意力引导掩膜作为掩膜分割分支来实现对于目标形态的分割，并采用迁移学习的方式实现模型的训练。实验结果表明，所提算法对于抛洒物目标的检测率为94.82%，相较常见实例分割算法Mask R-CNN，所提的道路抛洒物检测算法在边界框检测上的平均精度（AP）提高了8.10个百分点。

关键词: 实例分割, 道路抛洒物, 空洞卷积, 距离交并比函数, 深度学习

CLC Number:

TP391.41

Yue ZHANG, Liang ZHANG, Fei XIE, Jiale YANG, Rui ZHANG, Yijian LIU. Road abandoned object detection algorithm based on optimized instance segmentation model[J]. Journal of Computer Applications, 2021, 41(11): 3228-3233.

章悦, 张亮, 谢非, 杨嘉乐, 张瑞, 刘益剑. 基于实例分割模型优化的道路抛洒物检测算法[J]. 《计算机应用》唯一官方网站, 2021, 41(11): 3228-3233.

Figures/Tables 12

Fig. 1 Structure of instance segmentation model CenterMask

Fig. 2 Overall process of road abandoned object detection

Fig. 3 Structure of backbone neural network ResNet50

Fig. 4 Schematic diagram of dilated convolution comparison

Fig. 5 Structure of backbone network and multi-scale processing

Fig. 6 Schematic diagram of DIoU principle

Fig. 7 Original image in dataset and labeling process

Fig. 8 Recognition and segmentation results of several types of common road abandoned objects

Fig. 9 Comparison of recognition results of abandoned objects on the same road section

Tab. 1 Result comparison of optimization algorithms for road abandoned object detection

算法	AP	AP₅₀	AP₇₅	AP_s	AP_m	AP_l
CenterMask	54.30	89.50	65.60	47.20	66.80	59.30
CenterMask+DIoU	59.40	89.20	67.50	53.30	74.80	53.50
CenterMask+ Dilated CNN	60.40	91.30	71.00	49.80	73.90	66.60
本文算法	61.70	89.40	71.50	50.40	76.50	71.50

Tab. 2 Comparison of test performance of different algorithms

算法	AP/%		单张图像平均耗时/s	检测率/%
算法	边界框检测	掩膜分割	单张图像平均耗时/s	检测率/%
CenterMask	54.30	52.40	0.28	93.39
Mask R-CNN	53.60	50.30	0.35	93.14
YOLACT	42.80	41.60	0.11	92.56
本文算法	61.70	54.10	0.29	94.82

Fig. 10 Detection result comparison of different algorithms

References 22

1	KHATOONABADI S H， BAJIC I V. Video object tracking in the compressed domain using spatio-temporal Markov random field ［J］. IEEE Transactions on Image Processing， 2013， 22（1）： 300-313. 10.1109/tip.2012.2214049
2	ASVADI A， PEIXOTO P， NUNES U. Detection and tracking of moving objects using 2.5D motion grid ［C］// Proceedings of the IEEE 18th International Conference on Intelligent Transportation Systems. Piscataway： IEEE， 2015： 788-793. 10.1109/itsc.2015.133
3	汪贵平，马力旺，郭璐，等.高速公路抛洒物事件图像检测算法［J］.长安大学学报（自然科学版），2017，37（5）：81-88. 10.18057/icass2018.p.123
	WANG G P， MA L W， GUO L， et al. Image detection algorithm for incident of discarded things in highway ［J］. Journal of Chang’an University （Natural Science Edition）， 2017， 37（5）： 81-88. 10.18057/icass2018.p.123
4	李清瑶，邹皓，赵群，等.基于帧间差分自适应法的车辆抛洒物检测［J］.长春理工大学学报（自然科学版），2018，41（4）：108-113.
	LI Q Y， ZOU H， ZHAO Q， et al. Vehicles throwing detection based on inter-frame difference adaptive method ［J］. Journal of Changchun University of Science and Technology （Natural Science Edition）， 2018， 41（4）： 108-113.
5	金瑶，张锐，尹东.城市道路视频中小像素目标检测［J］.光电工程，2019，46（9）：74-81. 10.32657/10356/144136
	JIN Y， ZHANG R， YIN D. Object detection for small pixel in urban roads videos ［J］. Opto-Electronic Engineering， 2019， 46（9）： 74-81. 10.32657/10356/144136
6	程文冬，马勇，魏庆媛.驾驶人手机通话行为中基于图像特征决策融合的手势识别方法［J］.交通运输工程学报，2019，19（4）：171-181. 10.3969/j.issn.1671-1637.2019.04.016
	CHENG W D， MA Y， WEI Q Y. Hand gesture recognition method in driver’s phone-call behavior based on decision fusion of image features ［J］. Journal of Traffic and Transportation Engineering， 2019， 19（4）： 171-181. 10.3969/j.issn.1671-1637.2019.04.016
7	陆德彪，郭子明，蔡伯根，等.基于深度数据的车辆目标检测与跟踪方法［J］.交通运输系统工程与信息，2018，18（3）：55-62. 10.16097/j.cnki.1009-6744.2018.03.009
	LU D B， GUO Z M， CAI B G， et al. A vehicle detection and tracking method based on range data ［J］. Journal of Transportation System Engineering and Information Technology， 2018， 18（3）： 55-62. 10.16097/j.cnki.1009-6744.2018.03.009
8	孙首群，刘康亚，刘硕妍，等.铁路客运站复杂环境中的运动目标检测［J］.交通运输工程学报，2013，13（3）：113-120. 10.3969/j.issn.1671-1637.2013.03.016
	SUN S Q， LIU K Y， LIU S Y， et al. Moving target detection in complex environment of railway station ［J］. Journal of Traffic and Transportation Engineering， 2013， 13（3）： 113-120. 10.3969/j.issn.1671-1637.2013.03.016
9	周雨阳，龚艺，姚琳，等.无人机广域视频的机动车交通参数计算及分析［J］.交通运输系统工程与信息，2015，15（6）：67-73. 10.3969/j.issn.1009-6744.2015.06.011
	ZHOU Y Y， GONG Y， YAO L， et al. Calculation and analysis of the traffic parameters of vehicles based on the wide-area drone video ［J］. Journal of Transportation System Engineering and Information Technology， 2015， 15（6）： 67-73. 10.3969/j.issn.1009-6744.2015.06.011
10	郑文博，王坤峰，王飞跃.基于贝叶斯生成对抗网络的背景消减算法［J］.自动化学报，2018，44（5）：878-890.
	ZHENG W B， WANG K F， WANG F Y. Background subtraction algorithm with Bayesian generative adversarial networks ［J］. Acta Automatica Sinica， 2018， 44（5）： 878-890.
11	卢胜男，李小和.结合双向光流约束的特征点匹配车辆跟踪方法［J］.交通运输系统工程与信息，2017，17（4）：76-82. 10.16097/j.cnki.1009-6744.2017.04.012
	LU S N， LI X H. Vehicle tracking method using feature point matching combined with bidirectional optical flow ［J］. Journal of Transportation System Engineering and Information Technology， 2017， 17（4）： 76-82. 10.16097/j.cnki.1009-6744.2017.04.012
12	蔡彪，沈宽，付金磊，等.基于Mask R-CNN的铸件X射线DR图像缺陷检测研究［J］.仪器仪表学报，2020，41（3）：61-69.
	CAI B， SHEN K， FU J L， et al. Research on defect detection of X-ray DR images of casting based on Mask R-CNN ［J］. Chinese Journal of Scientific Instrument， 2020， 41（3）： 61-69.
13	TIAN Z， SHEN C H， CHEN H， et al. FCOS： fully convolutional one-stage object detection ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 9626-9635. 10.1109/iccv.2019.00972
14	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
15	YAMASHITA T， FURUKAWA H， FUJIYOSHI H. Multiple skip connections of dilated convolution network for semantic segmentation ［C］// Proceedings of the 2018 25th IEEE International Conference on Image Processing. Piscataway： IEEE， 2018： 1593-1597. 10.1109/icip.2018.8451033
16	ZHU X Z， CHENG D Z， ZHANG Z， et al. An empirical study of spatial attention mechanisms in deep networks ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 6687-6696. 10.1109/iccv.2019.00679
17	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149. 10.1109/tpami.2016.2577031
18	REDMON J， DIVVALA S， GIRSHICK R， et al. You only look once： unified， real-time object detection ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 779-788. 10.1109/cvpr.2016.91
19	MASCI J， GIUSTI A， CIRESAN D， et al. A fast learning algorithm for image segmentation with max-pooling convolutional networks ［C］// Proceedings of the 2013 IEEE International Conference on Image Processing. Piscataway： IEEE， 2013： 2713-2717. 10.1109/icip.2013.6738559
20	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 936-944. 10.1109/cvpr.2017.106
21	LIU Z C， WANG S. Broken corn detection based on an adjusted YOLO with focal loss ［J］. IEEE Access， 2019， 7： 68281-68289. 10.1109/access.2019.2916842
22	丁松涛，曲仕茹.基于深度学习的交通目标感兴趣区域检测［J］.中国公路学报，2018，31（9）：167-174. 10.3969/j.issn.1001-7372.2018.09.019
	DING S T， QU S R. Traffic object detection based on deep learning with region of interest selection ［J］. China Journal of Highway and Transport， 2018， 31（9）： 167-174. 10.3969/j.issn.1001-7372.2018.09.019

[1]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[2]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[3]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[4]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[5]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[6]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[7]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[8]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[9]	Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650.
[10]	Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263.
[11]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[12]	Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP： defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086.
[13]	Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318.
[14]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.
[15]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.

Road abandoned object detection algorithm based on optimized instance segmentation model

基于实例分割模型优化的道路抛洒物检测算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 22

Related Articles 15

Recommended Articles

Metrics