Elongated pavement distress detection method based on convolutional neural network

doi:10.11772/j.issn.1001-9081.2021010206

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 265-272.DOI: 10.11772/j.issn.1001-9081.2021010206

• Frontier and comprehensive applications • Previous Articles Next Articles

Elongated pavement distress detection method based on convolutional neural network

Huiqing XU¹^,², Bin CHEN²(), Jingfei WANG³^,⁴, Zhiyi CHEN³^,⁴, Jian QIN⁵

^1.Chengdu Institute of Computer Applications，Chinese Academy of Sciences，Chengdu Sichuan 610041，China
^2.University of Chinese Academy of Sciences，Beijing 101408，China
^3.Guangdong Hualu Transport Technology Company Limited，Guangzhou Guangdong 510420，China
^4.Guangdong Jiaoke Testing Company Limited，Guangzhou Guangdong 510550，China
^5.Guangzhou Electronic Technology Company Limited，Chinese Academy of Sciences，Guangzhou Guangdong 510070，China

Received:2021-02-03 Revised:2021-04-25 Accepted:2021-04-28 Online:2022-01-11 Published:2022-01-10
Contact: Bin CHEN
About author:XU Huiqing， born in 1989， Ph. D. candidate. His research interests include target detection， semantic segmentation， intelligent detection of pavement distress.
CHEN Bin， born in 1970， Ph. D.， research fellow. His research interests include machine vision， deep learning.
WANG Jingfei， born in 1980， M. S.， senior engineer. His research interests include rapid pavement detection and evaluation.
CHEN Zhiyi， born in 1986， engineer. His research interests include rapid pavement detection.
QIN Jian， born in 1992， engineer. His research interests include industrial automation.

基于卷积神经网络的细长路面病害检测方法

许慧青¹^,², 陈斌²(), 王敬飞³^,⁴, 陈志毅³^,⁴, 覃健⁵

^1.中国科学院成都计算机应用研究所, 成都 610041
^2.中国科学院大学, 北京 101408
^3.广东华路交通科技有限公司, 广州 510420
^4.广东交科检测有限公司, 广州 510550
^5.中科院广州电子技术有限公司, 广州 510070

通讯作者: 陈斌
作者简介:许慧青（1989—），男，河南汝南人，博士研究生，主要研究方向：目标检测、语义分割、路面病害智能检测
陈斌（1970—），男，四川广汉人，研究员，博士，主要研究方向：机器视觉、深度学习
王敬飞（1980—），男，河南漯河人，高级工程师，硕士，主要研究方向：快速道路检测和评估
陈志毅（1986—），男，广东茂名人，工程师，主要研究方向：快速路面检测
覃健（1992—），男，广西贺州人，工程师，主要研究方向：工业自动化。

Abstract

Abstract:

Focusing on the problems of the large time consumption of manual detection and the insufficient precision of the current detection methods of elongated pavement distress， a two-stage elongated pavement distress detection method， named Epd RCNN （Elongated pavement distress Region-based Convolutional Neural Network）， which could accurately locate and classify the distress was proposed according to the weak semantic characteristics and abnormal geometric properties of the distress. Firstly， for the weak semantic characteristics of elongated pavement distress， a backbone network that reused low-level features and repeatedly fused the features of different stages was proposed. Secondly， in the training process， the high-quality positive samples for network training were generated by the anchor box mechanism conforming to the geometric property distribution of the distress. Then， the distress bounding boxes were predicted on a single high-resolution feature map， and a parallel cascaded dilated convolution module was used to this feature map to improve its multi-scale feature representation ability. Finally， for different shapes of region proposals， the region proposal features conforming to the distress geometric properties were extracted by the proposal feature improvement module composed of deformable Region of Interest Pooling （RoI Pooling） and spatial attention module. Experimental results show that the proposed method has the mean Average Precision （mAP） of 0.907 on images with sufficient illumination， the mAP of 0.891 on images with illumination problems and the comprehensive mAP of 0.899， indicating that the proposed method has good detection performance and robustness to illumination.

Key words: elongated pavement distress, Convolutional Neural Network (CNN), bounding box, geometric property, parallel cascaded dilated convolution, region proposal feature, spatial attention

摘要：

针对细长路面病害人工检测耗时长和当前检测方法精度不足的问题，依据病害的弱语义特性和异常几何属性，提出了能够精准定位和分类出病害的二阶段细长路面病害检测方法Epd RCNN。首先，针对细长路面病害的弱语义特性，提出了一种复用低层特征并反复融合不同阶段特征的骨干网络；其次，在训练过程中，使用一种符合病害几何属性分布的锚框机制来生成高质量的正样本供网络训练；然后，在单一高分辨率特征图上预测病害包围框，并针对该特征图使用并行级联空洞卷积模块来提升其多尺度特征表达能力；最后，针对形状各异的候选区域，使用由可变形感兴趣区域池化（RoI Pooling）和空间注意力模块组成的候选区域特征改良模块来提取符合病害几何属性的候选区域特征。实验结果表明，所提方法在光照充足图像上的平均准确率均值（mAP）为0.907，在存在光照问题图像上的mAP为0.891，综合mAP为0.899，表明该方法具有良好的检测性能和对光照的鲁棒性。

关键词: 细长路面病害, 卷积神经网络, 包围框, 几何属性, 并行级联空洞卷积, 候选区域特征, 空间注意力

CLC Number:

TP183

Huiqing XU, Bin CHEN, Jingfei WANG, Zhiyi CHEN, Jian QIN. Elongated pavement distress detection method based on convolutional neural network[J]. Journal of Computer Applications, 2022, 42(1): 265-272.

许慧青, 陈斌, 王敬飞, 陈志毅, 覃健. 基于卷积神经网络的细长路面病害检测方法[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 265-272.

Figures/Tables 11

References 34

1	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 936-944. 10.1109/cvpr.2017.106
2	LI Z M， PENG C， YU G， et al. DetNet： design backbone for object detection［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS11213. Cham： Springer， 2018： 339-354.
3	KRIZHEVSKY A， SUTSKEVER I， HINTON G E. ImageNet classification with deep convolutional neural networks［C］// Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2012： 1097-1105.
4	初秀民，王荣本，储江伟，等. 基于不变矩特征的沥青路面破损图像识别［J］. 吉林大学学报（工学版）， 2003， 33（1）： 1-7. 10.3969/j.issn.1671-5497.2003.01.001
	CHU X M， WANG R B， CHU J W， et al. Asphalt pavement surface distress image recognition based on moment invariant feature［J］. Journal of Jilin University （Engineering and Technology Edition）， 2003， 33（1）： 1-7. 10.3969/j.issn.1671-5497.2003.01.001
5	AYENU-PRAH A， ATTOH-OKINE N. Evaluating pavement cracks with bidimensional empirical mode decomposition［J］. EURASIP Journal on Advances in Signal Processing， 2008， 2008： No.861701. 10.1155/2008/861701
6	DIXIT A， WAGATSUMA H. Investigating the effectiveness of the Sobel operator in the MCA-based automatic crack detection［C］// Proceedings of the 4th International Conference on Optimization and Applications. Piscataway： IEEE， 2018： 1-6. 10.1109/icoa.2018.8370555
7	TANG J S， GU Y L. Automatic crack detection and segmentation using a hybrid algorithm for road distress analysis［C］// Proceedings of the 2013 IEEE International Conference on Systems， Man， and Cybernetics. Piscataway： IEEE， 2013： 3026-3030. 10.1109/smc.2013.516
8	LI Q Q， LIU X L. Novel approach to pavement image segmentation based on neighboring difference histogram method［C］// Proceedings of the 2008 Congress on Image and Signal Processing. Piscataway： IEEE， 2008： 792-796. 10.1109/CISP.2008.13
9	OLIVEIRA H， CORREIA P L. Automatic road crack segmentation using entropy and image dynamic thresholding［C］// Proceedings of the 17th European Signal Processing Conference. Piscataway： IEEE， 2009： 622-626. 10.1109/icip.2010.5653305
10	CHENG H D， MIYOJIM M. Automatic pavement distress detection system［J］. Information Sciences， 1998， 108（1/2/3/4）： 219-240. 10.1016/s0020-0255(97)10062-7
11	HU Y， ZHAO C X. A novel local binary pattern based methods for pavement crack detection［J］. Journal of Pattern Recognition Research， 2010， 5（1）： 140-147. 10.13176/11.167
12	SALMAN M， MATHAVAN S， KAMAL K， et al. Pavement crack detection using the Gabor filter［C］// Proceedings of the 16th International IEEE Conference on Intelligent Transportation Systems. Piscataway： IEEE， 2013： 2039-2044. 10.1109/itsc.2013.6728529
13	KAPELA R， ŚNIATAŁA P， TURKOT A， et al. Asphalt surfaced pavement cracks detection based on histograms of oriented gradients［C］// Proceedings of the 22nd International Conference “Mixed Design of Integrated Circuits and Systems”. Piscataway： IEEE， 2015： 579-584. 10.1109/mixdes.2015.7208590
14	HADJI I， WILDES R P. What do we understand about convolutional networks ？［EB/OL］. （2018-03-23）［2020-03-01］..
15	沙爱民，童峥，高杰. 基于卷积神经网络的路表病害识别与测量［J］. 中国路面学报， 2018， 31（1）： 1-10.
	SHA A M， TONG Z， GAO J. Recognition and measurement of pavement disasters based on convolutional neural networks［J］. China Journal of Highway and Transport， 2018， 31（1）： 1-10.
16	ZHANG L， YANG F， ZHANG Y D， et al. Road crack detection using deep convolutional neural network［C］// Proceedings of the 2016 IEEE International Conference on Image Processing. Piscataway： IEEE， 2016： 3708-3712. 10.1109/icip.2016.7533052
17	WANG X L， HU Z Z. Grid-based pavement crack analysis using deep learning［C］// Proceedings of the 4th International Conference on Transportation Information and Safety. Piscataway： IEEE， 2017： 917-924. 10.1109/ictis.2017.8047878
18	YANG X C， LI H， YU Y T， et al. Automatic pixel-level crack detection and measurement using fully convolutional network［J］. Computer-Aided Civil and Infrastructure Engineering， 2018， 33（12）： 1090-1109. 10.1111/mice.12412
19	HUANG H W， LI Q T， ZHANG D M. Deep learning based image recognition for crack and leakage defects of metro shield tunnel［J］. Tunnelling and Underground Space Technology， 2018， 77： 166-176. 10.1016/j.tust.2018.04.002
20	FAN R， BOCUS M J， ZHU Y L， et al. Road crack detection using deep convolutional neural network and adaptive thresholding［C］// Proceedings of the 2019 IEEE Intelligent Vehicles Symposium. Piscataway： IEEE， 2019： 474-479. 10.1109/ivs.2019.8814000
21	SHELHAMER E， LONG J， DARRELL T. Fully convolutional networks for semantic segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（4）： 640-651. 10.1109/tpami.2016.2572683
22	MAEDA H， SEKIMOTO Y， SETO T， et al. Road damage detection and classification using deep neural networks with smartphone images［J］. Computer-Aided Civil and Infrastructure Engineering， 2018， 33（12）： 1127-1141. 10.1111/mice.12387
23	WANG W Z， WU B， YANG S X， et al. Road damage detection and classification with Faster R-CNN［C］// Proceedings of the 2018 IEEE International Conference on Big Data. Piscataway： IEEE， 2018： 5220-5223. 10.1109/bigdata.2018.8622354
24	REN S Q， HE K M， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）：1137-1149. 10.1109/tpami.2016.2577031
25	GIRSHICK R. Fast R-CNN［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1440-1448. 10.1109/iccv.2015.169
26	ANAND S， GUPTA S， DARBARI V， et al. Crack-pot： autonomous road crack and pothole detection［C］// Proceedings of the 2018 Digital Image Computing： Techniques and Applications. Piscataway： IEEE， 2018： 1-6. 10.1109/dicta.2018.8615819
27	ZHU X Z， HU H， LIN S， et al. Deformable ConvNets V2： more deformable， better results［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 9300-9308. 10.1109/cvpr.2019.00953
28	WANG J D， SUN K， CHENG T H， et al. Deep high-resolution representation learning for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2021， 43（10）： 3349-3364. 10.1109/tpami.2020.2983686
29	HE K M， ZHANG X Y， REN S Q， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
30	BUSLAEV A， IGLOVIKOV V I， KHVEDCHENYA E， et al. Albumentations： fast and flexible image augmentations［J］. Information， 2020， 11（2）： No.125. 10.3390/info11020125
31	EVERINGHAM M， GOOL L VAN， WILLIAMS C K I， et al. The PASCAL Visual Object Classes （VOC） challenge［J］. International Journal of Computer Vision， 2010， 88（2）： 303-338. 10.1007/s11263-009-0275-4
32	CHEN K， PANG J Q， WANG J M， et al. MMDetection： open MMLab detection toolbox and benchmark［EB/OL］. （2019-06-17）［2020-07-12］..
33	LIN T Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： common objects in context［C］// Proceedings of the 2014 European Conference on Computer Vision， LNCS8693. Cham： Springer， 2014： 740-755.
34	LEE Y， PARK J. CenterMask： real-time anchor-free instance segmentation［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 13903-13912. 10.1109/cvpr42600.2020.01392

类别	训练集图像数	测试集图像数	图像的总数
纵向裂缝	1 866	207	2 073
横向裂缝	1 901	211	2 112
修补	1 923	213	2 136
修补不良	1 911	212	2 123

类别	训练集图像数	测试集图像数	图像的总数
纵向裂缝	1 866	207	2 073
横向裂缝	1 901	211	2 112
修补	1 923	213	2 136
修补不良	1 911	212	2 123

锚框机制	尺度	长宽比
FPN	［32， 64， 128， 256， 512］	［0.5， 1.0， 2.0］
MSAM	［32， 48， 64， 96， 128， 192， 384， 512， 768］	［0.04， 0.1， 0.5， 1.0， 2.0， 10.0， 25.0］

锚框机制	尺度	长宽比
FPN	［32， 64， 128， 256， 512］	［0.5， 1.0， 2.0］
MSAM	［32， 48， 64， 96， 128， 192， 384， 512， 768］	［0.04， 0.1， 0.5， 1.0， 2.0， 10.0， 25.0］

检测方法	骨干网络
Faster RCNN	ResNet101
Faster RCNN+	VovNetV2-57
Epd RCNN-	HRNETV2-W32
Epd RCNN	HRNet++

Elongated pavement distress detection method based on convolutional neural network

基于卷积神经网络的细长路面病害检测方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 11

References 34

Related Articles 15

Recommended Articles

Metrics

[1]	Yun LI, Fuyou WANG, Peiguang JING, Su WANG, Ao XIAO. Uncertainty-based frame associated short video event detection method [J]. Journal of Computer Applications, 2024, 44(9): 2903-2910.
[2]	Hong CHEN, Bing QI, Haibo JIN, Cong WU, Li’ang ZHANG. Class-imbalanced traffic abnormal detection based on 1D-CNN and BiGRU [J]. Journal of Computer Applications, 2024, 44(8): 2493-2499.
[3]	Chenqian LI, Jun LIU. Ultrasound carotid plaque segmentation method based on semi-supervision and multi-scale cascaded attention [J]. Journal of Computer Applications, 2024, 44(8): 2604-2610.
[4]	Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625.
[5]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[6]	Yuan TANG, Yanping CHEN, Ying HU, Ruizhang HUANG, Yongbin QIN. Relation extraction model based on multi-scale hybrid attention convolutional neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2011-2017.
[7]	Yangyi GAO, Tao LEI, Xiaogang DU, Suiyong LI, Yingbo WANG, Chongdan MIN. Crowd counting and locating method based on pixel distance map and four-dimensional dynamic convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2233-2242.
[8]	Mengyuan HUANG, Kan CHANG, Mingyang LING, Xinjie WEI, Tuanfa QIN. Progressive enhancement algorithm for low-light images based on layer guidance [J]. Journal of Computer Applications, 2024, 44(6): 1911-1919.
[9]	Jianjing LI, Guanfeng LI, Feizhou QIN, Weijun LI. Multi-relation approximate reasoning model based on uncertain knowledge graph embedding [J]. Journal of Computer Applications, 2024, 44(6): 1751-1759.
[10]	Wenshuo GAO, Xiaoyun CHEN. Point cloud classification network based on node structure [J]. Journal of Computer Applications, 2024, 44(5): 1471-1478.
[11]	Min SUN, Qian CHENG, Xining DING. CBAM-CGRU-SVM based malware detection method for Android [J]. Journal of Computer Applications, 2024, 44(5): 1539-1545.
[12]	Tianhua CHEN, Jiaxuan ZHU, Jie YIN. Bird recognition algorithm based on attention mechanism [J]. Journal of Computer Applications, 2024, 44(4): 1114-1120.
[13]	Lijun XU, Hui LI, Zuyang LIU, Kansong CHEN, Weixuan MA. 3D-GA-Unet： MRI image segmentation algorithm for glioma based on 3D-Ghost CNN [J]. Journal of Computer Applications, 2024, 44(4): 1294-1302.
[14]	Jie WANG, Hua MENG. Image classification algorithm based on overall topological structure of point cloud [J]. Journal of Computer Applications, 2024, 44(4): 1107-1113.
[15]	Jingxian ZHOU, Xina LI. UAV detection and recognition based on improved convolutional neural network and radio frequency fingerprint [J]. Journal of Computer Applications, 2024, 44(3): 876-882.

检测方法	骨干网络	平均精度				mAP	mAP_B	mAP_D
检测方法	骨干网络	纵向裂缝	横向裂缝	修补	修补不良	mAP	mAP_B	mAP_D
Faster RCNN	ResNet101	0.423	0.472	0.663	0.499	0.514	0.519	0.510
Faster RCNN +		0.456	0.495	0.683	0.534	0.542	0.551	0.533
Epd RCNN-		0.472	0.497	0.689	0.552	0.552	0.557	0.548
Epd RCNN		0.516	0.524	0.723	0.608	0.592	0.598	0.587
Faster RCNN	VoVNetV2-57	0.435	0.488	0.672	0.512	0.526	0.531	0.523
Faster RCNN +		0.446	0.492	0.692	0.523	0.538	0.542	0.534
Epd RCNN-		0.491	0.522	0.701	0.543	0.564	0.568	0.561
Epd RCNN		0.526	0.525	0.715	0.614	0.595	0.603	0.587
Faster RCNN	HRNetV2-W32	0.549	0.564	0.765	0.659	0.634	0.641	0.616
Faster RCNN +		0.556	0.596	0.779	0.662	0.648	0.651	0.646
Epd RCNN-		0.637	0.655	0.804	0.728	0.706	0.711	0.701
Epd RCNN		0.686	0.683	0.809	0.774	0.738	0.742	0.734
Faster RCNN	HRNet++	0.625	0.718	0.826	0.774	0.736	0.739	0.733
Faster RCNN +		0.703	0.732	0.846	0.792	0.768	0.771	0.766
Epd RCNN-		0.773	0.788	0.863	0.818	0.811	0.820	0.801
Epd RCNN		0.893	0.895	0.912	0.897	0.899	0.907	0.891

检测方法	骨干网络	平均精度				mAP	mAP_B	mAP_D
检测方法	骨干网络	纵向裂缝	横向裂缝	修补	修补不良	mAP	mAP_B	mAP_D
Faster RCNN	ResNet101	0.423	0.472	0.663	0.499	0.514	0.519	0.510
Faster RCNN +		0.456	0.495	0.683	0.534	0.542	0.551	0.533
Epd RCNN-		0.472	0.497	0.689	0.552	0.552	0.557	0.548
Epd RCNN		0.516	0.524	0.723	0.608	0.592	0.598	0.587
Faster RCNN	VoVNetV2-57	0.435	0.488	0.672	0.512	0.526	0.531	0.523
Faster RCNN +		0.446	0.492	0.692	0.523	0.538	0.542	0.534
Epd RCNN-		0.491	0.522	0.701	0.543	0.564	0.568	0.561
Epd RCNN		0.526	0.525	0.715	0.614	0.595	0.603	0.587
Faster RCNN	HRNetV2-W32	0.549	0.564	0.765	0.659	0.634	0.641	0.616
Faster RCNN +		0.556	0.596	0.779	0.662	0.648	0.651	0.646
Epd RCNN-		0.637	0.655	0.804	0.728	0.706	0.711	0.701
Epd RCNN		0.686	0.683	0.809	0.774	0.738	0.742	0.734
Faster RCNN	HRNet++	0.625	0.718	0.826	0.774	0.736	0.739	0.733
Faster RCNN +		0.703	0.732	0.846	0.792	0.768	0.771	0.766
Epd RCNN-		0.773	0.788	0.863	0.818	0.811	0.820	0.801
Epd RCNN		0.893	0.895	0.912	0.897	0.899	0.907	0.891