PIPNet: A lightweight asphalt pavement crack image segmentation network

doi:10.11772/j.issn.1001-9081.2023050911

Abstract

Abstract: Crack segmentation is an important prerequisite for evaluating the damage degree of pavement diseases. In order to balance the effectiveness and real-time of deep neural network segmentation, a lightweight asphalt pavement crack segmentation neural network(PIPNet) based on U-Net encoder-decoder structure was proposed in this paper. The encoding part was an inverted pyramid structure. Multi-branch parallel dilated convolution module with different dilatation rates was proposed to extract multi-scale information from the top, middle and bottom features and reduce model complexity, which combined deep separable convolutions with ordinary convolutions and gradually reduced the number of parallel convolutions. Drawing on the characteristics of GhostNet network, an inverse residual lightweight module was designed, which embedded with parallel dual pooling attention. Testing results on GAPS384 dataset show that the proposed lightweight segmentation method improves its mIOU value by 1.10 percentage points, respectively when the Params and MFLOPs values are only about one-sixth of the ResNet50 encoding. The mIOU value is 4.14 and 9.95 percentage points higher than the lightweight GhostNet and SegNet networks, respectively. Experimental results show that the proposed method has high crack segmentation performance while reducing the model complexity, and has good adaptability to segmentation of different road crack images.

Key words: asphalt pavement image, crack segmentation, lightweight neural network, inverted pyramid structure, parallel dilated convolution

摘要： 裂缝分割是对路面病害损坏程度评估的重要前提，为平衡深度神经网络分割的有效性与实时性，提出一种基于U-Net编码-解码器结构的轻量化沥青路面裂缝图像分割网络(PIPNet)。编码部分为倒金字塔结构，提出了具有不同空洞率的多分支并行空洞卷积模块，结合深度可分离卷积和普通卷积，逐级减少并行卷积的个数，对表层、中层及底层特征提取多尺度信息并降低模型复杂度，借鉴GhostNet特点，设计了逆残差轻量化模块，嵌入并行双池化注意力。在GAPs384数据集上的测试结果表明，PIPNet在参数量和计算量仅为ResNet50编码近1/6的情况下，其平均交并比提高了1.10个百分点，且较轻量化GhostNet和SegNet分别高出4.14与9.95个百分点。实验结果表明，PIPNet在降低模型复杂度的同时，有着较高的裂缝分割性能，且对不同路面裂缝图像分割适应性良好。

关键词: 沥青路面图像, 裂缝分割, 轻量化神经网络, 倒金字塔结构, 并行空洞卷积

封筠毕健康霍一儒李家宽. 轻量化沥青路面裂缝图像分割网络PIPNet [J]. 《计算机应用》唯一官方网站, DOI: 10.11772/j.issn.1001-9081.2023050911.

Figures/Tables 12

References 22

1	沙爱民，童峥，高杰.基于卷积神经网络的路表病害识别与测量［J］.中国公路学报，2018，31（1）：1-10. 10.3969/j.issn.1001-7372.2018.01.001
	SHA A M， TONG Z， GAO J. Recognition and measurement of pavement disasters based on convolutional neural networks［J］. China Journal of Highway and Transport， 2018， 31（1）： 1-10. 10.3969/j.issn.1001-7372.2018.01.001
2	郝巨鸣，杨景玉，韩淑梅，等.引入Ghost模块和ECA的YOLOv4公路路面裂缝检测方法［J］.计算机应用，2023，43（4）：1284-1290. 10.11772/j.issn.1001-9081.2022030410
	HAO J M， YANG J Y， HAN S M， et al. YOLOv4 highway pavement crack detection method using Ghost module and ECA［J］. Journal of Computer Applications， 2023， 43（4）： 1284-1290. 10.11772/j.issn.1001-9081.2022030410
3	蔡逢煌，张岳鑫，黄捷.基于YOLOv3与注意力机制的桥梁表面裂痕检测算法［J］.模式识别与人工智能，2020，33（10）：926-933. 10.16451/j.cnki.issn1003-6059.202010007
	CAI F H， ZHANG Y X， HUANG J. Bridge surface crack detection algorithm based on YOLOv3 and attention mechanism［J］. Pattern Recognition and Artificial Intelligence， 2020， 33（10）： 926-933. 10.16451/j.cnki.issn1003-6059.202010007
4	EISENBACH M， STRICKER R， SEICHTER D， et al. How to get pavement distress detection ready for deep learning？ A systematic approach［C］// Proceedings of 2017 International Joint Conference on Neural Networks. Piscataway： IEEE， 2017： 2039-2047. 10.1109/ijcnn.2017.7966101
5	封筠，赵颖，毕健康，等.多级卷积神经网络的沥青路面裂缝图像层次化筛选［J］.图学学报，2021，42（5）：719-728. 10.11996/JG.j.2095-302X.2021050719
	FENG J， ZHAO Y， BI J K， et al. Multi-level convolutional neural network for asphalt pavement crack image hierarchical filtering［J］. Journal of Graphics， 2021， 42（5）： 719-728. 10.11996/JG.j.2095-302X.2021050719
6	张德津，李清泉，陈颖，等.基于空间聚集特征的沥青路面裂缝检测方法［J］.自动化学报，2016，42（3）：443-454.
	ZHANG D J， LI Q Q， CHEN Y， et al. Asphalt pavement crack detection based on spatial clustering feature［J］. Acta Automatica Sinica， 2016， 42（3）： 443-454.
7	RONNEBERGER O， FISCHER P， BROX T. U‑Net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241. 10.1007/978-3-319-24574-4_28
8	LIU S， HUANG D， WANG Y. Receptive field block net for accurate and fast object detection［C］// Proceedings of the 15th European Conference on Computer Vision. Cham： Springer， 2018： 404-419. 10.1007/978-3-030-01252-6_24
9	HAN K， WANG Y， TIAN Q， et al. GhostNet： more features from cheap operations［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 1580-1589. 10.1109/cvpr42600.2020.00165
10	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
11	WANG Q， WU B， ZHU P， et al. ECA-Net： efficient channel attention for deep convolutional neural networks［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 11534-11542. 10.1109/cvpr42600.2020.01155
12	ZOU Q， CAO Y， LI Q， et al. CrackTree： automatic crack detection from pavement images［J］. Pattern Recognition Letters， 2012， 33（3）： 227-238. 10.1016/j.patrec.2011.11.004
13	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognitions［C］// Proceedings of the 3rd International Conference on Learning Representations. ［S.l.］： Computational and Biological Learning Society， 2015： 1-14.
14	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
15	SANDLER M， HOWARD A， ZHU M， et al. MobileNetB2： inverted residuals and linear bottlenecks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 4510-4520. 10.1109/cvpr.2018.00474
16	MA N， ZHANG X， ZHENG H-T， et al. ShuffleNet V2： practical guidelines for efficient CNN architecture design［C］// Proceedings of the 15th European Conference on Computer Vision. Cham： Springer， 2018： 122-138. 10.1007/978-3-030-01264-9_8
17	TAN M， LE Q. EfficientNet： rethinking model scaling for convolutional neural networks［J］. Proceedings of International Conference on Machine Learning， 2019， 97： 6105-6114.
18	TAN M， CHEN B， PANG R， et al. MnasNet： platform-aware neural architecture search for mobile［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 2820-2828. 10.1109/cvpr.2019.00293
19	ZHAO H， SHI J， QI X， et al. Pyramid scene parsing network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2881-2890. 10.1109/cvpr.2017.660
20	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： a deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495. 10.1109/tpami.2016.2644615
21	DING X， GUO Y， DING G， et al. AcNet： strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 1911-1920. 10.1109/iccv.2019.00200
22	XIE E， WANG W， YU Z， et al. SegFormer： simple and efficient design for semantic segmentation with transformers［J］. Advances in Neural Information Processing Systems， 2021， 34： 12077-12090.

编码网络类型	特征网络	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
经典卷积网络	原始U‑Net^［7］	22.22	61.77	25.07	32.35	62.12	118.48
	VGG16^［13］	26.49	65.50	32.14	39.06	38.79	74.01
	ResNet50^［14］	30.97	71.88	36.25	44.76	62.74	119.67
轻量化网络	MobileNetV2^［15］	26.55	63.06	31.55	38.86	10.32	19.62
	ShuffleNetV2^［16］	24.73	69.40	28.61	36.90	11.82	22.55
	EfficientNetB0^［17］	27.70	69.94	32.31	40.90	11.68	22.25
	MnasNet^［18］	24.35	69.09	27.10	36.67	15.92	30.40
	GhostNet^［9］	27.93	70.32	33.35	41.24	10.15	19.34
	PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

编码网络类型	特征网络	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
经典卷积网络	原始U‑Net^［7］	22.22	61.77	25.07	32.35	62.12	118.48
	VGG16^［13］	26.49	65.50	32.14	39.06	38.79	74.01
	ResNet50^［14］	30.97	71.88	36.25	44.76	62.74	119.67
轻量化网络	MobileNetV2^［15］	26.55	63.06	31.55	38.86	10.32	19.62
	ShuffleNetV2^［16］	24.73	69.40	28.61	36.90	11.82	22.55
	EfficientNetB0^［17］	27.70	69.94	32.31	40.90	11.68	22.25
	MnasNet^［18］	24.35	69.09	27.10	36.67	15.92	30.40
	GhostNet^［9］	27.93	70.32	33.35	41.24	10.15	19.34
	PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

方法	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
PspNet^［19］	27.37	56.70	34.28	39.03	93.58	178.39
SegNet^［20］	22.12	65.08	26.23	34.19	21.12	40.27
AcNet^［21］	26.77	73.97	30.90	38.15	62.48	120.57
SegFormer^［22］	39.17	68.81	53.75	60.35	33.56	64.10
PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

方法	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
PspNet^［19］	27.37	56.70	34.28	39.03	93.58	178.39
SegNet^［20］	22.12	65.08	26.23	34.19	21.12	40.27
AcNet^［21］	26.77	73.97	30.90	38.15	62.48	120.57
SegFormer^［22］	39.17	68.81	53.75	60.35	33.56	64.10
PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

注意力模块	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
无注意力	28.53	69.97	34.53	41.71	10.15	19.34
SE^［10］	29.46	69.11	35.62	42.91	10.35	19.67
ECA^［11］	31.82	68.83	38.64	45.88	10.32	19.59
PDA-GMP	30.04	68.75	37.28	44.39	10.31	19.59
PDA	32.07	69.04	36.19	43.68	10.32	19.59