PIPNet： lightweight asphalt pavement crack image segmentation network

doi:10.11772/j.issn.1001-9081.2023050911

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (5): 1520-1526.DOI: 10.11772/j.issn.1001-9081.2023050911

• The 19th China Conference on Machine Learning (CCML 2023) • Previous Articles

PIPNet： lightweight asphalt pavement crack image segmentation network

Jun FENG(), Jiankang BI, Yiru HUO, Jiakuan LI

School of Information Science and Technology，Shijiazhuang Tiedao University，Shijiazhuang Hebei 050043，China

Received:2023-07-11 Revised:2023-08-15 Accepted:2023-08-21 Online:2023-08-24 Published:2024-05-10
Contact: Jun FENG
About author:BI Jiankang， born in 1996， M. S. His research interests include image segmentation， deep learning.
HUO Yiru， born in 2000， M. S. candidate. His research interests include image segmentation， deep learning.
LI Jiakuan， born in 1998， M. S. candidate. His research interests include defect detection， deep learning.
Supported by:
Key Project of Science and Technology Research of Hebei Provincial Colleges and Universities(ZD2021333)

轻量化沥青路面裂缝图像分割网络PIPNet

封筠(), 毕健康, 霍一儒, 李家宽

石家庄铁道大学信息科学与技术学院，石家庄 050043

通讯作者: 封筠
作者简介:毕健康（1996—），男，河北沧州人，硕士，主要研究方向：图像分割、深度学习
霍一儒（2000—），男，河北邢台人，硕士研究生，主要研究方向：图像分割、深度学习
李家宽（1998—），男，河北石家庄人，硕士研究生，主要研究方向：缺陷检测、深度学习。
第一联系人：封筠（1971—），女，河北石家庄人，教授，博士，CCF会员，主要研究方向：计算机视觉、机器学习
基金资助:
河北省高等学校科学技术研究重点项目(ZD2021333)

Abstract

Abstract:

Crack segmentation is an important prerequisite for evaluating the damage degree of pavement diseases. In order to balance the effectiveness and real-time of deep neural network segmentation， a lightweight asphalt pavement crack segmentation neural network based on U?Net encoder-decoder structure was proposed， namely PIPNet （Parallel dilated convolution of Inverted Pyramid Network）. The encoding part was an inverted pyramid structure. Multi-branch parallel dilated convolution module with different dilatation rates was proposed to extract multi-scale information from the top， middle and bottom features and reduce model complexity， which combined deep separable convolutions with ordinary convolutions and gradually reduced the number of parallel convolutions. Drawing on the characteristics of GhostNet， an inverse residual lightweight module was designed， which was embedded with parallel dual pooling attention. Test results on GAPs384 dataset show that， compared with ResNet50 encoding method， PIPNet has mIoU （mean Intersection over Union） 1.10 percentage points higher with only about one-sixth of parameter quantity and MFLOPs （Million FLOating Point operations）， and its mIoU is 4.14 and 9.95 percentage points higher than those of lightweight GhostNet and SegNet， respectively. Experimental results show that PIPNet has high crack segmentation performance while reducing the model complexity， and has good adaptability to segmentation of different pavement crack images.

Key words: asphalt pavement image, crack segmentation, lightweight neural network, inverted pyramid structure, parallel dilated convolution

摘要：

裂缝分割是对路面病害损坏程度评估的重要前提，为平衡深度神经网络分割的有效性与实时性，提出一种基于U?Net编码-解码结构的轻量化沥青路面裂缝图像分割网络PIPNet（Parallel dilated convolution of Inverted Pyramid Network）。编码部分为倒金字塔结构，提出了具有不同空洞率的多分支并行空洞卷积模块，结合深度可分离卷积和普通卷积，逐级减少并行卷积的个数，对表层、中层及底层特征提取多尺度信息并降低模型复杂度；同时借鉴GhostNet特点，设计了逆残差轻量化模块，嵌入并行双池化注意力。在GAPs384数据集上的测试结果表明，PIPNet在参数量（Params）和计算量（MFLOPs）仅为ResNet50编码近1/6的情况下，平均交并比（mIoU）提高了1.10个百分点，且较轻量化GhostNet和SegNet分别高出4.14与9.95个百分点。实验结果表明，PIPNet在降低模型复杂度的同时，有着较好的裂缝分割性能，且对不同路面裂缝图像分割适应性良好。

关键词: 沥青路面图像, 裂缝分割, 轻量化神经网络, 倒金字塔结构, 并行空洞卷积

CLC Number:

TP391.9

Jun FENG, Jiankang BI, Yiru HUO, Jiakuan LI. PIPNet： lightweight asphalt pavement crack image segmentation network[J]. Journal of Computer Applications, 2024, 44(5): 1520-1526.

封筠, 毕健康, 霍一儒, 李家宽. 轻量化沥青路面裂缝图像分割网络PIPNet[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1520-1526.

Figures/Tables 12

References 22

1	沙爱民，童峥，高杰.基于卷积神经网络的路表病害识别与测量［J］.中国公路学报，2018，31（1）：1-10. 10.3969/j.issn.1001-7372.2018.01.001
	SHA A M， TONG Z， GAO J. Recognition and measurement of pavement disasters based on convolutional neural networks［J］. China Journal of Highway and Transport， 2018， 31（1）： 1-10. 10.3969/j.issn.1001-7372.2018.01.001
2	郝巨鸣，杨景玉，韩淑梅，等.引入Ghost模块和ECA的YOLOv4公路路面裂缝检测方法［J］.计算机应用，2023，43（4）：1284-1290. 10.11772/j.issn.1001-9081.2022030410
	HAO J M， YANG J Y， HAN S M， et al. YOLOv4 highway pavement crack detection method using Ghost module and ECA［J］. Journal of Computer Applications， 2023， 43（4）： 1284-1290. 10.11772/j.issn.1001-9081.2022030410
3	蔡逢煌，张岳鑫，黄捷.基于YOLOv3与注意力机制的桥梁表面裂痕检测算法［J］.模式识别与人工智能，2020，33（10）：926-933. 10.16451/j.cnki.issn1003-6059.202010007
	CAI F H， ZHANG Y X， HUANG J. Bridge surface crack detection algorithm based on YOLOv3 and attention mechanism［J］. Pattern Recognition and Artificial Intelligence， 2020， 33（10）： 926-933. 10.16451/j.cnki.issn1003-6059.202010007
4	EISENBACH M， STRICKER R， SEICHTER D， et al. How to get pavement distress detection ready for deep learning？ A systematic approach［C］// Proceedings of 2017 International Joint Conference on Neural Networks. Piscataway： IEEE， 2017： 2039-2047. 10.1109/ijcnn.2017.7966101
5	封筠，赵颖，毕健康，等.多级卷积神经网络的沥青路面裂缝图像层次化筛选［J］.图学学报，2021，42（5）：719-728. 10.11996/JG.j.2095-302X.2021050719
	FENG J， ZHAO Y， BI J K， et al. Multi-level convolutional neural network for asphalt pavement crack image hierarchical filtering［J］. Journal of Graphics， 2021， 42（5）： 719-728. 10.11996/JG.j.2095-302X.2021050719
6	张德津，李清泉，陈颖，等.基于空间聚集特征的沥青路面裂缝检测方法［J］.自动化学报，2016，42（3）：443-454.
	ZHANG D J， LI Q Q， CHEN Y， et al. Asphalt pavement crack detection based on spatial clustering feature［J］. Acta Automatica Sinica， 2016， 42（3）： 443-454.
7	RONNEBERGER O， FISCHER P， BROX T. U‑Net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham： Springer， 2015： 234-241. 10.1007/978-3-319-24574-4_28
8	LIU S， HUANG D， WANG Y. Receptive field block net for accurate and fast object detection［C］// Proceedings of the 15th European Conference on Computer Vision. Cham： Springer， 2018： 404-419. 10.1007/978-3-030-01252-6_24
9	HAN K， WANG Y， TIAN Q， et al. GhostNet： more features from cheap operations［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 1580-1589. 10.1109/cvpr42600.2020.00165
10	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
11	WANG Q， WU B， ZHU P， et al. ECA-Net： efficient channel attention for deep convolutional neural networks［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 11534-11542. 10.1109/cvpr42600.2020.01155
12	ZOU Q， CAO Y， LI Q， et al. CrackTree： automatic crack detection from pavement images［J］. Pattern Recognition Letters， 2012， 33（3）： 227-238. 10.1016/j.patrec.2011.11.004
13	SIMONYAN K， ZISSERMAN A. Very deep convolutional networks for large-scale image recognitions［C］// Proceedings of the 3rd International Conference on Learning Representations. ［S.l.］： Computational and Biological Learning Society， 2015： 1-14.
14	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
15	SANDLER M， HOWARD A， ZHU M， et al. MobileNetB2： inverted residuals and linear bottlenecks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 4510-4520. 10.1109/cvpr.2018.00474
16	MA N， ZHANG X， ZHENG H-T， et al. ShuffleNet V2： practical guidelines for efficient CNN architecture design［C］// Proceedings of the 15th European Conference on Computer Vision. Cham： Springer， 2018： 122-138. 10.1007/978-3-030-01264-9_8
17	TAN M， LE Q. EfficientNet： rethinking model scaling for convolutional neural networks［J］. Proceedings of International Conference on Machine Learning， 2019， 97： 6105-6114.
18	TAN M， CHEN B， PANG R， et al. MnasNet： platform-aware neural architecture search for mobile［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 2820-2828. 10.1109/cvpr.2019.00293
19	ZHAO H， SHI J， QI X， et al. Pyramid scene parsing network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2881-2890. 10.1109/cvpr.2017.660
20	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： a deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495. 10.1109/tpami.2016.2644615
21	DING X， GUO Y， DING G， et al. AcNet： strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 1911-1920. 10.1109/iccv.2019.00200
22	XIE E， WANG W， YU Z， et al. SegFormer： simple and efficient design for semantic segmentation with transformers［J］. Advances in Neural Information Processing Systems， 2021， 34： 12077-12090.

编码网络类型	特征网络	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
经典卷积网络	原始U‑Net^［7］	22.22	61.77	25.07	32.35	62.12	118.48
	VGG16^［13］	26.49	65.50	32.14	39.06	38.79	74.01
	ResNet50^［14］	30.97	71.88	36.25	44.76	62.74	119.67
轻量化网络	MobileNetV2^［15］	26.55	63.06	31.55	38.86	10.32	19.62
	ShuffleNetV2^［16］	24.73	69.40	28.61	36.90	11.82	22.55
	EfficientNetB0^［17］	27.70	69.94	32.31	40.90	11.68	22.25
	MnasNet^［18］	24.35	69.09	27.10	36.67	15.92	30.40
	GhostNet^［9］	27.93	70.32	33.35	41.24	10.15	19.34
	PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

编码网络类型	特征网络	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
经典卷积网络	原始U‑Net^［7］	22.22	61.77	25.07	32.35	62.12	118.48
	VGG16^［13］	26.49	65.50	32.14	39.06	38.79	74.01
	ResNet50^［14］	30.97	71.88	36.25	44.76	62.74	119.67
轻量化网络	MobileNetV2^［15］	26.55	63.06	31.55	38.86	10.32	19.62
	ShuffleNetV2^［16］	24.73	69.40	28.61	36.90	11.82	22.55
	EfficientNetB0^［17］	27.70	69.94	32.31	40.90	11.68	22.25
	MnasNet^［18］	24.35	69.09	27.10	36.67	15.92	30.40
	GhostNet^［9］	27.93	70.32	33.35	41.24	10.15	19.34
	PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

方法	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
PspNet^［19］	27.37	56.70	34.28	39.03	93.58	178.39
SegNet^［20］	22.12	65.08	26.23	34.19	21.12	40.27
AcNet^［21］	26.77	73.97	30.90	38.15	62.48	120.57
SegFormer^［22］	39.17	68.81	53.75	60.35	33.56	64.10
PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

方法	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
PspNet^［19］	27.37	56.70	34.28	39.03	93.58	178.39
SegNet^［20］	22.12	65.08	26.23	34.19	21.12	40.27
AcNet^［21］	26.77	73.97	30.90	38.15	62.48	120.57
SegFormer^［22］	39.17	68.81	53.75	60.35	33.56	64.10
PIPNet	32.07	69.04	36.19	43.68	10.32	19.59

注意力模块	mIoU/%	P/%	R/%	F1/%	MFLOPs	Params/10⁶
无注意力	28.53	69.97	34.53	41.71	10.15	19.34
SE^［10］	29.46	69.11	35.62	42.91	10.35	19.67
ECA^［11］	31.82	68.83	38.64	45.88	10.32	19.59
PDA-GMP	30.04	68.75	37.28	44.39	10.31	19.59
PDA	32.07	69.04	36.19	43.68	10.32	19.59

PIPNet： lightweight asphalt pavement crack image segmentation network

轻量化沥青路面裂缝图像分割网络PIPNet

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 22

Related Articles 1

Recommended Articles

Metrics

空洞率	mIoU/%	P /%	R/%	F1/%	MFLOPs	Params/10⁶
1，3，5，7	32.07	69.04	36.19	43.68	10.32	19.59
1，3，5	30.70	64.33	38.00	44.56	5.11	9.67
1，3	30.91	38.56	37.63	45.23	4.62	8.75