Incomplete instance guided aeroengine blade instance segmentation

doi:10.11772/j.issn.1001-9081.2023010037

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (1): 167-174.DOI: 10.11772/j.issn.1001-9081.2023010037

• Artificial intelligence • Previous Articles

Incomplete instance guided aeroengine blade instance segmentation

Rui HUANG¹, Chaoqun ZHANG¹, Xuyi CHENG¹, Yan XING¹(), Bao ZHANG²

^1.College of Computer Science and Technology，Civil Aviation University of China，Tianjin 300300，China
^2.College of Intelligence and Computing，Tianjin University，Tianjin 300000，China

Received:2023-01-15 Revised:2023-04-13 Accepted:2023-04-14 Online:2023-06-06 Published:2024-01-10
Contact: Yan XING
About author:HUANG Rui， born in 1987， Ph. D.， lecturer. His research interests include computer vision， image processing， machine learning.
ZHANG Chaoqun， born in 2000， M. S. candidate. His research interests include deep learning， computer vision.
CHENG Xuyi， born in 1997， M. S. candidate. His research interests include deep learning， computer vision.
ZHANG Bao， born in 1989， M. S.， engineer. His research interests include computer vision.
Supported by:
Scientific Research Startup Foundation of Civil Aviation University of China(2017QD15X)

不完整实例引导的航空发动机叶片实例分割

黄睿¹, 张超群¹, 成旭毅¹, 邢艳¹(), 张宝²

^1.中国民航大学计算机科学与技术学院, 天津 300300
^2.天津大学智能与计算学部, 天津 300000

通讯作者: 邢艳
作者简介:黄睿（1987—），男，宁夏中卫人，讲师，博士，CCF会员，主要研究方向：计算机视觉、图像处理、机器学习；
张超群（2000—），男，山东聊城人，硕士研究生，主要研究方向：深度学习、计算机视觉；
成旭毅（1997—），男，山西晋中人，硕士研究生，主要研究方向：深度学习、计算机视觉；
张宝（1989—），男，安徽灵璧人，工程师，硕士，主要研究方向：计算机视觉。
第一联系人：邢艳（1987—），女，河北沧州人，讲师，博士，CCF会员，主要研究方向：数据挖掘；
基金资助:
中国民航大学科研启动项目(2017QD15X)

Abstract

Abstract:

The current deep learning based instance segmentation methods cannot fully train the network model and result in sub-optimal segmentation results due to the lack of labeled engine blade data. To improve the precision of aeroengine blade instance segmentation， an aeroengine blade instance segmentation method based on incomplete instance guidance was proposed. Combining with an existing instance segmentation method and an interactive segmentation method， promising aeroengine blade instance segmentation results were obtained. First， a small amount of labeled data was used to train the instance segmentation network， which generated initial instance segmentation results of aeroengine blades. Secondly， the detected single blade instance was divided into foreground and background. By selecting foreground seed points and background seed points， the interactive segmentation method was used to generate complete segmentation results of the blade. After all the blade instances were processed in turn， the final segmentation result of engine blade instance was obtained by merging the results. All the 72 images were used to train the Sparse Instance activation map for real-time instance segmentation （SparseInst）， to produce the initial instance segmentation results. The testing dataset contained 56 images. The mean Average Precision （mAP） of the proposed method is higher than that of SparseInst by 5.1 percentage points. The mAP results of the proposed method are better than those of the state-of-the-art instance segmentation methods， e.g.， MASK R-CNN （Mask Region based Convolutional Neural Network）， YOLACT （You Only Look At CoefficienTs）， BMASK-RCNN （Boundary-preserving MASK R-CNN）.

Key words: aeroengine, instance segmentation, engine blade, damage detection, interactive segmentation

摘要：

当前基于深度学习的实例检测方法在进行发动机叶片分割时，由于缺少带标注的发动机叶片数据，导致无法充分训练网络模型，仅得到次优的分割结果。为了提升航空发动机叶片实例分割精度，提出了不完整实例引导的航空发动机叶片实例分割方法，通过结合已有的实例分割方法和交互式分割方法，可得到较好的发动机叶片分割结果。首先，使用少量标注数据训练实例分割网络，得到发动机叶片的初步分割结果；其次，将检测到的单个叶片分为前景和背景两部分，通过选择前景种子点和背景种子点，利用交互式分割方法的思想，产生完整的单个叶片的分割结果；依次处理完所有的叶片后，将结果合并得到最终的发动机叶片实例分割结果。使用72张图像训练基于稀疏实例激活图的实时实例分割方法（SparseInst）产生初始的实例分割结果，在56张图像上进行测试。所提方法的全类平均准确率（mAP）比SparseInst的全类平均准确率高5.1个百分点；且它的mAP结果均优于当前流行的实例分割方法MASK R-CNN（MASK Region based Convolutional Neural Network）、YOLACT （You Only Look At CoefficienTs）、BMASK-RCNN （Boundary-preserving MASK R-CNN）。

关键词: 航空发动机, 实例分割, 发动机叶片, 损伤检测, 交互式分割

CLC Number:

TP391.41

Rui HUANG, Chaoqun ZHANG, Xuyi CHENG, Yan XING, Bao ZHANG. Incomplete instance guided aeroengine blade instance segmentation[J]. Journal of Computer Applications, 2024, 44(1): 167-174.

黄睿, 张超群, 成旭毅, 邢艳, 张宝. 不完整实例引导的航空发动机叶片实例分割[J]. 《计算机应用》唯一官方网站, 2024, 44(1): 167-174.

Figures/Tables 17

Fig. 1 Segmentation result comparison between proposed method and SparseInst

Fig. 2 Framework of the proposed method

Fig. 3 Comparison of horizontal bounding box and rotating bounding box

Fig. 4 Candidate regions of positive and negative sample seed points

Tab. 1 Quantitative comparison of segmentation results of aeroengine blade among different instance segmentation methods

方法	mAP	AP_（0.5）	AP_（0.75）
BMASK R-CNN^［4］	0.393	0.695	0.381
YOLACT^［8］	0.325	0.571	0.318
MASK R-CNN^［2］	0.475	0.736	0.501
SparseInst^［7］	0.579	0.809	0.618
本文方法	0.630	0.855	0.725

Fig. 5 Segmentation results of different instance segmentation methods

Tab. 2 Effect of foreground and background points on instance segmentation result

方法	mAP	AP_（0.5）	AP_（0.75）
只使用背景点	0.098	0.112	0.100
只使用前景点	0.509	0.710	0.572
本文方法	0.630	0.855	0.725

Fig. 6 Effect of foreground and background points on instance segmentation results

Tab. 3 Effect of number of foreground points on instance segmentation result

K	mAP	AP_（0.5）	AP_（0.75）
3	0.518	0.706	0.589
4	0.630	0.855	0.725
5	0.629	0.853	0.725

Tab. 4 Comparison of segmentation metrics for results of MASK R-CNN and MASK R-CNN +proposed method

方法	mAP	AP_（0.5）	AP_（0.75）
MASK R-CNN^［2］	0.475	0.736	0.501
MASK R-CNN^［2］+本文方法	0.546	0.761	0.670

Fig. 7 Segmentation result comparison between MASK R-CNN and MASK R-CNN + proposed method

Tab. 5 Comparison of segmentation metrics for results of BMASK R-CNN and BMASK R-CNN +proposed method

方法	mAP	AP_（0.5）	AP_（0.75）
BMASK R-CNN^［4］	0.393	0.695	0.381
BMASK R-CNN^［4］+本文方法	0.424	0.732	0.415

Fig. 8 Segmentation result comparison between BMASK R-CNN and BMASK R-CNN + proposed method

Tab. 6 Comparison of segmentation metrics for results of YOLACT and YOLACT +proposed method

方法	mAP	AP_（0.5）	AP_（0.75）
YOLACT^［8］	0.325	0.571	0.318
YOLACT^［8］+本文方法	0.390	0.605	0.423

Fig. 9 Segmentation result comparison between YOLACT and YOLACT + proposed method

Tab. 7 Influence of threshold T1 and T2 on instance segmentation results

T₁	T₂	mAP	AP_（0.5）	AP_（0.75）
0.6	0.7	0.630	0.855	0.725
0.7	0.8	0.618	0.858	0.693
0.8	0.6	0.624	0.855	0.705
0.9	0.6	0.624	0.855	0.705
0.6	0.9	0.573	0.856	0.598

Fig. 10 Poor segmentation results obtained by proposed method

References 31

1	张勇.孔探工作在发动机维护中的重要性［J］.航空维修与工程， 2004（1）： 24-25. 10.3969/j.issn.1672-0989.2004.01.010
	ZHANG Y. The importance of borescope inspection in aircraft engine maintenance ［J］. Aviation Maintenance & Engineering， 2004（1）： 24-25. 10.3969/j.issn.1672-0989.2004.01.010
2	HE K， GKIOXARI G， DOLLÁR P， et al. Mask R-CNN ［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2980-2988. 10.1109/iccv.2017.322
3	LIN T-Y， MAIRE M， BELONGIE S， et al. Microsoft COCO： Common objects in context ［C］// Proceedings of the 2014 European Conference on Computer Vision. Cham： Springer， 2014： 740-755. 10.1007/978-3-319-10602-1_48
4	CHENG T， WANG X， HUANG L， et al. Boundary-preserving Mask R-CNN ［C］// Proceedings of the 2020 European Conference on Computer Vision. Cham： Springer， 2020： 660-676. 10.1007/978-3-030-58568-6_39
5	CHEN K， PANG J， WANG J， et al. Hybrid task cascade for instance segmentation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 4969-4978. 10.1109/cvpr.2019.00511
6	SOFIIUK K， PETROV I， BARINOVA O， et al. f-BRS： Rethinking backpropagating refinement for interactive segmentation ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 8620-8629. 10.1109/cvpr42600.2020.00865
7	CHENG T， WANG X， CHEN S， et al. Sparse instance activation for real-time instance segmentation ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 4423-4432. 10.1109/cvpr52688.2022.00439
8	旷可嘉.深度学习及其在航空发动机缺陷检测中的应用研究［D］.广州：华南理工大学， 2017.
	KUANG K J. Research on deep learning and its application on the defects detection for aero engine ［D］. Guangzhou： South China University of Technology， 2017.
9	KIM Y-H， LEE J-R. Videoscope-based inspection of turbofan engine blades using convolutional neural networks and image processing ［J］. Structural Health Monitoring， 2019， 18（5/6）： 2020-2039. 10.1177/1475921719830328
10	黄睿，段博坤，陈望，等.检测器和分类器级联的飞机发动机损伤检测［J］.中国图象图形学报， 2022， 27（11）： 3232-3242. 10.11834/jig.210734
	HUANG R， DUAN B K， CHEN W， et al. Detector and classifier cascaded aero-engine detection method ［J］. Journal of Image and Graphics， 2022， 27（11）： 3232-3242. 10.11834/jig.210734
11	LI D， LI Y， XIE Q， et al. Tiny defect detection in high-resolution aero-engine blade images via a coarse-to-fine framework ［J］. IEEE Transactions on Instrumentation and Measurement， 2021， 70： 1-12. 10.1109/tim.2021.3062175
12	ZHANG D， ZENG N， LIN L. Detection of blades damages in aero engine ［C］// Proceedings of the 2020 Chinese Automation Congress. Piscataway： IEEE， 2020： 6129-6134. 10.1109/cac51589.2020.9327202
13	BOLYA D， ZHOU C， XIAO F， et al. YOLACT： Real-time instance segmentation ［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 9156-9165. 10.1109/iccv.2019.00925
14	TIAN Z， SHEN C， CHEN H. Conditional convolutions for instance segmentation ［C］// Proceedings of the 2020 European Conference on Computer Vision. Cham： Springer， 2020： 282-298. 10.1007/978-3-030-58452-8_17
15	LIU S， QI L， QIN H， et al. Path aggregation network for instance segmentation ［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 8759-8768. 10.1109/cvpr.2018.00913
16	LIN G， MILAN A， SHEN C， et al. RefineNet： Multi-path refinement networks for high-resolution semantic segmentation ［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition， 2017： 5168-5177. 10.1109/cvpr.2017.549
17	WANG X， KONG T， SHEN C， et al. SOLO： Segmenting objects by locations ［C］// Proceedings of the 2020 European Conference on Computer Vision. Cham： Springer， 2020： 649-665. 10.1007/978-3-030-58523-5_38
18	樊玮，李晨炫，邢艳，等.航空发动机损伤图像的二分类到多分类递进式检测网络［J］.计算机应用， 2021， 41（8）： 2352-2357. 10.11772/j.issn.1001-9081.2020101575
	FAN W， LI C X， XING Y， et al. Binary classification to multiple classification progressive detection network for aero-engine damage images ［J］. Journal of Computer Applications， 2021， 41（8）： 2352-2357. 10.11772/j.issn.1001-9081.2020101575
19	CHEN L-C， PAPANDREOU G， KOKKINOS I， et al. DeepLab： Semantic image segmentation with deep convolutional nets， atrous convolution， and fully connected CRFs ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 40（4）： 834-848. 10.1109/tpami.2017.2699184
20	马佳良，陈斌，孙晓飞.基于改进的Faster R-CNN的通用目标检测框架［J］.计算机应用， 2021， 41（9）： 2712-2719. 10.11772/j.issn.1001-9081.2020111852
	MA J L， CHEN B， SUN X F. General object detection framework based on improved Faster R-CNN ［J］. Journal of Computer Applications， 2021， 41（9）： 2712-2719. 10.11772/j.issn.1001-9081.2020111852
21	ROTHER C， KOLMOGOROV V， BLAKE A. “GrabCut” interactive foreground extraction using iterated graph cuts ［J］. ACM Transactions on Graphics， 2004， 23（3）： 309-314. 10.1145/1015706.1015720
22	CHEN X， ZHAO Z， ZHANG Y， et al. FocalClick： Towards practical interactive image segmentation ［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 1290-1299. 10.1109/cvpr52688.2022.00136
23	SOFIIUK K， PETROV I A， KONUSHIN A. Reviving iterative training with mask guidance for interactive segmentation ［C］// Proceedings of the 2022 IEEE International Conference on Image Processing. Piscataway： IEEE， 2022： 3141-3145. 10.1109/icip46576.2022.9897365
24	XU N， PRICE B， COHEN S， et al. Deep interactive object selection ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 373-381. 10.1109/cvpr.2016.47
25	兰红，闵乐泉.多阈值优化交互式分割算法及其在医学图像中的应用［J］.计算机应用， 2013， 33（5）： 1435-1438， 1475. 10.3724/sp.j.1087.2013.01435
	LAN H， MIN L Q. Interactive segmentation algorithm optimized by multi-threshold with application in medical images ［J］. Journal of Computer Applications， 2013， 33（5）： 1435-1438， 1475. 10.3724/sp.j.1087.2013.01435
26	CHEN X， ZHAO Z， YU F， et al. Conditional diffusion for interactive segmentation ［C］// Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2021： 7325-7334. 10.1109/iccv48922.2021.00725
27	LIN Z， ZHANG Z， CHEN L-Z， et al. Interactive image segmentation with first click attention ［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 13336-13345. 10.1109/cvpr42600.2020.01335
28	ACUNA D， LING H， KAR A， et al. Efficient interactive annotation of segmentation datasets with polygon-RNN++ ［C］// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 859-868. 10.1109/cvpr.2018.00096
29	SHAMOS M I. Computational geometry ［D］. New Haven： Yale University， 1978.
30	SUN K， XIAO B， LIU D， et al. Deep high-resolution representation learning for human pose estimation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5686-5696. 10.1109/cvpr.2019.00584
31	GUPTA A， DOLLÁR P， GIRSHICK R. LVIS： A dataset for large vocabulary instance segmentation ［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5351-5359. 10.1109/cvpr.2019.00550

[1]	Yuanlong ZHAO, Yugang SHAN, Jie YUAN, Kangdi ZHAO. Object tracking based on instance segmentation and Pythagorean fuzzy decision-making [J]. Journal of Computer Applications, 2023, 43(6): 1930-1937.
[2]	Rong GAO, Jiawei SHEN, Xiongkai SHAO, Xinyun WU. Instance segmentation algorithm based on Fastformer and self-supervised contrastive learning [J]. Journal of Computer Applications, 2023, 43(4): 1062-1070.
[3]	Tianzouzi XIAO, Xiaobo ZHOU, Xin LUO, Qipeng TANG. Robust RGB-D SLAM system incorporating instance segmentation and clustering in dynamic environment [J]. Journal of Computer Applications, 2023, 43(4): 1220-1225.
[4]	MA Jialiang, CHEN Bin, SUN Xiaofei. General object detection framework based on improved Faster R-CNN [J]. Journal of Computer Applications, 2021, 41(9): 2712-2719.
[5]	FAN Wei, LI Chenxuan, XING Yan, HUANG Rui, PENG Hongjian. Binary classification to multiple classification progressive detection network for aero-engine damage images [J]. Journal of Computer Applications, 2021, 41(8): 2352-2357.
[6]	FENG Xingjie, ZHANG Tianze. Panoptic segmentation algorithm based on grouped convolution for feature fusion [J]. Journal of Computer Applications, 2021, 41(7): 2054-2061.
[7]	Yue ZHANG, Liang ZHANG, Fei XIE, Jiale YANG, Rui ZHANG, Yijian LIU. Road abandoned object detection algorithm based on optimized instance segmentation model [J]. Journal of Computer Applications, 2021, 41(11): 3228-3233.
[8]	TIAN Jin, YUAN Jiazheng, LIU Hongzhe. Instance segmentation based lane line detection and adaptive fitting algorithm [J]. Journal of Computer Applications, 2020, 40(7): 1932-1937.
[9]	FENG Tao, CHEN Bin, ZHANG Yuefei. Chromosome image segmentation framework based on improved Mask R-CNN [J]. Journal of Computer Applications, 2020, 40(11): 3332-3339.

Incomplete instance guided aeroengine blade instance segmentation

不完整实例引导的航空发动机叶片实例分割

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 17

References 31

Related Articles 9

Recommended Articles

Metrics