Binary classification to multiple classification progressive detection network for aero-engine damage images

doi:10.11772/j.issn.1001-9081.2020101575

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (8): 2352-2357.DOI: 10.11772/j.issn.1001-9081.2020101575

Special Issue: 前沿与综合应用

• Frontier and comprehensive applications • Previous Articles Next Articles

Binary classification to multiple classification progressive detection network for aero-engine damage images

FAN Wei¹, LI Chenxuan¹, XING Yan¹, HUANG Rui¹, PENG Hongjian²

1. College of Computer Science and Technology, Civil Aviation University of China, Tianjin 300300, China;
2. Maintenance and Engineering Department, Xiamen Airlines, Xiamen Fujian 361006, China

Received:2020-10-12 Revised:2021-01-05 Online:2021-08-10 Published:2021-01-27
Supported by:
This work is partially supported by the Special Program for CAUC of the Fundamental Research Funds for the Central Universities (3122018C020), the Scientific Research Program of Tianjin Municipal Education Commission (2019KJ126).

航空发动机损伤图像的二分类到多分类递进式检测网络

樊玮¹, 李晨炫¹, 邢艳¹, 黄睿¹, 彭洪健²

1. 中国民航大学计算机科学与技术学院, 天津 300300;
2. 厦门航空有限公司维修工程部, 福建厦门 361006

通讯作者: 黄睿
作者简介:樊玮(1968-),男,陕西咸阳人,教授,博士,CCF会员,主要研究方向:机器学习、收益管理;李晨炫(1994-),男,山东济宁人,硕士研究生,主要研究方向:深度学习、计算机视觉;邢艳(1987-),女,河北沧州人,讲师,博士,CCF会员,主要研究方向:数据挖掘;黄睿(1987-),男,宁夏中卫人,讲师,博士,CCF会员,主要研究方向:计算机视觉、图像处理、机器学习;彭洪健(1988-),男,福建龙岩人,工程师,主要研究方向:航空发动机孔探及叶片润滑。
基金资助:
中央高校基本科研业务费项目中国民航大学专项（3122018C020）；天津市教委科研计划项目（2019KJ126）。

Abstract

Abstract: Aero-engine damage is an important factor affecting flight safety. There are two main problems in the current computer vision-based damage detection of engine borescope image:one is that the complex background of borescope image makes the model detect the damage with low accuracy; the other one is that the data source of borescope image is limited, which leads to fewer detectable classes for the model. In order to solve these two problems, a Mask R-CNN (Mask Region-based Convolutional Neural Network) based progressive detection network from binary classification to multiple classification was proposed for aero-engine damage images. By adding a binary classification detection branch to the Mask R-CNN, firstly, the damage in the image was detected in binary way and regression optimization was performed to the localization coordinates. Secondly, the original detection branch was used to progressively perform multiple classification detection, so as to further optimize the damage detection results by regression and determine the damage class. Finally, instance segmentation was performed to the damage through the Mask branch according to the results of multiple classification detection. In order to increase the detection classes of the model and verify the effectiveness of the method, a dataset of 1 315 borescope images with 8 damage classes was constructed. The training and testing results on this set show that the Average Precision (AP) and AP75 (Average Precision under IoU (Intersection over Union) of 75%) of multiple classification detection are improved by 3.34% and 9.71% respectively, compared with those of Mask R-CNN. It can be seen that the proposed method can effectively improve the multiple classification detection accuracy for damages in borescope images.

Key words: borescope image, object detection, instance segmentation, aero-engine damage inspection, progressive detection

摘要： 航空发动机损伤是影响飞行安全的重要因素。当前基于计算机视觉的发动机孔探图像损伤检测存在两个主要问题：一是孔探图像背景复杂，使得模型对损伤的检测精度较低；二是孔探图像数据来源受限，导致模型可检测类别较少。为解决这两个问题，提出了基于Mask R-CNN的二分类到多分类递进式航空发动机损伤图像检测网络。通过在Mask R-CNN中增加二分类检测分支，首先对图像中的损伤进行二分类检测并对定位坐标进行回归优化；其次使用原始检测分支递进地进行多分类检测，以进一步回归优化损伤的检测结果并确定损伤类型；最后根据多分类检测的结果，通过Mask分支对对损伤进行实例分割。为了增加模型检测类别及验证方法的有效性，构建了包含八种损伤类型，共1 315张孔探图像的数据集。在该集合上进行的训练和测试结果表明，多分类检测的平均精度（AP）和AP75与Mask R-CNN相比分别提高3.34%、9.71%，可见所提方法能够有效提高对孔探图像中的损伤的多分类检测精度。

关键词: 孔探图像, 目标检测, 实例分割, 航空发动机探伤, 递进式检测

CLC Number:

TP391.4

FAN Wei, LI Chenxuan, XING Yan, HUANG Rui, PENG Hongjian. Binary classification to multiple classification progressive detection network for aero-engine damage images[J]. Journal of Computer Applications, 2021, 41(8): 2352-2357.

樊玮, 李晨炫, 邢艳, 黄睿, 彭洪健. 航空发动机损伤图像的二分类到多分类递进式检测网络[J]. 计算机应用, 2021, 41(8): 2352-2357.

References

[1] 裴广达. 孔探与现代航空发动机维护[J]. 航空工程与维修, 2000(5):27-28.(PEI G D. Borescope inspection and modern aeroengine maintenance[J]. Aviation Maintenance and Engineering,2000(5):27-28.)
[2] 丁莉芬. 机器视觉在航空发动机内窥检测中的应用研究[D]. 天津:中国民用航空学院,2005.(DING L F. Research on application of computer vision in hole-detection of aero-engine[D]. Tianjin:Civil Aviation University of China,2005.)
[3] SHEN Z J,WAN X L,YE F,et al. Deep learning based framework for automatic damage detection in aircraft engine borescope inspection[C]//Proceedings of the 2019 International Conference on Computing, Networking and Communications. Piscataway:IEEE,2019:1005-1010.
[4] 李华, 陈果, 陈新波,等. 航空发动机内部裂纹自动测量方法研究[J]. 计算机工程与应用,2016,52(11):233-237.(LI H, CHEN G,CHEN X B,et al. Study on automatic measurement method for aero-engine inner damage crack[J]. Computer Engineering and Applications,2016,52(11):233-237.)
[5] 李华, 陈果, 林桐, 等. 航空发动机叶片损伤自动测量方法研究[J]. 航空计算技术,2015,45(1):52-55.(LI H,CHEN G,LIN T,et al. Study on automatic measurement method for damage aero engine lamina[J]. Aeronautical Computing Technique,2015,45(1):52-55.)
[6] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[2020-09-12]. https://arxiv.org/pdf/1409.1556.pdf.
[7] LONG J,SHELHAMER E,DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:3431-3440.
[8] REN S Q,HE K M,GIRSHICK R,et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017, 39(6):1137-1149.
[9] LIU W,ANGUELOV D,ERHAN D,et al. SSD:single shot MultiBox detector[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9905. Cham:Springer, 2016:21-37.
[10] HE K M,GKIOXARI G,DOLLÁR P,et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway:IEEE,2017:2980-2988.
[11] 李长有, 马齐爽, 姚红宇. 基于小波变换的孔探图像边缘粗糙度分析[J]. 北京航空航天大学学报,2007,33(6):705-708. (LI C Y,MA Q S,YAO H Y. Analysis of coarseness of edges extracted from borescope images based on wavelet transform[J]. Journal of Beijing University of Aeronautics and Astronautics, 2007,33(6):705-708.)
[12] 张勇, 刘冠军, 邱静. 基于图像自动测量的航空发动机故障检测技术研究[J]. 机械科学与技术,2008,27(2):176-179. (ZHANG Y,LIU G J,QIU J. Aeroengine's fault detection technology based on image automatic measurement[J]. Mechanical Science and Technology for Aerospace Engineering, 2008,27(2):176-179.)
[13] 陈果, 汤洋. 基于孔探图像纹理特征的航空发动机损伤识别方法[J]. 仪器仪表学报,2008,29(8):1709-1713.(CHEN G, TANG Y. Aero-engine interior damage recognition based on texture features of borescope image[J]. Chinese Journal of Scientific Instrument,2008,29(8):1709-1713.)
[14] 孟娇茹, 王娟. 孔探成像在发动机损伤检测中的应用研究[J]. 节能技术,2009,27(1):69-73. (MENG J R,WANG J. Research on application of video borescope during engine inspection[J]. Energy Conservation Technology,2009,27(1):69-73.)
[15] SVENSÉN M,HARDWICK D S,POWRIE H E G. Deep neural networks analysis of borescope images[J]. PHM Society European Conference,2018,4(1):No. 401.
[16] KIM Y H,LEE J R. Videoscope-based inspection of turbofan engine blades using convolutional neural networks and image processing[J]. Structural Health Monitoring,2019,18(5/6):2020-2039.
[17] BIAN X,LIM S N,ZHOU N. Multiscale fully convolutional network with application to industrial inspection[C]//Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision. Piscataway:IEEE,2016:1-8.
[18] 旷可嘉. 深度学习及其在航空发动机缺陷检测中的应用研究[D]. 广州:华南理工大学,2017.(KUANG K J. Research on deep learning and its application on the defects detection for aero engine[D]. Guangzhou:South China University of Technology, 2017.)
[19] 樊玮, 段博坤, 黄睿, 等. 基于风格迁移的交互式航空发动机孔探图像扩展方法[J]. 计算机应用,2020,40(12):3631-3636. (FAN W, DUAN B K, HUANG R, et al. Interactive augmentation method for aircraft engine borescope inspection images based on style transfer[J]. Journal of Computer Applications,2020,40(12):3631-3636.)
[20] DENG J,DONG W,SOCHER R,et al. ImageNet:a large-scale hierarchical image database[C]//Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2009:248-255.
[21] BOCHKOVSKIY A,WANG C Y,LIAO H Y M. YOLOv4:optimal speed and accuracy of object detection[EB/OL]. (2020-04-23)[2020-09-12]. https://arxiv.org/pdf/2004.10934.pdf.
[22] LIN T Y,MAIRE M,BELONGIE S,et al. Microsoft COCO:common objects in context[C]//Proceedings of the 2014 European Conference on Computer Vision,LNCS 8693. Cham:Springer, 2014:740-755.
[23] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:unified,real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2016:779-788.

Binary classification to multiple classification progressive detection network for aero-engine damage images

航空发动机损伤图像的二分类到多分类递进式检测网络

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	MA Jialiang, CHEN Bin, SUN Xiaofei. General object detection framework based on improved Faster R-CNN [J]. Journal of Computer Applications, 2021, 41(9): 2712-2719.
[2]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[3]	CHEN Jing, MAO Yingchi, CHEN Hao, WANG Longbao, WANG Zicheng. Dam defect object detection method based on improved single shot multibox detector [J]. Journal of Computer Applications, 2021, 41(8): 2366-2372.
[4]	XIAO Zhenyuan, WANG Yihan, LUO Jianqiao, XIONG Ying, LI Bailin. RefineDet based on subsection weighted loss function [J]. Journal of Computer Applications, 2021, 41(7): 1928-1932.
[5]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[6]	FENG Xingjie, ZHANG Tianze. Panoptic segmentation algorithm based on grouped convolution for feature fusion [J]. Journal of Computer Applications, 2021, 41(7): 2054-2061.
[7]	LI Chao, LAN Hai, WEI Xian. Attention-based object detection with millimeter wave radar-lidar fusion [J]. Journal of Computer Applications, 2021, 41(7): 2137-2144.
[8]	HOU Yunlong, ZHU Lei, CHEN Qin, LYU Suidong. Salient object detection based on difference of Gaussian feature network [J]. Journal of Computer Applications, 2021, 41(3): 706-713.
[9]	GUO Wenxu, SU Yuanqi, LIU Yuehu. YOLOv3 compression and acceleration based on ZYNQ platform [J]. Journal of Computer Applications, 2021, 41(3): 669-676.
[10]	CAO Jianrong, LYU Junjie, WU Xinying, ZHANG Xu, YANG Hongjuan. Fall detection algorithm integrating motion features and deep learning [J]. Journal of Computer Applications, 2021, 41(2): 583-589.
[11]	JIANG Ning, FANG Jinglong, YANG Qing. Global-local domain adaptive object detection based on single shot multibox detector [J]. Journal of Computer Applications, 2021, 41(2): 517-522.
[12]	ZHENG Sicheng, KONG Linghua, YOU Tongfei, YI Dingrong. Semantic SLAM algorithm based on deep learning in dynamic environment [J]. Journal of Computer Applications, 2021, 41(10): 2945-2951.
[13]	WANG Hongyu, ZHANG Yu, YANG Heng, MU Nan. Salient object detection in weak light images based on ant colony optimization algorithm [J]. Journal of Computer Applications, 2021, 41(10): 2970-2978.
[14]	PEI Yiyao, GUO Huiming, ZHANG Danpu, CHEN Wenbo. Robust 3D object detection method based on localization uncertainty [J]. Journal of Computer Applications, 2021, 41(10): 2979-2984.
[15]	FANG Shuya, LIU Shouyin. Non-perception class attendance method based on student body detection [J]. Journal of Computer Applications, 2020, 40(9): 2519-2524.