Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (12): 3631-3636.DOI: 10.11772/j.issn.1001-9081.2020040585

• Virtual reality and multimedia computing • Previous Articles     Next Articles

Interactive augmentation method for aircraft engine borescope inspection images based on style transfer

FAN Wei1, DUAN Bokun1, HUANG Rui1, LIU Ting1, ZHANG Ning2   

  1. 1. College of Computer Science and Technology, Civil Aviation University of China, Tianjin 300300, China;
    2. Information Department, Xiamen Airlines, Xiamen Fujian 361006, China
  • Received:2020-05-06 Revised:2020-07-07 Online:2020-12-10 Published:2020-07-20
  • Supported by:
    This work is partially supported by the Scientific Research Program of Tianjin Education Commission(2019KJ126), the Foundation Project for Central University of CAUC(3122018C020), the Scientific Research Starting Foundation of CAUC(600001050115).

基于风格迁移的交互式航空发动机孔探图像扩展方法

樊玮1, 段博坤1, 黄睿1, 刘挺1, 张宁2   

  1. 1. 中国民航大学 计算机科学与技术学院, 天津 300300;
    2. 厦门航空 信息部, 福建 厦门 361006
  • 通讯作者: 黄睿(1987-),男,宁夏中卫人,讲师,博士,主要研究方向:计算机视觉、机器学习.rhuang@cauc.edu.cn
  • 作者简介:樊玮(1968-),男,陕西咸阳人,教授,博士,主要研究方向:机器学习、收益管理;段博坤(1996-),男,山西临汾人,硕士研究生,主要研究方向:计算机视觉、机器学习;刘挺(1994-),男,陕西榆林人,硕士研究生,主要研究方向:计算机视觉、机器学习;张宁(1965-),男,浙江诸暨人,高级工程师,硕士,主要研究方向:计算机视觉
  • 基金资助:
    天津市教委科研计划项目(2019KJ126);中国民航大学中央高校基金资助项目(3122018C020);中国民航大学科研启动基金资助项目(600001050115)。

Abstract: The number of defect region samples is far less than that of the normal region samples in aircraft engine borescope inspection image defect detection task, and the defect samples cannot cover the whole sample space, which result in poor generalization of the detection algorithms. In order to solve the problems, a new interactive data augmentation method based on style transfer technology was proposed. Firstly, background image and defect targets were selected according to the interactive interface, and the informations such as size, angle and position of the target needed to be pasted were specified according to the background image. Then, the style of background image was transferred to the target image through style transfer technology, so that the background image and the target to be detected had the same style. Finally, the boundary of the fusion region was modified by Poisson fusion algorithm to achieve the effect of natural transition of the connected region. Two-class classification and defect detection were conducted to verify the effectiveness of the proposed method. The testers achieve 44.0% classification error rate for the two-class classification on the dataset with real images and augmented images averagely. In the detection task based on Mask Region-based Convolutional Neural Network (Mask R-CNN) model, the proposed method has the Average Precision (AP) of classification and segmentation improved by 99.5% and 91.9% respectively compared to those of the traditional methods.

Key words: interactive, Poisson fusion, style transfer, data augmentation, borescope inspection image

摘要: 在航空发动机孔探图像缺陷检测任务中,缺陷区域样本数量远少于正常样本数量,且缺陷样本无法覆盖整个样本空间,导致检测算法泛化能力较差。针对上述问题,提出了一种基于风格迁移技术的交互式数据扩展方法。首先,通过交互界面选择背景图像和缺陷目标,并根据背景图像指定需要粘贴的目标的大小、角度和位置等信息;其次,通过风格迁移技术将背景图像的风格迁移到目标图像上,使得背景图像和待检测目标具有相同的风格;最后,利用泊松融合算法对融合区域的边界进行修正,以达到连接区域自然过渡的效果。通过二分类和缺陷检测验证了该方法的有效性。在包含真实图像和扩展后图像的二分类实验中,测试人员的平均分类错误率达到了44.0%;而在基于Mask R-CNN模型的检测任务中,所提方法的检测和分割平均精度(AP)较传统方法分别提高了99.5%和91.9%。

关键词: 交互式, 泊松融合, 风格迁移, 数据扩展, 孔探图像

CLC Number: