Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (10): 3260-3266.DOI: 10.11772/j.issn.1001-9081.2022091452
Special Issue: 多媒体计算与计算机仿真
• Multimedia computing and computer simulation • Previous Articles Next Articles
Received:
2022-09-30
Revised:
2022-12-17
Accepted:
2022-12-28
Online:
2023-03-23
Published:
2023-10-10
Contact:
Guohuan HE, Jiangping ZHU
About author:
ZHU Jiangping, born in 1984, Ph. D., associate professor. His research interests include computer vision, three dimensional reconstruction, defect detection.
Supported by:
通讯作者:
何国欢,朱江平
作者简介:
何国欢(1996—),男,陕西安康人,硕士研究生,主要研究方向:计算机视觉、缺陷检测;
基金资助:
CLC Number:
Guohuan HE, Jiangping ZHU. WT-U-Net++: surface defect detection network based on wavelet transform[J]. Journal of Computer Applications, 2023, 43(10): 3260-3266.
何国欢, 朱江平. WT-U-Net++:基于小波变换的表面缺陷检测网络[J]. 《计算机应用》唯一官方网站, 2023, 43(10): 3260-3266.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2022091452
卷积层 | 描述 |
---|---|
Conv0 | (3×3conv+BN+ReLU)×2 |
Conv1 | 3×3conv+BN+ReLU)×2 |
Conv2 | 3×3conv+BN+ReLU+1×1conv+BN+ReLU |
Conv3 | 1×1conv+BN+ReLU+3×3conv+BN+ReLU |
Dil-Conv | (3×3conv(空洞率为2)+BN+ReLU)+ 1×1conv+BN+ReLU+3×3conv+BN+ReLU |
Conv | 1×1conv+BN+ReLU+(3×3conv+BN+ReLU)×2 |
Tab. 1 Composition of convolution layer
卷积层 | 描述 |
---|---|
Conv0 | (3×3conv+BN+ReLU)×2 |
Conv1 | 3×3conv+BN+ReLU)×2 |
Conv2 | 3×3conv+BN+ReLU+1×1conv+BN+ReLU |
Conv3 | 1×1conv+BN+ReLU+3×3conv+BN+ReLU |
Dil-Conv | (3×3conv(空洞率为2)+BN+ReLU)+ 1×1conv+BN+ReLU+3×3conv+BN+ReLU |
Conv | 1×1conv+BN+ReLU+(3×3conv+BN+ReLU)×2 |
类别 | 参数 |
---|---|
CPU | Intel Core i7-9700K@3.60 GHz |
GPU | NVIDIA RTX3080Ti(12 GB) |
内存 | 32 GB |
运行环境 | PyCharm |
脚本语言 | Python3.7 |
Tab. 2 Experimental running environment
类别 | 参数 |
---|---|
CPU | Intel Core i7-9700K@3.60 GHz |
GPU | NVIDIA RTX3080Ti(12 GB) |
内存 | 32 GB |
运行环境 | PyCharm |
脚本语言 | Python3.7 |
数据集 | 缺陷图像数 | 尺寸 | 缺陷类型 |
---|---|---|---|
铁轨 | 67 | 160×1 024 | 孔洞 |
磁瓦 | 118 | 160×320 | 气孔、裂纹、磨损、断裂、不均匀 |
硅钢油污 | 16 | 320×240 | 擦痕、划痕等小损伤缺陷 |
Tab. 3 Datasets
数据集 | 缺陷图像数 | 尺寸 | 缺陷类型 |
---|---|---|---|
铁轨 | 67 | 160×1 024 | 孔洞 |
磁瓦 | 118 | 160×320 | 气孔、裂纹、磨损、断裂、不均匀 |
硅钢油污 | 16 | 320×240 | 擦痕、划痕等小损伤缺陷 |
序号 | 增强方式 | 描述 |
---|---|---|
1 | Blur | 使用随机大小的内核模糊输入图像 |
2 | VerticalFlip | 围绕X轴垂直翻转输入 |
3 | RandomGamma | 随机Gamma |
4 | ShiftScaleRotate | 随机应用仿射变换:平移、缩放和旋转 |
5 | GridDistortion | 网格失真 |
6 | ElasticTransform | 弹性变换 |
7 | HueSaturationValue | 随机色调、饱和度和值变化 |
8 | RGBShift | 随机平移R、G、B通道值 |
9 | RandomBrightness | 随机亮度变化 |
10 | RandomContrast | 随机对比度变化 |
11 | HueSaturationValue | 随机更改输入图像的色相、饱和度和值 |
12 | GaussianBlur | 使用具有随机核大小的高斯滤波器对 输入图像进行模糊处理 |
13 | ChannelShuffle | 随机重新排列输入RGB图像的通道 |
14 | RandomFog | 图像雾化 |
15 | OpticalDistortion | 对图像进行光学畸变 |
16 | MotionBlur | 运动模糊 |
17 | MedianBlur | 图像中值滤波 |
18 | GaussianBlur | 图像高斯平滑滤波 |
19 | GaussNoise | 给图像增加高斯噪声 |
20 | CLAHE | 对比度受限的自适应直方图均衡化 应用于输入图像 |
Tab. 4 Data augmentation methods
序号 | 增强方式 | 描述 |
---|---|---|
1 | Blur | 使用随机大小的内核模糊输入图像 |
2 | VerticalFlip | 围绕X轴垂直翻转输入 |
3 | RandomGamma | 随机Gamma |
4 | ShiftScaleRotate | 随机应用仿射变换:平移、缩放和旋转 |
5 | GridDistortion | 网格失真 |
6 | ElasticTransform | 弹性变换 |
7 | HueSaturationValue | 随机色调、饱和度和值变化 |
8 | RGBShift | 随机平移R、G、B通道值 |
9 | RandomBrightness | 随机亮度变化 |
10 | RandomContrast | 随机对比度变化 |
11 | HueSaturationValue | 随机更改输入图像的色相、饱和度和值 |
12 | GaussianBlur | 使用具有随机核大小的高斯滤波器对 输入图像进行模糊处理 |
13 | ChannelShuffle | 随机重新排列输入RGB图像的通道 |
14 | RandomFog | 图像雾化 |
15 | OpticalDistortion | 对图像进行光学畸变 |
16 | MotionBlur | 运动模糊 |
17 | MedianBlur | 图像中值滤波 |
18 | GaussianBlur | 图像高斯平滑滤波 |
19 | GaussNoise | 给图像增加高斯噪声 |
20 | CLAHE | 对比度受限的自适应直方图均衡化 应用于输入图像 |
网络 | 铁轨 | 磁瓦 | 硅钢油污 | |||
---|---|---|---|---|---|---|
IoU | DSC | IoU | DSC | IoU | DSC | |
U-Net | 0.611 | 0.733 | 0.661 | 0.791 | 0.688 | 0.809 |
UNet++ | 0.702 | 0.821 | 0.778 | 0.871 | 0.721 | 0.840 |
AttentionU-Net | 0.699 | 0.810 | 0.678 | 0.795 | 0.673 | 0.796 |
SegNet | 0.648 | 0.777 | 0.760 | 0.862 | 0.427 | 0.582 |
WT-U-Net++ | 0.758 | 0.856 | 0.814 | 0.897 | 0.784 | 0.879 |
Tab. 5 Comparison of experimental results among different networks
网络 | 铁轨 | 磁瓦 | 硅钢油污 | |||
---|---|---|---|---|---|---|
IoU | DSC | IoU | DSC | IoU | DSC | |
U-Net | 0.611 | 0.733 | 0.661 | 0.791 | 0.688 | 0.809 |
UNet++ | 0.702 | 0.821 | 0.778 | 0.871 | 0.721 | 0.840 |
AttentionU-Net | 0.699 | 0.810 | 0.678 | 0.795 | 0.673 | 0.796 |
SegNet | 0.648 | 0.777 | 0.760 | 0.862 | 0.427 | 0.582 |
WT-U-Net++ | 0.758 | 0.856 | 0.814 | 0.897 | 0.784 | 0.879 |
WT | MCI | ATT | D-Up | 铁轨 | 磁瓦 | ||
---|---|---|---|---|---|---|---|
IoU | DSC | IoU | DSC | ||||
0.709 | 0.826 | 0.728 | 0.837 | ||||
√ | 0.715 | 0.828 | 0.749 | 0.851 | |||
√ | √ | 0.722 | 0.832 | 0.795 | 0.885 | ||
√ | √ | √ | 0.742 | 0.843 | 0.803 | 0.893 | |
√ | √ | √ | 0.735 | 0.841 | 0.799 | 0.889 | |
√ | √ | √ | √ | 0.758 | 0.856 | 0.814 | 0.897 |
Tab. 6 Results of ablation experiment
WT | MCI | ATT | D-Up | 铁轨 | 磁瓦 | ||
---|---|---|---|---|---|---|---|
IoU | DSC | IoU | DSC | ||||
0.709 | 0.826 | 0.728 | 0.837 | ||||
√ | 0.715 | 0.828 | 0.749 | 0.851 | |||
√ | √ | 0.722 | 0.832 | 0.795 | 0.885 | ||
√ | √ | √ | 0.742 | 0.843 | 0.803 | 0.893 | |
√ | √ | √ | 0.735 | 0.841 | 0.799 | 0.889 | |
√ | √ | √ | √ | 0.758 | 0.856 | 0.814 | 0.897 |
1 | LIU Y, XU K, XU J. An improved MB-LBP defect recognition approach for the surface of steel plates[J]. Applied Sciences, 2019, 9(20): No.4222. 10.3390/app9204222 |
2 | SU B, CHEN H, ZHU Y, et al. Classification of manufacturing defects in multicrystalline solar cells with novel feature descriptor[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(12): 4675-4688. 10.1109/tim.2019.2900961 |
3 | LI S, LI D, YUAN W. Wood chip crack detection based on linear scale-space differential[J]. Measurement, 2021, 175: No.109095. 10.1016/j.measurement.2021.109095 |
4 | RYU S G, KOO G, KIM S W. An adaptive selection of filter parameters: defect detection in steel image using wavelet reconstruction method[J]. ISIJ International, 2020, 60(8):1703-1713. 10.2355/isijinternational.isijint-2019-464 |
5 | REN H, TIAN K, HONG S, et al. Visualized investigation of defect in cementitious materials with electrical resistance tomography[J]. Construction and Building Materials, 2019, 196:428-436. 10.1016/j.conbuildmat.2018.11.129 |
6 | MA N, GAO X, WANG C, et al. Influence of hysteresis effect on contrast of welding defects profile in magneto-optical image[J]. IEEE Sensors Journal, 2020, 20(24): 15034-15042. 10.1109/jsen.2020.3009478 |
7 | PAN Y, LU R, ZHANG T. FPGA-accelerated textured surface defect segmentation based on complete period Fourier reconstruction[J]. Journal of Real-Time Image Processing, 2020, 17(5): 1659-1673. 10.1007/s11554-019-00927-1 |
8 | LI R, TIAN F, CHEN S. Research on surface defect detection method of E-TPU midsole based on machine vision[J]. Journal of Computer and Communications, 2020, 8(11):145-160. 10.4236/jcc.2020.811011 |
9 | SOUKUP D, HUBER-MÖRK R. Convolutional neural networks for steel surface defect detection from photometric stereo images[C]// Proceedings of the 2014 International Symposium on Visual Computing, LNCS 8887. Cham: Springer, 2014: 668-677. |
10 | DEITSCH S, CHRISTLEIN V, BERGER S, et al. Automatic classification of defective photovoltaic module cells in electroluminescence images[J]. Solar Energy, 2019, 185:455-468. 10.1016/j.solener.2019.02.067 |
11 | 陈仁祥,詹赞,胡小林,等. 基于多注意力Faster RCNN的噪声干扰下印刷电路板缺陷检测[J]. 仪器仪表学报, 2021, 42(12):167-174. |
CHEN R X, ZHAN Z, HU X L, et al. Printed circuit board defect detection based on the multi-attentive Faster RCNN under noise interference[J]. Chinese Journal of Scientific Instrument, 2021, 42(12):167-174. | |
12 | 郭文明,刘凯,渠慧帆. 基于Faster R-CNN模型X-射线图像的焊接缺陷检测(英文)[J]. 北京邮电大学学报, 2019, 42(6):20-28. 10.13190/j.jbupt.2019-097 |
GUO W M, LIU K, QU H F. Welding defect detection of X-ray images based on Faster R-CNN model[J]. Journal of Beijing University of Posts and Telecommunications, 2019, 42(6):20-28. 10.13190/j.jbupt.2019-097 | |
13 | LI Y, HUANG H, XIE Q, et al. Research on a surface defect detection algorithm based on MobileNet-SSD[J]. Applied Sciences, 2018, 8(9): No.1678. 10.3390/app8091678 |
14 | 韩航迪,徐亦睿,孙博,等. 基于改进Tiny-YOLOv3网络的航天电子焊点缺陷主动红外检测研究[J]. 仪器仪表学报, 2020, 41(11):42-49. 10.19650/j.cnki.cjsi.J2006720 |
HAN H D, XU Y R, SUN B, et al. Using active thermography for defect detection of aerospace electronic solder joint base on the improved Tiny-YOLOv3 network[J]. Chinese Journal of Scientific Instrument, 2020, 41(11): 42-49. 10.19650/j.cnki.cjsi.J2006720 | |
15 | DUNG C V, ANH L D. Autonomous concrete crack detection using deep fully convolutional neural network[J]. Automation in Construction, 2019, 99: 52-58. 10.1016/j.autcon.2018.11.028 |
16 | BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495. 10.1109/tpami.2016.2644615 |
17 | ZOU Q, ZHANG Z, LI Q, et al. DeepCrack: learning hierarchical convolutional features for crack detection[J]. IEEE Transactions on Image Processing, 2019, 28(3): 1498-1512. 10.1109/tip.2018.2878966 |
18 | RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention, LNCS 9351. Cham: Springer, 2015: 234-241. |
19 | ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet++: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6):1856-1867. 10.1109/tmi.2019.2959609 |
20 | SHENSA M J. The discrete wavelet transform: wedding the a trous and Mallat algorithms[J]. IEEE Transactions on Signal Processing, 1992, 40(10):2464-2482. 10.1109/78.157290 |
21 | SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:2818-2826. 10.1109/cvpr.2016.308 |
22 | WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018: 3-19. |
23 | GAN J, LI Q, WANG J, et al. A hierarchical extractor-based visual rail surface inspection system[J]. IEEE Sensors Journal, 2017, 17(23): 7935-7944. 10.1109/jsen.2017.2761858 |
24 | HUANG Y, QIU C, YUAN K. Surface defect saliency of magnetic tile[J]. The Visual Computer, 2020, 36(1): 85-96. 10.1007/s00371-018-1588-5 |
25 | 硅钢油污数据集[DS/OL]. [2021-05-22].. 10.13188/2330-0396.1000040 |
Oil pollution defect database[DS/OL]. [2021-05-22].. 10.13188/2330-0396.1000040 | |
26 | RUSSELL B C, TORRALBA A, MURPHY K P, et al. LabelMe: a database and web-based tool for image annotation[J]. International Journal of Computer Vision, 2008, 77(1/2/3): 157-173. 10.1007/s11263-007-0090-8 |
27 | BUSLAEV A, IGLOVIKOV V I, KHVEDCHENYA E, et al. Albumentations: fast and flexible image augmentations[J]. Information, 2020, 11(2): No.125. 10.3390/info11020125 |
28 | OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: learning where to look for the pancreas[EB/OL]. (2018-05-20) [2022-02-24].. |
[1] | Tong CHEN, Fengyu YANG, Yu XIONG, Hong YAN, Fuxing QIU. Construction method of voiceprint library based on multi-scale frequency-channel attention fusion [J]. Journal of Computer Applications, 2024, 44(8): 2407-2413. |
[2] | Yuan TANG, Yanping CHEN, Ying HU, Ruizhang HUANG, Yongbin QIN. Relation extraction model based on multi-scale hybrid attention convolutional neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2011-2017. |
[3] | Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet [J]. Journal of Computer Applications, 2024, 44(4): 1310-1316. |
[4] | Mengmeng CHEN, Zhiwei QIAO. Sparse reconstruction of CT images based on Uformer with fused channel attention [J]. Journal of Computer Applications, 2023, 43(9): 2948-2954. |
[5] | Meijia LIANG, Xinwu LIU, Xiaopeng HU. Small target detection algorithm for train operating environment image based on improved YOLOv3 [J]. Journal of Computer Applications, 2023, 43(8): 2611-2618. |
[6] | Kai ZHANG, Zhengchu QIN, Yue LIU, Xinyi QIN. Multi-learning behavior collaborated knowledge tracing model [J]. Journal of Computer Applications, 2023, 43(5): 1422-1429. |
[7] | Ruilin JIANG, Renchao QIN. Multi-neural network malicious code detection model based on depthwise separable convolution [J]. Journal of Computer Applications, 2023, 43(5): 1527-1533. |
[8] | Qinghai XU, Shifei DING, Tongfeng SUN, Jian ZHANG, Lili GUO. Improved capsule network based on multipath feature [J]. Journal of Computer Applications, 2023, 43(5): 1330-1335. |
[9] | Xuedong HE, Shibin XUAN, Kuan WANG, Mengnan CHEN. DeepLabV3+ image segmentation algorithm fusing cumulative distribution function and channel attention mechanism [J]. Journal of Computer Applications, 2023, 43(3): 936-942. |
[10] | Tao PENG, Yalong KANG, Feng YU, Zili ZHANG, Junping LIU, Xinrong HU, Ruhan HE, Li LI. Pedestrian trajectory prediction based on multi-head soft attention graph convolutional network [J]. Journal of Computer Applications, 2023, 43(3): 736-743. |
[11] | Haifeng LI, Fan ZHANG, Minnan PIAO, Huaichao WANG, Nansha LI, Zhongcheng GUI. Automatic detection of targets under airport pavement based on channel and spatial attention [J]. Journal of Computer Applications, 2023, 43(3): 930-935. |
[12] | Li’an ZHU, Hong ZHANG. Nonhomogeneous image dehazing based on dual-branch conditional generative adversarial network [J]. Journal of Computer Applications, 2023, 43(2): 567-574. |
[13] | SUN Zeqiang, CHEN Bingcai, CUI Xiaobo, WANG Lei, LU Yanuo. Strip steel surface defect detection by YOLOv5 algorithm fusing frequency domain attention mechanism and decoupled head [J]. Journal of Computer Applications, 2023, 43(1): 242-249. |
[14] | Jinghu LI, Qianguo XING, Xiangyang ZHENG, Lin LI, Lili WANG. Noctiluca scintillans red tide extraction method from UAV images based on deep learning [J]. Journal of Computer Applications, 2022, 42(9): 2969-2974. |
[15] | Zanxia QIANG, Xianfu BAO. Residual attention deraining network based on convolutional long short-term memory [J]. Journal of Computer Applications, 2022, 42(9): 2858-2864. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||