Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (4): 1310-1316.DOI: 10.11772/j.issn.1001-9081.2023040453
Special Issue: 多媒体计算与计算机仿真
• Multimedia computing and computer simulation • Previous Articles Next Articles
Boyue WANG, Yingxiang LI(), Jiandan ZHONG
Received:
2023-04-21
Revised:
2023-07-05
Accepted:
2023-07-05
Online:
2023-12-04
Published:
2024-04-10
Contact:
Yingxiang LI
About author:
WANG Boyue, born in 1999, M. S. candidate. His research interests include intelligent image processing, artificial intelligence.Supported by:
通讯作者:
李英祥
作者简介:
王铂越(1999—),男,河南洛阳人,硕士研究生,主要研究方向:智能图像处理、人工智能基金资助:
CLC Number:
Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet[J]. Journal of Computer Applications, 2024, 44(4): 1310-1316.
王铂越, 李英祥, 钟剑丹. 基于改进Res-UNet的昼夜地基云图分割网络[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1310-1316.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2023040453
卷积层名称 | 输出尺寸 | 层内部结构 |
---|---|---|
Conv1 | 160×160 | 7×7,64,步长=2 |
Conv2_x | 80×80 | 残差块×3 |
Conv3_x | 40×40 | 残差块×4 |
Conv4_x | 20×20 | 残差块×6 |
Conv5_x | 10×10 | 残差块×3 |
Tab. 1 ResNet50 feature extraction network structure
卷积层名称 | 输出尺寸 | 层内部结构 |
---|---|---|
Conv1 | 160×160 | 7×7,64,步长=2 |
Conv2_x | 80×80 | 残差块×3 |
Conv3_x | 40×40 | 残差块×4 |
Conv4_x | 20×20 | 残差块×6 |
Conv5_x | 10×10 | 残差块×3 |
数据集 | 分割网络 | 准确率 | 精确率 | 召回率 | F1值 | ER | MIoU |
---|---|---|---|---|---|---|---|
白天 地基 云图 | U-Net | 0.942 | 0.918 | 0.917 | 0.916 | 0.058 | 0.847 |
PSPNet | 0.941 | 0.941 | 0.941 | 0.941 | 0.059 | 0.889 | |
DeepLabv3+ | 0.934 | 0.933 | 0.934 | 0.933 | 0.066 | 0.875 | |
CloudU-Net | 0.951 | 0.951 | 0.951 | 0.951 | 0.049 | 0.906 | |
CloudRes-UNet | 0.958 | 0.958 | 0.949 | 0.953 | 0.042 | 0.912 | |
夜间 地基 云图 | U-Net | 0.952 | 0.951 | 0.951 | 0.951 | 0.048 | 0.907 |
PSPNet | 0.963 | 0.962 | 0.963 | 0.962 | 0.037 | 0.927 | |
DeepLabv3+ | 0.958 | 0.958 | 0.957 | 0.958 | 0.042 | 0.919 | |
CloudU-Net | 0.973 | 0.972 | 0.973 | 0.972 | 0.027 | 0.947 | |
CloudRes-UNet | 0.986 | 0.983 | 0.981 | 0.982 | 0.014 | 0.965 | |
昼夜 地基 云图 | U-Net | 0.931 | 0.931 | 0.931 | 0.931 | 0.069 | 0.871 |
PSPNet | 0.947 | 0.947 | 0.947 | 0.947 | 0.053 | 0.900 | |
DeepLabv3+ | 0.937 | 0.936 | 0.937 | 0.936 | 0.063 | 0.882 | |
CloudU-Net | 0.951 | 0.954 | 0.954 | 0.954 | 0.049 | 0.908 | |
CloudRes-UNet | 0.966 | 0.964 | 0.958 | 0.961 | 0.034 | 0.922 |
Tab. 2 Comparison of experimental results of different segmentation networks on three datasets
数据集 | 分割网络 | 准确率 | 精确率 | 召回率 | F1值 | ER | MIoU |
---|---|---|---|---|---|---|---|
白天 地基 云图 | U-Net | 0.942 | 0.918 | 0.917 | 0.916 | 0.058 | 0.847 |
PSPNet | 0.941 | 0.941 | 0.941 | 0.941 | 0.059 | 0.889 | |
DeepLabv3+ | 0.934 | 0.933 | 0.934 | 0.933 | 0.066 | 0.875 | |
CloudU-Net | 0.951 | 0.951 | 0.951 | 0.951 | 0.049 | 0.906 | |
CloudRes-UNet | 0.958 | 0.958 | 0.949 | 0.953 | 0.042 | 0.912 | |
夜间 地基 云图 | U-Net | 0.952 | 0.951 | 0.951 | 0.951 | 0.048 | 0.907 |
PSPNet | 0.963 | 0.962 | 0.963 | 0.962 | 0.037 | 0.927 | |
DeepLabv3+ | 0.958 | 0.958 | 0.957 | 0.958 | 0.042 | 0.919 | |
CloudU-Net | 0.973 | 0.972 | 0.973 | 0.972 | 0.027 | 0.947 | |
CloudRes-UNet | 0.986 | 0.983 | 0.981 | 0.982 | 0.014 | 0.965 | |
昼夜 地基 云图 | U-Net | 0.931 | 0.931 | 0.931 | 0.931 | 0.069 | 0.871 |
PSPNet | 0.947 | 0.947 | 0.947 | 0.947 | 0.053 | 0.900 | |
DeepLabv3+ | 0.937 | 0.936 | 0.937 | 0.936 | 0.063 | 0.882 | |
CloudU-Net | 0.951 | 0.954 | 0.954 | 0.954 | 0.049 | 0.908 | |
CloudRes-UNet | 0.966 | 0.964 | 0.958 | 0.961 | 0.034 | 0.922 |
网络模型 | 参数量/MB | 训练时间/h | 测试时间/s |
---|---|---|---|
CloudRes-UNet | 156.8 | 2.8 | 105 |
U-Net | 94.9 | 3.3 | 218 |
PSPNet | 9.3 | 2.1 | 47 |
DeepLabv3+ | 22.4 | 0.9 | 63 |
CloudU-Net | 138.6 | 2.6 | 91 |
Tab. 3 Comparison of parameters, training time and test time among several networks
网络模型 | 参数量/MB | 训练时间/h | 测试时间/s |
---|---|---|---|
CloudRes-UNet | 156.8 | 2.8 | 105 |
U-Net | 94.9 | 3.3 | 218 |
PSPNet | 9.3 | 2.1 | 47 |
DeepLabv3+ | 22.4 | 0.9 | 63 |
CloudU-Net | 138.6 | 2.6 | 91 |
网络 | Multi-Stage | ECA-Net | 准确率 | ER | MIoU |
---|---|---|---|---|---|
网络a | 0.931 | 0.069 | 0.871 | ||
网络b | √ | 0.954 | 0.046 | 0.912 | |
网络c | √ | 0.957 | 0.043 | 0.916 | |
本文网络 | √ | √ | 0.966 | 0.034 | 0.922 |
Tab. 4 Ablation experiment results
网络 | Multi-Stage | ECA-Net | 准确率 | ER | MIoU |
---|---|---|---|---|---|
网络a | 0.931 | 0.069 | 0.871 | ||
网络b | √ | 0.954 | 0.046 | 0.912 | |
网络c | √ | 0.957 | 0.043 | 0.916 | |
本文网络 | √ | √ | 0.966 | 0.034 | 0.922 |
1 | STEPHENS G L. Cloud feedbacks in the climate system: a critical review[J]. Journal of Climate, 2005, 18(2):237-273. 10.1175/jcli-3243.1 |
2 | 刘翼飞, 崔承刚. 基于地基云图云量特征的光伏发电功率区间预测[J]. 南方电网技术, 2023, 17(2):92-100. |
LIU Y F, CUI C G. Interval prediction for short-term solar power based on cloud features of ground-based cloud images[J]. Southern Power System Technology, 2023, 17(2): 92-100. | |
3 | 顾轶,韩潮,刘建勋,等.基于卫星云图的大区域云层预测方法[J].中国空间科学技术, 2023,43(2):165-173. |
GU Y, HAN C, LIU J X, et al. Research on large area cloud forecasting method based on satellite cloud images[J]. Chinese Space Science and Technology, 2023, 43(2):165-173. | |
4 | WANG Y, WANG C, SHI C, et al. Short-term cloud coverage prediction using the ARIMA time series model[J]. Remote Sensing Letters, 2018, 9(3):274-283. 10.1080/2150704x.2017.1418992 |
5 | LONG C N, SABBURG J M, CALBÓ J, et al. Retrieving cloud characteristics from ground-based daytime color all-sky images[J]. Journal Atmospheric Oceanic Technolog, 2006, 23:633-652. 10.1175/jtech1875.1 |
6 | HEINLE A, MACKE A, SRIVASTAV A. Automatic cloud classification of whole sky images[J]. Atmospheric Measurement Techniques, 2010, 3(3):557-567. 10.5194/amt-3-557-2010 |
7 | LI Q, LU W, YANG J. A hybrid thresholding algorithm for cloud detection on ground-based color images[J]. Journal of Atmospheric & Oceanic Technology, 2011, 28:1286-1296. 10.1175/jtech-d-11-00009.1 |
8 | LIU S, ZHANG L, ZHANG Z, et al. Automatic cloud detection for all-sky images using superpixel segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(2):354-358. 10.1109/lgrs.2014.2341291 |
9 | SHI C, WANG Y, WANG C, et al. Ground-based cloud detection using graph model built upon superpixels[J]. IEEE Geoscience & Remote Sensing Letters, 2017, 14(5): 719-723. 10.1109/lgrs.2017.2676007 |
10 | 吉茹, 张银胜, 杨宇龙,等. 基于多尺度特征融合的改进型云图分割方法[J]. 国外电子测量技术, 2022, 41(11):37-44. |
JI R, ZHANG Y S, YANG Y L,et al. Improved cloud image segmentation method based on multi-scale feature fusion[J]. Foreign Electronic Measurement Technology, 2022, 41(11):37-44. | |
11 | 张雪, 贾克斌, 刘钧,等. 面向轻量化的地基云图分割技术研究[J]. 测控技术, 2022, 41(9):37-43. |
ZHANG X, JIA K B, LIU J,et al. Segmentation technology of ground-based cloud image for lightweight[J]. Measurement & Control Technology, 2022, 41(9):37-43. | |
12 | GACAL G F B, ANTIOQUIA C, LAGROSAS N. Ground-based detection of nighttime clouds above Manila Observatory (14.64°N, 121.07°E) using a digital camera[J]. Applied Optics, 2016, 55(22):6040-6045. 10.1364/ao.55.006040 |
13 | DEV S, LEE Y H, WINKLER S. Color-based segmentation of sky/cloud images from ground-based cameras[J]. IEEE Journal Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(1):231-242. 10.1109/jstars.2016.2558474 |
14 | SHI C, ZHOU Y, QIU B, et al. Diurnal and nocturnal cloud segmentation of All-Sky Imager (ASI) images using enhancement fully convolutional networks[J]. Atmospheric Measurement Techniques, 2019, 12(9):4713-4724. 10.5194/amt-12-4713-2019 |
15 | DEV S, NAUTIYAL A, LEE Y H, et al. CloudSegNet: a deep network for nychthemeron cloud image segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(12):1814-1818. 10.1109/lgrs.2019.2912140 |
16 | 张男男, 李丽莎, 王宝珠,等. 基于MobileNet的地基云图分割方法研究[J]. 电子技术与软件工程, 2022, 18:129-132. |
ZHANG N N, LI L S, WANG B Z,et al. Research on MobileNet-based ground-based cloud segmentation method[J]. Electronic Technology & Software Engineering, 2022, 18:129-132. | |
17 | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceeding of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:770-778. 10.1109/cvpr.2016.90 |
18 | WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]// Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2020:11531-11539. 10.1109/cvpr42600.2020.01155 |
19 | SHI C, ZHOU Y, QIU B. CloudU-Netv2: a cloud segmentation method for ground-based cloud images based on deep learning[J]. Neural Processing Letters, 2021, 53:2715-2728. 10.1007/s11063-021-10457-2 |
20 | SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. 10.1109/tpami.2016.2572683 |
21 | RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention,LNCS 9351. Cham: Springer, 2015: 234-241. |
22 | BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. 10.1109/tpami.2016.2644615 |
23 | ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017:6230-6239. 10.1109/cvpr.2017.660 |
24 | CHEN L-C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018:833-851. |
25 | LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[EB/OL]. [2023-05-10]. . 10.1109/iccv48922.2021.00986 |
26 | LIU Z, MAO H, WU C-Y, et al. A ConvNet for the 2020s[EB/OL]. [2023-05-10].. 10.1109/cvpr52688.2022.01167 |
27 | HE K, CHEN X, XIE S, et al. Masked autoencoders are scalable vision learners[EB/OL]. [2023-05-10]. . 10.1109/cvpr52688.2022.01553 |
28 | CARON M, TOUVRON H, MISRA I, et al. Emerging properties in self-supervised vision Transformers[EB/OL]. [2023-05-10]. . 10.1109/iccv48922.2021.00951 |
29 | STEPHAN M, SANTRA A. Radar-based human target detection using deep residual U-Net for smart home applications[C]// Proceedings of the 2019 18th IEEE International Conference on Machine Learning and Applications. Piscataway: IEEE, 2019: 175-182. 10.1109/icmla.2019.00035 |
30 | 冯兴杰, 张天泽. 基于分组卷积进行特征融合的全景分割算法[J]. 计算机应用, 2021, 41(7):2054-2061. 10.11772/j.issn.1001-9081.2020091523 |
FENG X J, ZHANG T Z. Panoptic segmentation algorithm based on grouped convolution for feature fusion[J]. Journal of Computer Applications, 2021, 41(7): 2054-2061. 10.11772/j.issn.1001-9081.2020091523 | |
31 | 谷静, 吴怡宁, 孟鑫昊. 基于膨胀卷积的多尺度焊缝缺陷检测算法[J]. 光电子·激光, 2022, 33(1):61-66. |
GU J, WU Y N, MENG X H. Weld defect detection based on expansion convolution multi-scale fusion[J]. Journal of Optoelectronics·Laser, 2022, 33(1):61-66. | |
32 | YU F, KOLYUN V. Multi-scale context aggregation by dilated convolutions [EB/OL]. [2023-03-23]. . 10.1109/cvpr.2017.75 |
33 | ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices [C]// Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018:6548-6856. 10.1109/cvpr.2018.00716 |
34 | HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023. 10.1109/tpami.2019.2913372 |
35 | SHI C, ZHOU Y, QIU B, et al. CloudU-Net: a deep convolutional neural network architecture for daytime and nighttime cloud images’segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(10):1688-1692. 10.1109/lgrs.2020.3009227 |
[1] | Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877. |
[2] | Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969. |
[3] | Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974. |
[4] | Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918. |
[5] | Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703. |
[6] | Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557. |
[7] | Yanjie GU, Yingjun ZHANG, Xiaoqian LIU, Wei ZHOU, Wei SUN. Traffic flow forecasting via spatial-temporal multi-graph fusion [J]. Journal of Computer Applications, 2024, 44(8): 2618-2625. |
[8] | Qianhong SHI, Yan YANG, Yongquan JIANG, Xiaocao OUYANG, Wubo FAN, Qiang CHEN, Tao JIANG, Yuan LI. Multi-granularity abrupt change fitting network for air quality prediction [J]. Journal of Computer Applications, 2024, 44(8): 2643-2650. |
[9] | Zheng WU, Zhiyou CHENG, Zhentian WANG, Chuanjian WANG, Sheng WANG, Hui XU. Deep learning-based classification of head movement amplitude during patient anaesthesia resuscitation [J]. Journal of Computer Applications, 2024, 44(7): 2258-2263. |
[10] | Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072. |
[11] | Zhi ZHANG, Xin LI, Naifu YE, Kaixi HU. DKP: defending against model stealing attacks based on dark knowledge protection [J]. Journal of Computer Applications, 2024, 44(7): 2080-2086. |
[12] | Yiqun ZHAO, Zhiyu ZHANG, Xue DONG. Anisotropic travel time computation method based on dense residual connection physical information neural networks [J]. Journal of Computer Applications, 2024, 44(7): 2310-2318. |
[13] | Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199. |
[14] | Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215. |
[15] | Yaxing BING, Yangping WANG, Jiu YONG, Haomou BAI. Six degrees of freedom object pose estimation algorithm based on filter learning network [J]. Journal of Computer Applications, 2024, 44(6): 1920-1926. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||