《计算机应用》唯一官方网站 ›› 2024, Vol. 44 ›› Issue (4): 1310-1316.DOI: 10.11772/j.issn.1001-9081.2023040453
• 多媒体计算与计算机仿真 • 上一篇
收稿日期:
2023-04-21
修回日期:
2023-07-05
接受日期:
2023-07-05
发布日期:
2023-12-04
出版日期:
2024-04-10
通讯作者:
李英祥
作者简介:
王铂越(1999—),男,河南洛阳人,硕士研究生,主要研究方向:智能图像处理、人工智能基金资助:
Boyue WANG, Yingxiang LI(), Jiandan ZHONG
Received:
2023-04-21
Revised:
2023-07-05
Accepted:
2023-07-05
Online:
2023-12-04
Published:
2024-04-10
Contact:
Yingxiang LI
About author:
WANG Boyue, born in 1999, M. S. candidate. His research interests include intelligent image processing, artificial intelligence.Supported by:
摘要:
针对昼夜地基云图在分割中细节信息丢失、分割精度低等问题,提出一种基于改进Res-UNet(Residual network-UNetwork)的昼夜地基云图分割网络CloudRes-UNet(Cloud ResNet-UNetwork),整体采用编码器-解码器的网络结构。首先,编码器使用ResNet50提取特征,增强特征提取能力;其次,设计多级特征提取(Multi-Stage)模块,该模块结合分组卷积、膨胀卷积和通道打乱这3种技巧,获取高强度语义信息;再次,加入高效通道注意力(ECA?Net)模块,在通道维度上聚焦重要信息,加强对地基云图中云区域的关注,提高分割精度;最后,解码器使用双线性插值对特征进行上采样,提高分割图像的清晰度并减少目标和位置信息丢失。实验结果表明,与当前基于深度学习表现较好的地基云图分割网络(Cloud-UNet)相比,CloudRes-UNet在昼夜地基云图分割数据集上的分割准确率提升了1.5个百分点,平均交并比(MIoU)上升了1.4个百分点,更准确地获取了云量信息,对天气预报、气候研究和光伏发电等方面具有积极意义。
中图分类号:
王铂越, 李英祥, 钟剑丹. 基于改进Res-UNet的昼夜地基云图分割网络[J]. 计算机应用, 2024, 44(4): 1310-1316.
Boyue WANG, Yingxiang LI, Jiandan ZHONG. Segmentation network for day and night ground-based cloud images based on improved Res-UNet[J]. Journal of Computer Applications, 2024, 44(4): 1310-1316.
卷积层名称 | 输出尺寸 | 层内部结构 |
---|---|---|
Conv1 | 160×160 | 7×7,64,步长=2 |
Conv2_x | 80×80 | 残差块×3 |
Conv3_x | 40×40 | 残差块×4 |
Conv4_x | 20×20 | 残差块×6 |
Conv5_x | 10×10 | 残差块×3 |
表1 ResNet50特征提取网络结构
Tab. 1 ResNet50 feature extraction network structure
卷积层名称 | 输出尺寸 | 层内部结构 |
---|---|---|
Conv1 | 160×160 | 7×7,64,步长=2 |
Conv2_x | 80×80 | 残差块×3 |
Conv3_x | 40×40 | 残差块×4 |
Conv4_x | 20×20 | 残差块×6 |
Conv5_x | 10×10 | 残差块×3 |
数据集 | 分割网络 | 准确率 | 精确率 | 召回率 | F1值 | ER | MIoU |
---|---|---|---|---|---|---|---|
白天 地基 云图 | U-Net | 0.942 | 0.918 | 0.917 | 0.916 | 0.058 | 0.847 |
PSPNet | 0.941 | 0.941 | 0.941 | 0.941 | 0.059 | 0.889 | |
DeepLabv3+ | 0.934 | 0.933 | 0.934 | 0.933 | 0.066 | 0.875 | |
CloudU-Net | 0.951 | 0.951 | 0.951 | 0.951 | 0.049 | 0.906 | |
CloudRes-UNet | 0.958 | 0.958 | 0.949 | 0.953 | 0.042 | 0.912 | |
夜间 地基 云图 | U-Net | 0.952 | 0.951 | 0.951 | 0.951 | 0.048 | 0.907 |
PSPNet | 0.963 | 0.962 | 0.963 | 0.962 | 0.037 | 0.927 | |
DeepLabv3+ | 0.958 | 0.958 | 0.957 | 0.958 | 0.042 | 0.919 | |
CloudU-Net | 0.973 | 0.972 | 0.973 | 0.972 | 0.027 | 0.947 | |
CloudRes-UNet | 0.986 | 0.983 | 0.981 | 0.982 | 0.014 | 0.965 | |
昼夜 地基 云图 | U-Net | 0.931 | 0.931 | 0.931 | 0.931 | 0.069 | 0.871 |
PSPNet | 0.947 | 0.947 | 0.947 | 0.947 | 0.053 | 0.900 | |
DeepLabv3+ | 0.937 | 0.936 | 0.937 | 0.936 | 0.063 | 0.882 | |
CloudU-Net | 0.951 | 0.954 | 0.954 | 0.954 | 0.049 | 0.908 | |
CloudRes-UNet | 0.966 | 0.964 | 0.958 | 0.961 | 0.034 | 0.922 |
表2 不同分割网络在3个数据集上的实验结果对比
Tab. 2 Comparison of experimental results of different segmentation networks on three datasets
数据集 | 分割网络 | 准确率 | 精确率 | 召回率 | F1值 | ER | MIoU |
---|---|---|---|---|---|---|---|
白天 地基 云图 | U-Net | 0.942 | 0.918 | 0.917 | 0.916 | 0.058 | 0.847 |
PSPNet | 0.941 | 0.941 | 0.941 | 0.941 | 0.059 | 0.889 | |
DeepLabv3+ | 0.934 | 0.933 | 0.934 | 0.933 | 0.066 | 0.875 | |
CloudU-Net | 0.951 | 0.951 | 0.951 | 0.951 | 0.049 | 0.906 | |
CloudRes-UNet | 0.958 | 0.958 | 0.949 | 0.953 | 0.042 | 0.912 | |
夜间 地基 云图 | U-Net | 0.952 | 0.951 | 0.951 | 0.951 | 0.048 | 0.907 |
PSPNet | 0.963 | 0.962 | 0.963 | 0.962 | 0.037 | 0.927 | |
DeepLabv3+ | 0.958 | 0.958 | 0.957 | 0.958 | 0.042 | 0.919 | |
CloudU-Net | 0.973 | 0.972 | 0.973 | 0.972 | 0.027 | 0.947 | |
CloudRes-UNet | 0.986 | 0.983 | 0.981 | 0.982 | 0.014 | 0.965 | |
昼夜 地基 云图 | U-Net | 0.931 | 0.931 | 0.931 | 0.931 | 0.069 | 0.871 |
PSPNet | 0.947 | 0.947 | 0.947 | 0.947 | 0.053 | 0.900 | |
DeepLabv3+ | 0.937 | 0.936 | 0.937 | 0.936 | 0.063 | 0.882 | |
CloudU-Net | 0.951 | 0.954 | 0.954 | 0.954 | 0.049 | 0.908 | |
CloudRes-UNet | 0.966 | 0.964 | 0.958 | 0.961 | 0.034 | 0.922 |
网络模型 | 参数量/MB | 训练时间/h | 测试时间/s |
---|---|---|---|
CloudRes-UNet | 156.8 | 2.8 | 105 |
U-Net | 94.9 | 3.3 | 218 |
PSPNet | 9.3 | 2.1 | 47 |
DeepLabv3+ | 22.4 | 0.9 | 63 |
CloudU-Net | 138.6 | 2.6 | 91 |
表3 几种网络的参数量、训练时间和测试时间对比
Tab. 3 Comparison of parameters, training time and test time among several networks
网络模型 | 参数量/MB | 训练时间/h | 测试时间/s |
---|---|---|---|
CloudRes-UNet | 156.8 | 2.8 | 105 |
U-Net | 94.9 | 3.3 | 218 |
PSPNet | 9.3 | 2.1 | 47 |
DeepLabv3+ | 22.4 | 0.9 | 63 |
CloudU-Net | 138.6 | 2.6 | 91 |
网络 | Multi-Stage | ECA-Net | 准确率 | ER | MIoU |
---|---|---|---|---|---|
网络a | 0.931 | 0.069 | 0.871 | ||
网络b | √ | 0.954 | 0.046 | 0.912 | |
网络c | √ | 0.957 | 0.043 | 0.916 | |
本文网络 | √ | √ | 0.966 | 0.034 | 0.922 |
表4 消融实验结果
Tab. 4 Ablation experiment results
网络 | Multi-Stage | ECA-Net | 准确率 | ER | MIoU |
---|---|---|---|---|---|
网络a | 0.931 | 0.069 | 0.871 | ||
网络b | √ | 0.954 | 0.046 | 0.912 | |
网络c | √ | 0.957 | 0.043 | 0.916 | |
本文网络 | √ | √ | 0.966 | 0.034 | 0.922 |
1 | STEPHENS G L. Cloud feedbacks in the climate system: a critical review[J]. Journal of Climate, 2005, 18(2):237-273. 10.1175/jcli-3243.1 |
2 | 刘翼飞, 崔承刚. 基于地基云图云量特征的光伏发电功率区间预测[J]. 南方电网技术, 2023, 17(2):92-100. |
LIU Y F, CUI C G. Interval prediction for short-term solar power based on cloud features of ground-based cloud images[J]. Southern Power System Technology, 2023, 17(2): 92-100. | |
3 | 顾轶,韩潮,刘建勋,等.基于卫星云图的大区域云层预测方法[J].中国空间科学技术, 2023,43(2):165-173. |
GU Y, HAN C, LIU J X, et al. Research on large area cloud forecasting method based on satellite cloud images[J]. Chinese Space Science and Technology, 2023, 43(2):165-173. | |
4 | WANG Y, WANG C, SHI C, et al. Short-term cloud coverage prediction using the ARIMA time series model[J]. Remote Sensing Letters, 2018, 9(3):274-283. 10.1080/2150704x.2017.1418992 |
5 | LONG C N, SABBURG J M, CALBÓ J, et al. Retrieving cloud characteristics from ground-based daytime color all-sky images[J]. Journal Atmospheric Oceanic Technolog, 2006, 23:633-652. 10.1175/jtech1875.1 |
6 | HEINLE A, MACKE A, SRIVASTAV A. Automatic cloud classification of whole sky images[J]. Atmospheric Measurement Techniques, 2010, 3(3):557-567. 10.5194/amt-3-557-2010 |
7 | LI Q, LU W, YANG J. A hybrid thresholding algorithm for cloud detection on ground-based color images[J]. Journal of Atmospheric & Oceanic Technology, 2011, 28:1286-1296. 10.1175/jtech-d-11-00009.1 |
8 | LIU S, ZHANG L, ZHANG Z, et al. Automatic cloud detection for all-sky images using superpixel segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(2):354-358. 10.1109/lgrs.2014.2341291 |
9 | SHI C, WANG Y, WANG C, et al. Ground-based cloud detection using graph model built upon superpixels[J]. IEEE Geoscience & Remote Sensing Letters, 2017, 14(5): 719-723. 10.1109/lgrs.2017.2676007 |
10 | 吉茹, 张银胜, 杨宇龙,等. 基于多尺度特征融合的改进型云图分割方法[J]. 国外电子测量技术, 2022, 41(11):37-44. |
JI R, ZHANG Y S, YANG Y L,et al. Improved cloud image segmentation method based on multi-scale feature fusion[J]. Foreign Electronic Measurement Technology, 2022, 41(11):37-44. | |
11 | 张雪, 贾克斌, 刘钧,等. 面向轻量化的地基云图分割技术研究[J]. 测控技术, 2022, 41(9):37-43. |
ZHANG X, JIA K B, LIU J,et al. Segmentation technology of ground-based cloud image for lightweight[J]. Measurement & Control Technology, 2022, 41(9):37-43. | |
12 | GACAL G F B, ANTIOQUIA C, LAGROSAS N. Ground-based detection of nighttime clouds above Manila Observatory (14.64°N, 121.07°E) using a digital camera[J]. Applied Optics, 2016, 55(22):6040-6045. 10.1364/ao.55.006040 |
13 | DEV S, LEE Y H, WINKLER S. Color-based segmentation of sky/cloud images from ground-based cameras[J]. IEEE Journal Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(1):231-242. 10.1109/jstars.2016.2558474 |
14 | SHI C, ZHOU Y, QIU B, et al. Diurnal and nocturnal cloud segmentation of All-Sky Imager (ASI) images using enhancement fully convolutional networks[J]. Atmospheric Measurement Techniques, 2019, 12(9):4713-4724. 10.5194/amt-12-4713-2019 |
15 | DEV S, NAUTIYAL A, LEE Y H, et al. CloudSegNet: a deep network for nychthemeron cloud image segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(12):1814-1818. 10.1109/lgrs.2019.2912140 |
16 | 张男男, 李丽莎, 王宝珠,等. 基于MobileNet的地基云图分割方法研究[J]. 电子技术与软件工程, 2022, 18:129-132. |
ZHANG N N, LI L S, WANG B Z,et al. Research on MobileNet-based ground-based cloud segmentation method[J]. Electronic Technology & Software Engineering, 2022, 18:129-132. | |
17 | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceeding of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016:770-778. 10.1109/cvpr.2016.90 |
18 | WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]// Proceedings of the 2020 IEEE/CVF International Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2020:11531-11539. 10.1109/cvpr42600.2020.01155 |
19 | SHI C, ZHOU Y, QIU B. CloudU-Netv2: a cloud segmentation method for ground-based cloud images based on deep learning[J]. Neural Processing Letters, 2021, 53:2715-2728. 10.1007/s11063-021-10457-2 |
20 | SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651. 10.1109/tpami.2016.2572683 |
21 | RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention,LNCS 9351. Cham: Springer, 2015: 234-241. |
22 | BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. 10.1109/tpami.2016.2644615 |
23 | ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017:6230-6239. 10.1109/cvpr.2017.660 |
24 | CHEN L-C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11211. Cham: Springer, 2018:833-851. |
25 | LIU Z, LIN Y, CAO Y, et al. Swin Transformer: hierarchical vision Transformer using shifted windows[EB/OL]. [2023-05-10]. . 10.1109/iccv48922.2021.00986 |
26 | LIU Z, MAO H, WU C-Y, et al. A ConvNet for the 2020s[EB/OL]. [2023-05-10].. 10.1109/cvpr52688.2022.01167 |
27 | HE K, CHEN X, XIE S, et al. Masked autoencoders are scalable vision learners[EB/OL]. [2023-05-10]. . 10.1109/cvpr52688.2022.01553 |
28 | CARON M, TOUVRON H, MISRA I, et al. Emerging properties in self-supervised vision Transformers[EB/OL]. [2023-05-10]. . 10.1109/iccv48922.2021.00951 |
29 | STEPHAN M, SANTRA A. Radar-based human target detection using deep residual U-Net for smart home applications[C]// Proceedings of the 2019 18th IEEE International Conference on Machine Learning and Applications. Piscataway: IEEE, 2019: 175-182. 10.1109/icmla.2019.00035 |
30 | 冯兴杰, 张天泽. 基于分组卷积进行特征融合的全景分割算法[J]. 计算机应用, 2021, 41(7):2054-2061. 10.11772/j.issn.1001-9081.2020091523 |
FENG X J, ZHANG T Z. Panoptic segmentation algorithm based on grouped convolution for feature fusion[J]. Journal of Computer Applications, 2021, 41(7): 2054-2061. 10.11772/j.issn.1001-9081.2020091523 | |
31 | 谷静, 吴怡宁, 孟鑫昊. 基于膨胀卷积的多尺度焊缝缺陷检测算法[J]. 光电子·激光, 2022, 33(1):61-66. |
GU J, WU Y N, MENG X H. Weld defect detection based on expansion convolution multi-scale fusion[J]. Journal of Optoelectronics·Laser, 2022, 33(1):61-66. | |
32 | YU F, KOLYUN V. Multi-scale context aggregation by dilated convolutions [EB/OL]. [2023-03-23]. . 10.1109/cvpr.2017.75 |
33 | ZHANG X, ZHOU X, LIN M, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices [C]// Proceeding of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018:6548-6856. 10.1109/cvpr.2018.00716 |
34 | HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023. 10.1109/tpami.2019.2913372 |
35 | SHI C, ZHOU Y, QIU B, et al. CloudU-Net: a deep convolutional neural network architecture for daytime and nighttime cloud images’segmentation[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(10):1688-1692. 10.1109/lgrs.2020.3009227 |
[1] | 李威, 陈玲, 徐修远, 朱敏, 郭际香, 周凯, 牛颢, 张煜宸, 易珊烨, 章毅, 罗凤鸣. 基于多任务学习的间质性肺病分割算法[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1285-1293. |
[2] | 万泽轩, 谢春丽, 吕泉润, 梁瑶. 基于依赖增强的分层抽象语法树的代码克隆检测[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1259-1268. |
[3] | 唐睿, 岳士博, 张睿智, 刘川, 庞川林. UAV协助下非正交多址接入使能的数据采集系统中能效优化机制[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1209-1218. |
[4] | 孙祥杰, 魏强, 王奕森, 杜江. 代码相似性检测技术综述[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1248-1258. |
[5] | 张鹏飞, 韩李涛, 冯恒健, 李洪梅. 基于注意力机制和全局特征优化的点云语义分割[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1086-1092. |
[6] | 杨先凤, 汤依磊, 李自强. 基于交替注意力机制和图卷积网络的方面级情感分析模型[J]. 《计算机应用》唯一官方网站, 2024, 44(4): 1058-1064. |
[7] | 徐大鹏, 侯新民. 基于网络结构设计的图神经网络特征选择方法[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 663-670. |
[8] | 唐瑶瑶, 朱叶晨, 刘仰川, 高欣. CT图像环形伪影去除方法研究现状及展望[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 890-900. |
[9] | 蔡美玉, 朱润哲, 吴飞, 张开昱, 李家乐. 基于注意力机制和多粒度特征融合的跨视角匹配模型[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 901-908. |
[10] | 李雨秋, 侯利萍, 薛健, 吕科, 王泳. 基于内容解译的遥感图像推荐方法[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 722-731. |
[11] | 董炜娜, 刘佳, 潘晓中, 陈立峰, 孙文权. 基于编码-解码网络的大容量鲁棒图像隐写方案[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 772-779. |
[12] | 赵奎, 仇慧琪, 李旭, 徐知非. 结合注意力和多路径融合的实时肺结节检测算法[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 945-952. |
[13] | 吴宁, 罗杨洋, 许华杰. 基于多尺度特征融合的遥感图像语义分割方法[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 737-744. |
[14] | 刘祥, 华蓓, 林飞, 魏宏原. 面向深度学习应用的组件式开发框架的设计实现[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 526-535. |
[15] | 宋钰丹, 王晶, 王雪徽, 马朝阳, 林友芳. 基于自适应多任务学习的睡眠生理时序分类方法[J]. 《计算机应用》唯一官方网站, 2024, 44(2): 654-662. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||