Waterweed image segmentation method based on improved U-Net

doi:10.11772/j.issn.1001-9081.2021091614

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (10): 3177-3183.DOI: 10.11772/j.issn.1001-9081.2021091614

• Multimedia computing and computer simulation • Previous Articles

Waterweed image segmentation method based on improved U-Net

Qiwen WU, Jianhua WANG, Xiang ZHENG, Ju FENG, Hongyan JIANG, Yubo WANG

Key Laboratory of Marine Technology and Control Engineering，Ministry of Transport （Shanghai Maritime University），Shanghai 201306，China

Received:2021-09-13 Revised:2021-12-24 Accepted:2022-01-04 Online:2022-04-15 Published:2022-10-10
Contact: Jianhua WANG
About author:WU Qiwen, born in 1997， M. S. candidate. Her research interests include machine vision.
WANG Jianhua, born in 1965， Ph. D. ， associate professor. His research interests include surface robot， machine vision.
ZHENG Xiang, born in 1981， Ph. D. ， lecturer. Her research interests include motion control of unmanned surface vehicles， vibration control of smart structures.
FENG Ju, born in 1997， M. S. candidate. Her research interests include unmanned surface vehicle perception decision-making and control.
JIANG Hongyan, born in 1997， M. S. candidate. His research interests include ship and port automation technology.
WANG Yubo, born in 1997， M. S. candidate. His research interests include unmanned surface vehicle visual navigation，autonomous berthing.
Supported by:
National Natural Science Foundation of China(62176150)

基于改进U-Net的水草图像分割方法

吴奇文, 王建华, 郑翔, 冯居, 姜洪岩, 王昱博

航运技术与控制工程交通运输行业重点实验室（上海海事大学），上海 201306

通讯作者: 王建华
作者简介:第一联系人：吴奇文（1997—），女，江西上饶人，硕士研究生，主要研究方向：机器视觉
王建华（1965—），男，云南鹤庆人，副教授，博士，主要研究方向：水面机器人、机器视觉; jhwang@shmtu.edu.cn
郑翔（1981—），女，江苏扬州人，讲师，博士，主要研究方向：无人艇运动控制、智能结构减振控制
冯居（1997—），女，江苏盐城人，硕士研究生，主要研究方向：无人艇感知决策与控制
姜洪岩（1997—），男，吉林德惠人，硕士研究生，主要研究方向：船舶与港口自动化技术
王昱博（1997—），男，河南洛阳人，硕士研究生，主要研究方向：无人艇视觉导航、自主靠泊。
基金资助:
国家自然科学基金资助项目(62176150)

Abstract

Abstract:

During the operation of the Unmanned Surface Vehicles （USVs）， the propellers are easily gotten entangled by waterweeds， which is a problem encountered by the whole industry. Concerning the global distribution， dispersivity， and complexity of the edge and texture of waterweeds in the water surface images， the U-Net was improved and used to classify all pixels in the image， in order to reduce the feature loss of the network， and enhance the extraction of both global and local features， thereby improving the overall segmentation performance. Firstly， the image data of waterweeds in multiple locations and multiple periods were collected， and a comprehensive dataset of waterweeds for semantic segmentation was built. Secondly， three scales of input images were introduced into the network to enable full extraction of the features via the network， and three loss functions for the upsampled images were introduced to balance the overall loss brought by the three different scales of input images. In addition， a hybrid attention module， including the dilated convolution branch and the channel attention enhancement branch， was proposed and introduced to the network. Finally， the proposed network was verified on the newly built waterweed dataset. Experimental results show that the accuracy， mean Intersection over Union （mIoU） and mean Pixel Accuracy （mPA） values of the proposed method can reach 96.8%， 91.22% and 95.29%， respectively， which are improved by 4.62 percentage points， 3.87 percentage points and 3.12 percentage points compared with those of U-Net （VGG16） segmentation method. The proposed method can be applied to unmanned surface vehicles for detection of waterweeds， and perform the corresponding path planning to realize waterweed avoidance.

Key words: waterweed, U-Net, semantic segmentation, attention mechanism, multi-scale input, loss function

摘要：

无人艇（USV）在河道水面作业过程中，水草会缠绕推进器，这是整个业界应用都遇到的困扰。针对水面图像中水草分布的全局性、分散性以及边缘和纹理的复杂性，对U-Net进行改进并用于对图像所有的像素进行分类，以减少网络特征信息的丢失，并加强全局和局部特征的提取，从而提高分割性能。首先，采集多地多时段水草图像数据，制作了一个比较全面的水草语义分割数据集；其次，提出在U-Net中引入三个尺度的图像输入，从而使得网络对特征进行充分提取，并引进三种上采样图像的损失函数来平衡三种尺度的输入图像带来的总体损失；此外，还提出了一种混合注意力模块并引入到网络中，其包含空洞卷积和通道注意增强两个分支；最后，在新构建的水草数据集上对所提网络进行验证。实验结果显示，所提方法的准确率、均交并比（mIoU）和平均像素精度（mPA）值分别可达96.8%、91.22%和95.29%，与U-Net（VGG16）分割方法相比，分别提高了4.62个百分点、3.87个百分点和3.12个百分点。所提方法可应用于水面无人艇对水草的检测，并进行相应的路径规划来实现水草避让。

关键词: 水草, U-Net, 语义分割, 注意力机制, 多尺度输入, 损失函数

CLC Number:

TP391.4

Qiwen WU, Jianhua WANG, Xiang ZHENG, Ju FENG, Hongyan JIANG, Yubo WANG. Waterweed image segmentation method based on improved U-Net[J]. Journal of Computer Applications, 2022, 42(10): 3177-3183.

吴奇文, 王建华, 郑翔, 冯居, 姜洪岩, 王昱博. 基于改进U-Net的水草图像分割方法[J]. 《计算机应用》唯一官方网站, 2022, 42(10): 3177-3183.

Figures/Tables 10

References 20

1	罗艾娜，郑骏. 基于深度神经网络的复杂光照下的蓝藻图片语义分割［J］. 计算机应用与软件， 2018， 35（4）：254-259. 10.3969/j.issn.1000-386x.2018.04.047
	LUO A N， ZHENG J. Semantic segmentation of cyanobacteria pictures based on DCNN under complicated illumination［J］. Computer Applications and Software， 2018， 35（4）：254-259. 10.3969/j.issn.1000-386x.2018.04.047
2	杨朔，陈丽芳，石瑀，等. 基于深度生成式对抗网络的蓝藻语义分割［J］. 计算机应用， 2018， 38（6）：1554-1561. 10.11772/j.issn.1001-9081.2017122872
	YANG S， CHEN L F， SHI Y， et al. Semantic segmentation of blue-green algae based on deep generative adversarial net［J］. Journal of Computer Applications， 2018， 38（6）：1554-1561. 10.11772/j.issn.1001-9081.2017122872
3	李国进，姚冬宜，艾矫燕，等. 基于改进Faster R-CNN的水面漂浮物识别与定位［J］. 信阳师范学院学报（自然科学版）， 2021， 34（2）：292-299.
	LI G J， YAO D Y， AI J Y， et al. Detection and localization of floating objects via improved Faster R-CNN［J］. Journal of Xinyang Normal University （Natural Science Edition）， 2021， 34（2）：292-299.
4	ZHAO R Y， WANG J H， ZHENG X， et al. Maritime visible image classification based on double transfer method［J］. IEEE Access， 2020， 8： 166335-166346. 10.1109/access.2020.3022883
5	邹宜洋. 基于卷积神经网络的船舶识别跟踪技术研究［D］. 大连：大连海事大学， 2020：68-88.
	ZOU Y J. Research on ship identification and tracking technology based on convolutional neural network［D］. Dalian： Dalian Maritime University， 2020： 68-88.
6	涂昊，刁云峰，程文明，等. 基于深度学习的实时集装箱箱号识别算法［J］. 起重运输机械， 2021（13）：58-63. 10.3969/j.issn.1001-0785.2021.13.018
	TU H， DIAO Y F， CHENG W M， et al. Real-time container number recognition algorithm based on deep learning［J］. Hoisting and Conveying Machinery， 2021（13）：58-63. 10.3969/j.issn.1001-0785.2021.13.018
7	LONG J， SHELHAMER E， DARRELL T. Fully convolutional networks for semantic segmentation［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 3431-3440. 10.1109/cvpr.2015.7298965
8	RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
9	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
10	WOO S， PARK J， LEE J Y， et al. CBAM： convolutional block attention module［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer， 2018： 3-19.
11	LI X， WANG W H， HU X L， et al. Selective kernel networks［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 510-519. 10.1109/cvpr.2019.00060
12	YU F， KOLTUN V. Multi-scale context aggregation by dilated convolutions ［EB/OL］. （2016-04-30）［2021-08-11］. .
13	CHEN L C， PAPANDREOU G， SCHROFF F， et al. Rethinking atrous convolution for semantic image segmentation ［EB/OL］. （2017-12-05）［2021-08-15］.. 10.1007/978-3-030-01234-2_49
14	WANG P Q， CHEN P F， YUAN Y， et al. Understanding convolution for semantic segmentation［C］// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2018： 1451-1460. 10.1109/wacv.2018.00163
15	TAO A， SPARA K， CATANZARO B. Hierarchical multi-scale attention for semantic segmentation［J］. （2020-05-21）［2021-08-15］.. 10.48550/arXiv.2005.10821
16	HADSELL R， CHOPRA S， LeCUN Y. Dimensionality reduction by learning an invariant mapping［C］// Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2006： 1735-1742. 10.1109/cvpr.2006.9
17	WU C Y， MANMATHA R， SMOLA A J， et al. Sampling matters in deep embedding learning［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2859-2867. 10.1109/iccv.2017.309
18	MILLETARI F， NAVAB N， AHMADI S A. V-Net： fully convolutional neural networks for volumetric medical image segmentation［C］// Proceedings of the 4th International Conference on 3D Vision. Piscataway： IEEE， 2016： 565-571. 10.1109/3dv.2016.79
19	KLINE D M， BERARDI V L. Revisiting squared-error and cross-entropy functions for training neural network classifiers［J］. Neural Computing and Applications， 2005， 14（4）： 310-318. 10.1007/s00521-005-0467-y
20	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： a deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495. 10.1109/tpami.2016.2644615

实际标签	预测标签
实际标签	是水草	不是水草
是水草	真正TP（True Positive）	假正FP（False Positive）
不是水草	假负FN（False Negative）	真负TN（True Negative）

实际标签	预测标签
实际标签	是水草	不是水草
是水草	真正TP（True Positive）	假正FP（False Positive）
不是水草	假负FN（False Negative）	真负TN（True Negative）

方法	准确率	mIoU	mPA
FCN（VGG16）^［7］	89.22	75.98	86.90
SegNet^［20］	90.78	79.80	88.12
U-Net（VGG16）^［8］	92.18	87.35	92.17
U-Net（ResNet18）^［8］	93.52	89.42	94.27
DeepLabv3^［13］	95.93	89.96	94.86
本文方法	96.80	91.22	95.29

方法	准确率	mIoU	mPA
FCN（VGG16）^［7］	89.22	75.98	86.90
SegNet^［20］	90.78	79.80	88.12
U-Net（VGG16）^［8］	92.18	87.35	92.17
U-Net（ResNet18）^［8］	93.52	89.42	94.27
DeepLabv3^［13］	95.93	89.96	94.86
本文方法	96.80	91.22	95.29

方法	准确率	mIoU	mPA
U-Net（ResNet18）	93.52	89.42	94.27
+混合注意力模块	95.89	90.05	94.71
+多尺度图像输入	95.79	89.45	94.29
+多尺度图像输入+混合损失函数	96.13	90.66	94.67
本文方法	96.80	91.22	95.29

Waterweed image segmentation method based on improved U-Net

基于改进U-Net的水草图像分割方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 20

Related Articles 15

Recommended Articles

Metrics

[1]	Haiyun WEI, Qianying ZHENG, Jinling YU. Motion blurred image restoration algorithm based on multi-scale network [J]. Journal of Computer Applications, 2022, 42(9): 2838-2844.
[2]	Wentao ZHANG, Yuanyu WANG, Saize LI. Depth estimation model of single haze image based on conditional generative adversarial network [J]. Journal of Computer Applications, 2022, 42(9): 2865-2875.
[3]	Yaoshun LI, Lizhi LIU. Lightweight network for rebar detection with attention mechanism [J]. Journal of Computer Applications, 2022, 42(9): 2900-2908.
[4]	Hongjun HENG, Tianbao XU. Attention sentiment analysis model based on multi-scale convolution and gating mechanism [J]. Journal of Computer Applications, 2022, 42(9): 2674-2679.
[5]	Yuefeng LIU, Xiaoyan ZHANG, Wei GUO, Haodong BIAN, Yingjie HE. Remaining useful life prediction method of aero-engine based on optimized hybrid model [J]. Journal of Computer Applications, 2022, 42(9): 2960-2968.
[6]	Kai WEN, Weiwei TANG, Junchen XIONG. Real-time segmentation algorithm based on attention mechanism and effective factorized convolution [J]. Journal of Computer Applications, 2022, 42(9): 2659-2666.
[7]	Xudong HOU, Fei TENG, Yi ZHANG. Medical named entity recognition model based on deep auto-encoding [J]. Journal of Computer Applications, 2022, 42(9): 2686-2692.
[8]	Kun LI, Qing HOU. Lightweight human pose estimation based on attention mechanism [J]. Journal of Computer Applications, 2022, 42(8): 2407-2414.
[9]	Huazhong JIN, Xiuyang ZHANG, Zhiwei YE, Wenqi ZHANG, Xiaoyu XIA. Image denoising model based on approximate U-shaped network structure [J]. Journal of Computer Applications, 2022, 42(8): 2571-2577.
[10]	Liying ZHANG, Chunjiang PANG, Xinying WANG, Guoliang LI. Multi-scale object detection algorithm based on improved YOLOv3 [J]. Journal of Computer Applications, 2022, 42(8): 2423-2431.
[11]	Xinyu ZHANG, Sheng DING, Zhipei YANG. Traffic sign detection algorithm based on improved attention mechanism [J]. Journal of Computer Applications, 2022, 42(8): 2378-2385.
[12]	Yinglü XUAN, Yuan WAN, Jiahui CHEN. Time series classification by LSTM based on multi-scale convolution and attention mechanism [J]. Journal of Computer Applications, 2022, 42(8): 2343-2352.
[13]	Minghui WU, Guangjie ZHANG, Canghong JIN. Time series prediction model based on multimodal information fusion [J]. Journal of Computer Applications, 2022, 42(8): 2326-2332.
[14]	Zhenhu LYU, Xinzheng XU, Fangyan ZHANG. Lightweight attention mechanism module based on squeeze and excitation [J]. Journal of Computer Applications, 2022, 42(8): 2353-2360.
[15]	Chengxia XU, Qing YAN, Teng LI, Kaichao MIAO. De-raining algorithm based on joint attention mechanism for single image [J]. Journal of Computer Applications, 2022, 42(8): 2578-2585.