Waterweed image segmentation method based on improved U-Net

doi:10.11772/j.issn.1001-9081.2021091614

Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (10): 3177-3183.DOI: 10.11772/j.issn.1001-9081.2021091614

• Multimedia computing and computer simulation • Previous Articles Next Articles

Waterweed image segmentation method based on improved U-Net

Qiwen WU, Jianhua WANG, Xiang ZHENG, Ju FENG, Hongyan JIANG, Yubo WANG

Key Laboratory of Marine Technology and Control Engineering，Ministry of Transport （Shanghai Maritime University），Shanghai 201306，China

Received:2021-09-13 Revised:2021-12-24 Accepted:2022-01-04 Online:2022-04-15 Published:2022-10-10
Contact: Jianhua WANG
About author:WU Qiwen, born in 1997， M. S. candidate. Her research interests include machine vision.
WANG Jianhua, born in 1965， Ph. D. ， associate professor. His research interests include surface robot， machine vision.
ZHENG Xiang, born in 1981， Ph. D. ， lecturer. Her research interests include motion control of unmanned surface vehicles， vibration control of smart structures.
FENG Ju, born in 1997， M. S. candidate. Her research interests include unmanned surface vehicle perception decision-making and control.
JIANG Hongyan, born in 1997， M. S. candidate. His research interests include ship and port automation technology.
WANG Yubo, born in 1997， M. S. candidate. His research interests include unmanned surface vehicle visual navigation，autonomous berthing.
Supported by:
National Natural Science Foundation of China(62176150)

基于改进U-Net的水草图像分割方法

吴奇文, 王建华, 郑翔, 冯居, 姜洪岩, 王昱博

航运技术与控制工程交通运输行业重点实验室（上海海事大学），上海 201306

通讯作者: 王建华
作者简介:第一联系人：吴奇文（1997—），女，江西上饶人，硕士研究生，主要研究方向：机器视觉
王建华（1965—），男，云南鹤庆人，副教授，博士，主要研究方向：水面机器人、机器视觉; jhwang@shmtu.edu.cn
郑翔（1981—），女，江苏扬州人，讲师，博士，主要研究方向：无人艇运动控制、智能结构减振控制
冯居（1997—），女，江苏盐城人，硕士研究生，主要研究方向：无人艇感知决策与控制
姜洪岩（1997—），男，吉林德惠人，硕士研究生，主要研究方向：船舶与港口自动化技术
王昱博（1997—），男，河南洛阳人，硕士研究生，主要研究方向：无人艇视觉导航、自主靠泊。
基金资助:
国家自然科学基金资助项目(62176150)

Abstract

Abstract:

During the operation of the Unmanned Surface Vehicles （USVs）， the propellers are easily gotten entangled by waterweeds， which is a problem encountered by the whole industry. Concerning the global distribution， dispersivity， and complexity of the edge and texture of waterweeds in the water surface images， the U-Net was improved and used to classify all pixels in the image， in order to reduce the feature loss of the network， and enhance the extraction of both global and local features， thereby improving the overall segmentation performance. Firstly， the image data of waterweeds in multiple locations and multiple periods were collected， and a comprehensive dataset of waterweeds for semantic segmentation was built. Secondly， three scales of input images were introduced into the network to enable full extraction of the features via the network， and three loss functions for the upsampled images were introduced to balance the overall loss brought by the three different scales of input images. In addition， a hybrid attention module， including the dilated convolution branch and the channel attention enhancement branch， was proposed and introduced to the network. Finally， the proposed network was verified on the newly built waterweed dataset. Experimental results show that the accuracy， mean Intersection over Union （mIoU） and mean Pixel Accuracy （mPA） values of the proposed method can reach 96.8%， 91.22% and 95.29%， respectively， which are improved by 4.62 percentage points， 3.87 percentage points and 3.12 percentage points compared with those of U-Net （VGG16） segmentation method. The proposed method can be applied to unmanned surface vehicles for detection of waterweeds， and perform the corresponding path planning to realize waterweed avoidance.

Key words: waterweed, U-Net, semantic segmentation, attention mechanism, multi-scale input, loss function

摘要：

无人艇（USV）在河道水面作业过程中，水草会缠绕推进器，这是整个业界应用都遇到的困扰。针对水面图像中水草分布的全局性、分散性以及边缘和纹理的复杂性，对U-Net进行改进并用于对图像所有的像素进行分类，以减少网络特征信息的丢失，并加强全局和局部特征的提取，从而提高分割性能。首先，采集多地多时段水草图像数据，制作了一个比较全面的水草语义分割数据集；其次，提出在U-Net中引入三个尺度的图像输入，从而使得网络对特征进行充分提取，并引进三种上采样图像的损失函数来平衡三种尺度的输入图像带来的总体损失；此外，还提出了一种混合注意力模块并引入到网络中，其包含空洞卷积和通道注意增强两个分支；最后，在新构建的水草数据集上对所提网络进行验证。实验结果显示，所提方法的准确率、均交并比（mIoU）和平均像素精度（mPA）值分别可达96.8%、91.22%和95.29%，与U-Net（VGG16）分割方法相比，分别提高了4.62个百分点、3.87个百分点和3.12个百分点。所提方法可应用于水面无人艇对水草的检测，并进行相应的路径规划来实现水草避让。

关键词: 水草, U-Net, 语义分割, 注意力机制, 多尺度输入, 损失函数

CLC Number:

TP391.4

Qiwen WU, Jianhua WANG, Xiang ZHENG, Ju FENG, Hongyan JIANG, Yubo WANG. Waterweed image segmentation method based on improved U-Net[J]. Journal of Computer Applications, 2022, 42(10): 3177-3183.

吴奇文, 王建华, 郑翔, 冯居, 姜洪岩, 王昱博. 基于改进U-Net的水草图像分割方法[J]. 《计算机应用》唯一官方网站, 2022, 42(10): 3177-3183.

Figures/Tables 10

References 20

1	罗艾娜，郑骏. 基于深度神经网络的复杂光照下的蓝藻图片语义分割［J］. 计算机应用与软件， 2018， 35（4）：254-259. 10.3969/j.issn.1000-386x.2018.04.047
	LUO A N， ZHENG J. Semantic segmentation of cyanobacteria pictures based on DCNN under complicated illumination［J］. Computer Applications and Software， 2018， 35（4）：254-259. 10.3969/j.issn.1000-386x.2018.04.047
2	杨朔，陈丽芳，石瑀，等. 基于深度生成式对抗网络的蓝藻语义分割［J］. 计算机应用， 2018， 38（6）：1554-1561. 10.11772/j.issn.1001-9081.2017122872
	YANG S， CHEN L F， SHI Y， et al. Semantic segmentation of blue-green algae based on deep generative adversarial net［J］. Journal of Computer Applications， 2018， 38（6）：1554-1561. 10.11772/j.issn.1001-9081.2017122872
3	李国进，姚冬宜，艾矫燕，等. 基于改进Faster R-CNN的水面漂浮物识别与定位［J］. 信阳师范学院学报（自然科学版）， 2021， 34（2）：292-299.
	LI G J， YAO D Y， AI J Y， et al. Detection and localization of floating objects via improved Faster R-CNN［J］. Journal of Xinyang Normal University （Natural Science Edition）， 2021， 34（2）：292-299.
4	ZHAO R Y， WANG J H， ZHENG X， et al. Maritime visible image classification based on double transfer method［J］. IEEE Access， 2020， 8： 166335-166346. 10.1109/access.2020.3022883
5	邹宜洋. 基于卷积神经网络的船舶识别跟踪技术研究［D］. 大连：大连海事大学， 2020：68-88.
	ZOU Y J. Research on ship identification and tracking technology based on convolutional neural network［D］. Dalian： Dalian Maritime University， 2020： 68-88.
6	涂昊，刁云峰，程文明，等. 基于深度学习的实时集装箱箱号识别算法［J］. 起重运输机械， 2021（13）：58-63. 10.3969/j.issn.1001-0785.2021.13.018
	TU H， DIAO Y F， CHENG W M， et al. Real-time container number recognition algorithm based on deep learning［J］. Hoisting and Conveying Machinery， 2021（13）：58-63. 10.3969/j.issn.1001-0785.2021.13.018
7	LONG J， SHELHAMER E， DARRELL T. Fully convolutional networks for semantic segmentation［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 3431-3440. 10.1109/cvpr.2015.7298965
8	RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
9	HU J， SHEN L， SUN G. Squeeze-and-excitation networks［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 7132-7141. 10.1109/cvpr.2018.00745
10	WOO S， PARK J， LEE J Y， et al. CBAM： convolutional block attention module［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11211. Cham： Springer， 2018： 3-19.
11	LI X， WANG W H， HU X L， et al. Selective kernel networks［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 510-519. 10.1109/cvpr.2019.00060
12	YU F， KOLTUN V. Multi-scale context aggregation by dilated convolutions ［EB/OL］. （2016-04-30）［2021-08-11］. .
13	CHEN L C， PAPANDREOU G， SCHROFF F， et al. Rethinking atrous convolution for semantic image segmentation ［EB/OL］. （2017-12-05）［2021-08-15］.. 10.1007/978-3-030-01234-2_49
14	WANG P Q， CHEN P F， YUAN Y， et al. Understanding convolution for semantic segmentation［C］// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2018： 1451-1460. 10.1109/wacv.2018.00163
15	TAO A， SPARA K， CATANZARO B. Hierarchical multi-scale attention for semantic segmentation［J］. （2020-05-21）［2021-08-15］.. 10.48550/arXiv.2005.10821
16	HADSELL R， CHOPRA S， LeCUN Y. Dimensionality reduction by learning an invariant mapping［C］// Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2006： 1735-1742. 10.1109/cvpr.2006.9
17	WU C Y， MANMATHA R， SMOLA A J， et al. Sampling matters in deep embedding learning［C］// Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2017： 2859-2867. 10.1109/iccv.2017.309
18	MILLETARI F， NAVAB N， AHMADI S A. V-Net： fully convolutional neural networks for volumetric medical image segmentation［C］// Proceedings of the 4th International Conference on 3D Vision. Piscataway： IEEE， 2016： 565-571. 10.1109/3dv.2016.79
19	KLINE D M， BERARDI V L. Revisiting squared-error and cross-entropy functions for training neural network classifiers［J］. Neural Computing and Applications， 2005， 14（4）： 310-318. 10.1007/s00521-005-0467-y
20	BADRINARAYANAN V， KENDALL A， CIPOLLA R. SegNet： a deep convolutional encoder-decoder architecture for image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（12）： 2481-2495. 10.1109/tpami.2016.2644615

实际标签	预测标签
实际标签	是水草	不是水草
是水草	真正TP（True Positive）	假正FP（False Positive）
不是水草	假负FN（False Negative）	真负TN（True Negative）

实际标签	预测标签
实际标签	是水草	不是水草
是水草	真正TP（True Positive）	假正FP（False Positive）
不是水草	假负FN（False Negative）	真负TN（True Negative）

方法	准确率	mIoU	mPA
FCN（VGG16）^［7］	89.22	75.98	86.90
SegNet^［20］	90.78	79.80	88.12
U-Net（VGG16）^［8］	92.18	87.35	92.17
U-Net（ResNet18）^［8］	93.52	89.42	94.27
DeepLabv3^［13］	95.93	89.96	94.86
本文方法	96.80	91.22	95.29

方法	准确率	mIoU	mPA
FCN（VGG16）^［7］	89.22	75.98	86.90
SegNet^［20］	90.78	79.80	88.12
U-Net（VGG16）^［8］	92.18	87.35	92.17
U-Net（ResNet18）^［8］	93.52	89.42	94.27
DeepLabv3^［13］	95.93	89.96	94.86
本文方法	96.80	91.22	95.29

方法	准确率	mIoU	mPA
U-Net（ResNet18）	93.52	89.42	94.27
+混合注意力模块	95.89	90.05	94.71
+多尺度图像输入	95.79	89.45	94.29
+多尺度图像输入+混合损失函数	96.13	90.66	94.67
本文方法	96.80	91.22	95.29

Waterweed image segmentation method based on improved U-Net

基于改进U-Net的水草图像分割方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 20

Related Articles 15

Recommended Articles

Metrics

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[3]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[4]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[5]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[6]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[7]	Kaili DENG, Weibo WEI, Zhenkuan PAN. Industrial defect detection method with improved masked autoencoder [J]. Journal of Computer Applications, 2024, 44(8): 2595-2603.
[8]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[9]	Zhe KONG, Han LI, Shaowei GAN, Mingru KONG, Bingtao HE, Ziyu GUO, Ducheng JIN, Zhaowen QIU. Structure segmentation model for 3D kidney images based on asymmetric multi-decoder and attention module [J]. Journal of Computer Applications, 2024, 44(7): 2216-2224.
[10]	Minghao SUN, Han YU, Yuqing CHEN, Kai LU. First-arrival picking and inversion of seismic waveforms based on U-shaped multilayer perceptron network [J]. Journal of Computer Applications, 2024, 44(7): 2301-2309.
[11]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[12]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[13]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[14]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[15]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.