Multi-scale ship detection based on adaptive feature fusion in complex scenes

doi:10.11772/j.issn.1001-9081.2022101593

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (11): 3587-3593.DOI: 10.11772/j.issn.1001-9081.2022101593

• Multimedia computing and computer simulation • Previous Articles

Multi-scale ship detection based on adaptive feature fusion in complex scenes

Fang LUO¹(), Yang LIU¹, G. T. S HO²

^1.School of Computer Science and Artificial Intelligence，Wuhan University of Technology，Wuhan Hubei 430070，China
^2.Department of Supply Chain and Information Management，Hang Seng University of Hong Kong，Hong Kong 999077，China

Received:2022-10-25 Revised:2023-01-10 Accepted:2023-01-16 Online:2023-04-12 Published:2023-11-10
Contact: Fang LUO
About author:LUO Fang， born in 1977， Ph. D.， associate professor. Her research interests include data mining， intelligent information processing.
LIU Yang， born in 1999， M. S. candidate. His research interests include computer vision， object detection.
G. T. S HO， born in 1979， Ph. D.， assistant professor. His research interests include big data analysis， cloud computing.
Supported by:
Guangdong-Macao Innovation Technology Cooperation Funding Project(2021A0505080008);Zhuhai-Hong Kong-Macao Industry,Academic and Research Cooperation Project(ZH22017002200001PWC)

复杂场景下自适应特征融合的多尺度船舶检测

罗芳¹(), 刘阳¹, 何道森²

^1.武汉理工大学计算机与人工智能学院，武汉 430070
^2.香港恒生大学供应链及资讯管理学系，香港 999077

通讯作者: 罗芳
作者简介:罗芳（1977—），女，湖北天门人，副教授，博士，CCF会员，主要研究方向：数据挖掘、智能信息处理 luof@whut.edu.cn
刘阳（1999—），男，湖北十堰人，硕士研究生，主要研究方向：计算机视觉、目标检测
何道森（1979—），男，中国香港人，助理教授，博士，主要研究方向：大数据分析、云计算。
基金资助:
粤澳科技创新联合资助项目(2021A0505080008);产学研珠港澳合作项目(ZH22017002200001PWC)

Abstract

Abstract:

Under the influence of complex weather such as typhoon， heavy fog， rain and snow， as well as occlusions and scale changes， the existing ship detection methods have the problems of false detection and missed detection. In order to solve the above complex scene problems， based on YOLOX-S model， a multi-scale ship detection method based on adaptive feature fusion was proposed. Firstly， a feature augmentation module was introduced into the backbone feature extraction network to suppress the interference of complex background noise on ship feature extraction. Then， considering the problem of deep and shallow feature fusion proportion， an adaptive feature fusion module was designed to make full use of deep and shallow features， thereby improving the multi-scale ship detection ability of the model. Finally， in the detection head network， the detection head was decoupled and an adaptive multi-task loss function was introduced to balance classification tasks and regression tasks， thereby improving the multi-scale ship detection robustness of the model. Experimental results show that the detection mean Average Precision （mAP） of the proposed method on the public ship detection datasets SeaShips and McShips is 97.43% and 96.10%， respectively. The detection speed of the proposed method reaches 189 frames per second， which meets the requirements of real-time detection， demonstrating that the proposed method achieves high-precision detection of multi-scale ship targets even in complex scenes.

Key words: multi-scale ship detection, YOLOX, adaptive feature fusion, feature augmentation, multi-task loss function

摘要：

受台风、大雾、雨雪等复杂天气以及遮挡、尺度变化等影响，现有船舶检测方法存在误检和漏检问题。针对上述复杂场景问题，在YOLOX-S模型的基础上，提出一种自适应特征融合的多尺度船舶检测方法。首先，在主干特征提取网络中引入特征增强模块，抑制复杂背景噪声对船舶特征提取的干扰；其次，考虑深浅层次特征融合比例问题，设计自适应特征融合模块，充分利用深浅层次特征，提高模型的多尺度船舶检测能力；最后，在检测头网络，将检测头解耦，并引入自适应的多任务损失函数，平衡分类任务和回归任务，提高船舶检测的鲁棒性。实验结果显示，所提方法在公开船舶检测数据集SeaShips和McShips上的检测平均精度均值（mAP）分别达到了97.43%和96.10%，检测速度达到每秒189帧，满足实时检测的要求，验证了所提方法在复杂场景下仍能对多尺度船舶目标实现高精度检测。

关键词: 多尺度船舶检测, YOLOX, 自适应特征融合, 特征增强, 多任务损失函数

CLC Number:

TP391.4

Fang LUO, Yang LIU, G. T. S HO. Multi-scale ship detection based on adaptive feature fusion in complex scenes[J]. Journal of Computer Applications, 2023, 43(11): 3587-3593.

罗芳, 刘阳, 何道森. 复杂场景下自适应特征融合的多尺度船舶检测[J]. 《计算机应用》唯一官方网站, 2023, 43(11): 3587-3593.

Figures/Tables 12

References 20

1	王培玉，李峰，周书仁，等. 复杂码头环境下的船舶检测与跟踪算法［J］.计算机工程与科学， 2017， 39（5）：992-998. 10.3969/j.issn.1007-130X.2017.05.026
	WANG P Y， LI F， ZHOU S R， et al. A ship detection and tracking algorithm under complex wharf environment［J］. Computer Engineering and Science， 2017， 39（5）： 992-998. 10.3969/j.issn.1007-130X.2017.05.026
2	REN L， RAN X， PENG J， et al. Saliency detection for small maritime target using singular value decomposition of amplitude spectrum［J］. IETE Technical Review， 2017， 34（6）： 631-641. 10.1080/02564602.2016.1231023
3	黄致君，桑庆兵. 改进R-FCN的船舶识别方法［J］. 计算机科学与探索， 2020， 14（6）：1045-1053. 10.3778/j.issn.1673-9418.1904061
	HUANG Z J， SANG Q B. Ship detection based on improved R-FCN［J］. Journal of Frontiers of Computer Science and Technology， 2020， 14（6）：1045-1053. 10.3778/j.issn.1673-9418.1904061
4	李佳东，张丹普，范亚琼，等. 基于改进YOLOv5的轻量级船舶目标检测算法［J］. 计算机应用， 2023， 43（3）：923-929. 10.11772/j.issn.1001-9081.2022071096
	LI J D， ZHANG D P， FAN Y Q， et al. Lightweight ship target detection algorithm based on improved YOLOv5［J］. Journal of Computer Applications， 2023， 43（3）：923-929. 10.11772/j.issn.1001-9081.2022071096
5	姚婷婷，张波，柳晓鸣. 特征增强全卷积网络下的船舶检测［J］. 计算机辅助设计与图形学学报， 2022， 34（7）：1028-1036. 10.3724/sp.j.1089.2022.19105
	YAO T T， ZHANG B， LIU X M. Feature enhanced fully convolutional network for ship detection［J］. Journal of Computer-Aided Design and Computer Graphics， 2022， 34（7）： 1028-1036. 10.3724/sp.j.1089.2022.19105
6	王文亮，李延祥，张一帆，等. MPANet-YOLOv5：多路径聚合网络复杂海域目标检测［J］. 湖南大学学报（自然科学版）， 2022， 49（10）：69-76. 10.16339/j.cnki.hdxbzkb.2022360
	WANG W L， LI Y X， ZHANG Y F， et al. MPANet-YOLOv5： multi-path aggregation network for complex sea object detection［J］. Journal of Hunan University （Natural Sciences）， 2022， 49（10）：69-76. 10.16339/j.cnki.hdxbzkb.2022360
7	HUANG G， WAN Z， LIU X， et al. Ship detection based on squeeze excitation skip-connection path networks for optical remote sensing images［J］. Neurocomputing， 2019， 332： 215-223. 10.1016/j.neucom.2018.12.050
8	李辉. 面向航运船舶的检测与跟踪方法研究［D］. 武汉：武汉科技大学， 2019： 1-62.
	LI H. Research on detection and tracking methods for shipping vessel［D］. Wuhan： Wuhan University of Science and Technology， 2019： 1-62.
9	聂鑫，刘文，吴巍.复杂场景下基于增强YOLOv3的船舶目标检测［J］.计算机应用，2020，40（9）：2561-2570. 10.11772/j.issn.1001-9081.2020010097
	NIE X， LIU W， WU W. Ship detection based on enhanced YOLOv3 under complex environments［J］. Journal of Computer Applications， 2020， 40（9）： 2561-2570. 10.11772/j.issn.1001-9081.2020010097
10	GE Z， LIU S， WANG F， et al. YOLOX： exceeding YOLO series in 2021［EB/OL］. ［2022-07-18］..
11	YUN S， HAN D， CHUN S， et al. CutMix： regularization strategy to train strong classifiers with localizable features［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 6022-6031. 10.1109/iccv.2019.00612
12	LIN T Y， DOLLÁR P， GIRSHICK R， et al. Feature pyramid networks for object detection［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 936-944. 10.1109/cvpr.2017.106
13	HOU Q， ZHOU D， FENG J. Coordinate attention for efficient mobile network design［C］// Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2021： 13708-13717. 10.1109/cvpr46437.2021.01350
14	CIPOLLA R， GAI Y， KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics［C］// Proceedings of the 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 7482-7491. 10.1109/cvpr.2018.00781
15	SHAO Z， WU W， WANG Z， et al. SeaShips： a large-scale precisely annotated dataset for ship detection［J］. IEEE Transactions on Multimedia， 2018， 20（10）： 2593-2604. 10.1109/tmm.2018.2865686
16	ZHENG Y， ZHANG S. McShips： a large-scale ship dataset for detection and fine-grained categorization in the wild［C］// Proceedings of the 2020 IEEE International Conference on Multimedia and Expo. Piscataway： IEEE， 2020： 1-6. 10.1109/icme46284.2020.9102907
17	REN S， HE K， GIRSHICK R， et al. Faster R-CNN： towards real-time object detection with region proposal networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2017， 39（6）： 1137-1149. 10.1109/tpami.2016.2577031
18	LIU W， ANGUELOV D， ERHAN D， et al. SSD： single shot multiBox detector［C］// Proceedings of the 2016 European Conference on Computer Vision， LNCS 9905. Cham： Springer， 2016： 21-37.
19	REDMON J， FARHADI A. YOLOv3： an incremental improvement［EB/OL］. （2018-04-08）［2022-04-08］.. 10.1109/cvpr.2017.690
20	WANG C Y， BOCHKOVSKIY A， LIAO H Y M. YOLOv7： trainable bag-of-freebies sets new state-of-the-art for real-time object detectors［C］// Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2023： 7464-7475. 10.1109/cvpr52729.2023.00721

类型	输入	输出
Focus	640×640×3	320×320×12
Conv2D_BN_SiLU	320×320×12	320×320×32
Conv2D_BN_SiLU	320×320×32	160×160×64
CSPLayer	160×160×64	160×160×64
Conv2D_BN_SiLU	160×160×64	80×80×128
CSPLayer	80×80×128	80×80×128
特征增强模块	80×80×128	80×80×128
Conv2D_BN_SiLU	80×80×256	40×40×256
CSPLayer	40×40×256	40×40×256
特征增强模块	40×40×256	40×40×256
Conv2D_BN_SiLU	40×40×256	20×20×512
SPPBottleNeck	20×20×512	20×20×512
CSPLayer	20×20×512	20×20×512
特征增强模块	20×20×512	20×20×512

类型	输入	输出
Focus	640×640×3	320×320×12
Conv2D_BN_SiLU	320×320×12	320×320×32
Conv2D_BN_SiLU	320×320×32	160×160×64
CSPLayer	160×160×64	160×160×64
Conv2D_BN_SiLU	160×160×64	80×80×128
CSPLayer	80×80×128	80×80×128
特征增强模块	80×80×128	80×80×128
Conv2D_BN_SiLU	80×80×256	40×40×256
CSPLayer	40×40×256	40×40×256
特征增强模块	40×40×256	40×40×256
Conv2D_BN_SiLU	40×40×256	20×20×512
SPPBottleNeck	20×20×512	20×20×512
CSPLayer	20×20×512	20×20×512
特征增强模块	20×20×512	20×20×512

配置/参数	属性/取值	配置/参数	属性/取值
GPU	RTX 3080Ti	初始学习率	0.01
CPU	Intel i7-12700K	最小学习率	0.000 1
梯度下降方法	随机梯度下降	训练代数	100
初始动量	0.937	图片输入尺寸	640×640
学习率下降方式	Cos

配置/参数	属性/取值	配置/参数	属性/取值
GPU	RTX 3080Ti	初始学习率	0.01
CPU	Intel i7-12700K	最小学习率	0.000 1
梯度下降方法	随机梯度下降	训练代数	100
初始动量	0.937	图片输入尺寸	640×640
学习率下降方式	Cos

SeaShipst		McShips
船舶类别	数量	船舶类别	数量
矿砂船	2 199	民用船	5 945
散装货船	1 505	军用船	5 382
集装箱船	1 952
杂货船	901
渔船	2 190
客船	474

Multi-scale ship detection based on adaptive feature fusion in complex scenes

复杂场景下自适应特征融合的多尺度船舶检测

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 20

Related Articles 2

Recommended Articles

Metrics

模型	AP						精确率	召回率	mAP
模型	散装货船	集装箱船	渔船	杂货船	矿砂船	客船	精确率	召回率	mAP
Faster-RCNN	95.92	98.77	93.28	97.20	98.28	91.51	69.21	97.18	91.04
SSD	96.59	98.77	90.82	97.16	94.92	94.97	96.63	85.13	90.87
YOLOv3	94.76	97.92	93.47	97.11	95.65	89.63	94.81	87.64	83.67
YOLOv7	93.62	97.47	93.60	94.14	94.17	91.39	95.60	83.93	93.83
YOLOX	99.28	98.96	97.17	99.22	99.34	96.55	97.40	95.85	96.81
本文模型	99.52	98.96	97.89	99.55	99.55	98.84	97.37	96.54	97.43

改进方法	特征增强	自适应特征融合	多任务损失函数	AP						mAP
改进方法	特征增强	自适应特征融合	多任务损失函数	散装货船	集装箱船	渔船	杂货船	矿砂船	客船	mAP
a				97.02	98.61	95.81	97.25	97.42	94.75	96.81
b	√			97.30	99.27	95.61	98.06	97.78	94.66	97.12
c		√		98.19	99.69	95.75	97.53	96.82	95.44	97.24
d			√	97.92	99.26	95.81	97.86	97.43	94.17	97.07
本文方法	√	√	√	97.68	99.29	96.14	98.24	97.42	95.79	97.43

改进方法	特征增强	自适应特征融合	多任务损失函数	AP		mAP
改进方法	特征增强	自适应特征融合	多任务损失函数	民用船	军用船	mAP
a				93.61	97.71	95.66
b	√			93.66	97.67	95.67
c		√		94.04	97.33	95.69
d			√	93.98	97.69	95.83
本文方法	√	√	√	94.62	97.58	96.10

[1]	Doudou LI, Wanggen LI, Yichun XIA, Yang SHU, Kun GAO. Skeleton-based action recognition based on feature interaction and adaptive fusion [J]. Journal of Computer Applications, 2023, 43(8): 2581-2587.
[2]	Yiyu GUO, Luoyu ZHOU, Xinyu LIU, Yao LI. Dangerous goods detection method in elevator scene based on improved attention mechanism [J]. Journal of Computer Applications, 2023, 43(7): 2295-2302.

模型	AP		精确率	召回率	mAP
模型	民用船	军用船	精确率	召回率	mAP
Faster-RCNN	87.75	86.92	48.57	93.68	87.33
SSD	86.51	94.10	91.79	83.30	90.31
YOLOv3	90.04	92.59	92.21	82.92	91.31
YOLOv7	91.54	94.98	93.28	73.01	93.26
YOLOX	93.61	97.71	93.24	91.25	95.66
本文模型	94.62	97.58	93.22	91.68	96.10

模型	AP		精确率	召回率	mAP
模型	民用船	军用船	精确率	召回率	mAP
Faster-RCNN	87.75	86.92	48.57	93.68	87.33
SSD	86.51	94.10	91.79	83.30	90.31
YOLOv3	90.04	92.59	92.21	82.92	91.31
YOLOv7	91.54	94.98	93.28	73.01	93.26
YOLOX	93.61	97.71	93.24	91.25	95.66
本文模型	94.62	97.58	93.22	91.68	96.10