High-quality sonar image generation method based on multi-scale feature fusion

doi:10.11772/j.issn.1001-9081.2024121742

Journal of Computer Applications ›› 2025, Vol. 45 ›› Issue (12): 3987-3994.DOI: 10.11772/j.issn.1001-9081.2024121742

• Multimedia computing and computer simulation • Previous Articles Next Articles

High-quality sonar image generation method based on multi-scale feature fusion

Jing HUANG¹^,², Xin PENG², Wenhao LI², Kai HU², Teng WANG², Yamin HUANG³^,⁴, Yuanqiao WEN³^,⁴

^1.Research and Development Center of Transport Industry of New Generation of Artificial Intelligence Technology，Zhejiang Scientific Research Institute of Transport，Hangzhou Zhejiang 310023，China
^2.College of Computer and Artificial Intelligence，Wuhan University of Technology，Wuhan Hubei 430070，China
^3.Intelligent Transportation Systems Research Center （Wuhan University of Technology），Wuhan Hubei 430070，China
^4.National Engineering Research Center for Water Transport Safety，Wuhan Hubei 430063，China

Received:2024-12-10 Revised:2025-03-14 Accepted:2025-03-20 Online:2025-04-08 Published:2025-12-10
Contact: Jing HUANG
About author:HUANG Jing， born in 1977， Ph. D.， associate professor. His research interests include computer vision， artificial intelligence.
PENG Xin， born in 2000， M. S. candidate. His research interests include computer vision， artificial intelligence.
LI Wenhao， born in 1997， M. S. His research interests include computer vision， artificial intelligence.
HU Kai， born in 2001， M. S. candidate. His research interests include computer vision， artificial intelligence.
WANG Teng， born in 2000， M. S. candidate. His research interests include computer vision， artificial intelligence.
HUANG Yamin， born in 1990， Ph. D.， research fellow. His research interests include intelligent transportation.
WEN Yuanqiao， born in 1975， Ph. D.， professor. His research interests include intelligent transportation.
Supported by:
Project of Research and Development Center of Transport Industry of New Generation of Artificial Intelligence Technology(202302H);Research Program of Zhejiang Provincial Department of Transportation(2024006)

多尺度特征融合的高质量声呐图像生成方法

黄靖¹^,², 彭鑫², 李文豪², 胡凯², 王腾², 黄亚敏³^,⁴, 文元桥³^,⁴

^1.浙江省交通运输科学研究院新一代人工智能技术应用交通运输行业研发中心，杭州 310023
^2.武汉理工大学计算机与人工智能学院，武汉 430070
^3.智能交通系统研究中心（武汉理工大学），武汉 430070
^4.国家水运安全工程技术研究中心，武汉 430063

通讯作者: 黄靖
作者简介:黄靖（1977—），男，湖北潜江人，副教授，博士，主要研究方向：计算机视觉、人工智能
彭鑫（2000—），男，湖南株洲人，硕士研究生，主要研究方向：计算机视觉、人工智能
李文豪（1997—），男，河南周口人，硕士，主要研究方向：计算机视觉、人工智能
胡凯（2001—），男，湖北鄂州人，硕士研究生，主要研究方向：计算机视觉、人工智能
王腾（2000—），男，湖北黄冈人，硕士研究生，主要研究方向：计算机视觉、人工智能
黄亚敏（1990—），男，福建福州人，研究员，博士生导师，博士，主要研究方向：智能交通
文元桥（1975—），男，湖北松滋人，教授，博士生导师，博士，主要研究方向：智能交通。
基金资助:
新一代人工智能技术应用交通运输行业研发中心开放基金资助项目(202302H);浙江省交通厅科技项目(2024006)

Abstract

Abstract:

Due to the inherent characteristics of sonar imaging principles and the interference of complex underwater environments， underwater sonar images generally suffer from insufficient resolution and missing target details. To address these issues， a high-quality sonar image generation method based on multi-scale feature fusion was proposed. Firstly， the Residual Dense Blocks （RDBs） were used to extract image features at shallow level， thereby capturing basic texture and contour information， and establishing spatial layout of the image. Secondly， a Multi-Scale Attention feature extraction module （MSA） was designed to focus on key features at different scales adaptively and further enhance the expression of key features while suppressing redundant information expression through the attention mechanism. Finally， a discriminator network was constructed using a pixel-by-pixel discrimination strategy based on spectral normalization， which improved the reconstruction ability of complex object contours and details. Experimental results on an underwater sonar image dataset show that the proposed method achieves relative improvements of 6.7% and 5.4%， respectively， in Peak Signal-to-Noise Ratio （PSNR） and Structural SIMilarity （SSIM） metrics compared to the existing representative method ESRGAN （Enhanced Super-Resolution Generative Adversarial Network）. It can be seen that the proposed method improves the generation performance on underwater sonar image dataset effectively.

Key words: deep learning, high-quality image generation, Generative Adversarial Network (GAN), sonar image processing, feature fusion

摘要：

由于声呐图像成像原理本身的特点和复杂水下环境的干扰，水下声呐图像普遍存在分辨率不足和目标细节缺失的问题。针对这些问题，提出一种多尺度特征融合的高质量声呐图像生成方法。首先，通过残差密集连接模块（RDB）在浅层提取图像特征，以捕捉基本纹理与轮廓信息，并建立图像的空间布局；其次，设计多尺度注意力特征提取模块（MSA）自适应地聚焦不同尺度下的关键特征，并通过注意力机制在进一步增强关键特征表达的同时抑制冗余信息表达；最后，以基于谱归一化的逐像素判别策略构建判别器网络，从而提升复杂物体边缘和细节的重建能力。在水下声呐图像数据集上的实验结果表明，所提方法相较于现有代表性方法ESRGAN （Enhanced Super-Resolution Generative Adversarial Network）在峰值信噪比（PSNR）和结构相似性（SSIM）指标上分别取得了6.7%和5.4%的提升。可见，所提方法在水下声呐图像数据集上有效提升了生成效果。

关键词: 深度学习, 高质量图像生成, 生成对抗网络, 声呐图像处理, 特征融合

CLC Number:

TP391.4

Jing HUANG, Xin PENG, Wenhao LI, Kai HU, Teng WANG, Yamin HUANG, Yuanqiao WEN. High-quality sonar image generation method based on multi-scale feature fusion[J]. Journal of Computer Applications, 2025, 45(12): 3987-3994.

黄靖, 彭鑫, 李文豪, 胡凯, 王腾, 黄亚敏, 文元桥. 多尺度特征融合的高质量声呐图像生成方法[J]. 《计算机应用》唯一官方网站, 2025, 45(12): 3987-3994.

Figures/Tables 12

Fig. 1 Overall architecture of generator

Fig. 2 Structure of RDB

Fig. 3 Structure of MSF

Fig. 4 Structure of pixel-by-pixel discriminator

Fig. 5 Training process with mode collapse

Fig. 6 Improved training process

Tab. 1 Comparison of PSNR and SSIM values of different methods

方法	PSNR/dB	SSIM
双三次插值	20.62	0.32
SRCNN	21.42	0.51
SRGAN	23.53	0.53
ESRGAN	24.57	0.56
本文方法	26.21	0.59

Fig. 7 Comparison of generation effects of different methods

Fig. 8 Comparison of generation details of different methods

Tab. 2 Comparison of structural parameters and detection results for three model architectures

模型	参数量/10⁶	PSNR/dB	SSIM
RCB	14.3	22.24	0.55
MBC	12.3	23.42	0.61
MSF	13.6	24.56	0.60

Fig. 9 Comparison of image generation details of three models

Tab. 3 Results of ablation experiments on sonar dataset

对照组	RRDB	MSA	MSF	PSNR/dB
1	$√$	$√$	$√$	25.41
2	$√$	$×$	$×$	25.68
3	$√$	$×$	$√$	26.12
4	$√$	$√$	$×$	25.83
5	$√$	$√$	$√$	26.25

Tab. 3 Results of ablation experiments on sonar dataset

对照组	RRDB	MSA	MSF	PSNR/dB
1	$√$	$√$	$√$	25.41
2	$√$	$×$	$×$	25.68
3	$√$	$×$	$√$	26.12
4	$√$	$√$	$×$	25.83
5	$√$	$√$	$√$	26.25

References 32

[1]	KINGMA D P， WELLING M. Auto-encoding variational Bayes ［EB/OL］. ［2024-11-27］..
[2]	GOODFELLOW I， POUGET-ABADIE J， MIRZA M， et al. Generative adversarial nets［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 2. Cambridge： MIT Press， 2014： 2672-2680.
[3]	HO J， JAIN A， ABBEEL P. Denoising diffusion probabilistic models ［C］// Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook： Curran Associates Inc.， 2020： 6840-6851.
[4]	SONG J， MENG C， ERMON S. Denoising diffusion implicit models［EB/OL］. ［2024-03-02］..
[5]	刘彪. 基于声呐图像处理的水下目标探测关键技术研究［D］. 大理：大理大学， 2024： 16-17.
	LIU B. Research on key technology of underwater target detection based on sonar image processing ［D］. Dali： Dali University， 2024： 16-17.
[6]	魏光春，邢传玺，崔晶，等. 利用同步码字优化和正则化相结合的声呐图像降噪方法［J］. 云南民族大学学报（自然科学版）， 2024， 33（2）： 225-231.
	WEI G C， XING C X， CUI J， et al. A sonar image noise reduction method that combines simultaneous codeword optimization and regulation［J］. Journal of Yunnan Minzu University （Natural Sciences Edition）， 2024， 33（2）： 225-231.
[7]	冯伟，刘光宇，刘彪，等. 基于自适应维纳滤波和2D-VMD的声呐图像去噪算法［J］. 南京信息工程大学学报， 2024， 16（1）： 97-105.
	FENG W， LIU G Y， LIU B， et al. Sonar image denoising algorithm based on adaptive Wiener filtering and 2D-VMD［J］ Journal of Nanjing University of Information Science and Technology， 2024， 16（1）： 97-105.
[8]	王成杰. 基于各向异性引导滤波与双边滤波的前视声呐图像去噪［D］. 大连：大连海事大学， 2023： 34-38.
	WANG C J. Research on image denoising algorithm of forward-looking sonar based on anisotropic guided filter and bilateral filter［D］. Dalian： Dalian Maritime University， 2023： 34-38.
[9]	郭爱彬. 基于生成对抗网络的声呐图像生成方法研究［D］. 大连：大连海事大学， 2023： 23-27.
	GUO A B. Research on sonar image generation method based on generative adversarial network［D］. Dalian： Dalian Maritime University， 2023： 23-27.
[10]	LEDIG C， THEIS L， HUSZÁR F， et al. Photo-realistic single image super-resolution using a generative adversarial network［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 105-114.
[11]	刘秋逸. 基于生成对抗网络的声呐图像生成及盲去噪方法研究［D］. 哈尔滨：哈尔滨工程大学， 2020： 44-46.
	LIU Q Y. Generative adversarial networks based sonar image synthesis and blind denoising methods［D］. Harbin： Harbin Engineering University， 2020： 44-46.
[12]	LEE S， PARK B， KIM A. Deep learning from shallow dives： sonar image generation and training for underwater object detection［EB/OL］. ［2024-02-27］..
[13]	李响. 侧扫声纳图像目标样本生成和分类方法研究［D］. 哈尔滨：哈尔滨工程大学， 2020： 44-45.
	LI X. Research on target sample generation and classification of side scan sonar image［D］. Harbin： Harbin Engineering University， 2020： 44-45.
[14]	马麒翔. 基于深度学习的声呐图像目标检测算法研究［D］. 南京：东南大学， 2020： 59-60.
	MA Q X. Research on object detection algorithm of sonar image based on deep learning［D］. Nanjing： Southeast University， 2020： 59-60.
[15]	于淼. 基于深度学习的侧扫声呐图像目标检测方法研究［D］. 哈尔滨：哈尔滨工程大学， 2020： 9-11.
	YU M. Research on target detection method of side-scan sonar images based on deep learning［D］. Harbin： Harbin Engineering University， 2020： 9-11.
[16]	梁雪灿. 基于生成对抗网络的声学图像超分辨率研究［D］. 哈尔滨：哈尔滨工程大学， 2019： 51-53.
	LIANG X C. Research on acoustic image super-resolution based on generative adversarial network［D］. Harbin： Harbin Engineering University， 2019： 51-53.
[17]	李宝奇，黄海宁，刘纪元，等. 基于改进CycleGAN的光学图像迁移生成水下小目标合成孔径声纳图像算法研究［J］. 电子学报， 2021， 49（9）： 1746-1753.
	LI B Q， HUANG H N， LIU J Y， et al. Optical image-to-underwater small target synthetic aperture sonar image translation algorithm based on improved CycleGAN［J］. Acta Electronica Sinica， 2021， 49（9）： 1746-1753.
[18]	WANG Z， GUO Q， LEI M， et al. High-quality sonar image generation algorithm based on generative adversarial networks［C］// Proceedings of the 40th Chinese Control Conference. Piscataway： IEEE， 2021： 3099-3104.
[19]	程文博. 基于生成对抗网络的声呐图像超分辨率算法［J］. 数字海洋与水下攻防， 2023， 6（3）： 353-358.
	CHENG W B. Sonar image super-resolution algorithm based on generative adversarial network［J］. Digital Ocean and Underwater Warfare， 2023， 6（3）： 353-358.
[20]	RONNEBERGER O， FISCHER P， BROX T. U-Net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
[21]	GURROLA-RAMOS J， DALMAU O， ALARCÓN T E. A residual dense U-Net neural network for image denoising［J］. IEEE Access， 2021， 9： 31742-31754.
[22]	HUANG G， LIU Z， VAN DER MAATEN L， et al. Densely connected convolutional networks［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 2261-2269.
[23]	SZEGEDY C， LIU W， JIA Y， et al. Going deeper with convolutions［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 1-9.
[24]	IOFFE S， SZEGEDY C. Batch normalization： accelerating deep network training by reducing internal covariate shift［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 448-456.
[25]	SZEGEDY C， VANHOUCKE V， IOFFE S， et al. Rethinking the inception architecture for computer vision［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 2818-2826.
[26]	SZEGEDY C， IOFFE S， VANHOUCKE V， et al. Inception-v4， Inception-ResNet and the impact of residual connections on learning［C］// Proceedings of the 31st AAAI Conference on Artificial Intelligence. Palo Alto： AAAI Press， 2017： 4278-4284.
[27]	HE K， ZHANG X， REN S， et al. Spatial pyramid pooling in deep convolutional networks for visual recognition［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2015， 37（9）： 1904-1916.
[28]	MIYATO T， KATAOKA T， KOYAMA M， et al. Spectral normalization for generative adversarial networks ［EB/OL］. ［2024-05-24］..
[29]	YOSHIDA Y， MIYATO T. Spectral norm regularization for improving the generalizability of deep learning ［EB/OL］. ［2024-05-28］..
[30]	CARLSON R E， FRITSCH F N. An algorithm for monotone piecewise bicubic interpolation［J］. SIAM Journal on Numerical Analysis， 1989， 26（1）： 230-238.
[31]	DONG C， LOY C C， HE K， et al. Image super-resolution using deep convolutional networks［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2016， 38（2）： 295-307.
[32]	WANG X， YU K， WU S， et al. ESRGAN： enhanced super-resolution generative adversarial networks［C］// Proceedings of the 2018 European Conference on Computer Vision Workshops， LNCS 11133. Cham： Springer， 2019： 63-79.

High-quality sonar image generation method based on multi-scale feature fusion

多尺度特征融合的高质量声呐图像生成方法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 12

References 32

Related Articles 15

Recommended Articles

Metrics

[1]	Weigang LI, Jiale SHAO, Zhiqiang TIAN. Point cloud classification and segmentation network based on dual attention mechanism and multi-scale fusion [J]. Journal of Computer Applications, 2025, 45(9): 3003-3010.
[2]	Zhixiong XU, Bo LI, Xiaoyong BIAN, Qiren HU. Adversarial sample embedded attention U-Net for 3D medical image segmentation [J]. Journal of Computer Applications, 2025, 45(9): 3011-3016.
[3]	Fang WANG, Jing HU, Rui ZHANG, Wenting FAN. Medical image segmentation network with content-guided multi-angle feature fusion [J]. Journal of Computer Applications, 2025, 45(9): 3017-3025.
[4]	Panfeng JING, Yudong LIANG, Chaowei LI, Junru GUO, Jinyu GUO. Semi-supervised image dehazing algorithm based on teacher-student learning [J]. Journal of Computer Applications, 2025, 45(9): 2975-2983.
[5]	Yiming LIANG, Jing FAN, Wenze CHAI. Multi-scale feature fusion sentiment classification based on bidirectional cross attention [J]. Journal of Computer Applications, 2025, 45(9): 2773-2782.
[6]	Hongjun ZHANG, Gaojun PAN, Hao YE, Yubin LU, Yiheng MIAO. Multi-source heterogeneous data analysis method combining deep learning and tensor decomposition [J]. Journal of Computer Applications, 2025, 45(9): 2838-2847.
[7]	Jin LI, Liqun LIU. SAR and visible image fusion based on residual Swin Transformer [J]. Journal of Computer Applications, 2025, 45(9): 2949-2956.
[8]	Bing YIN, Zhenhua LING, Yin LIN, Changfeng XI, Ying LIU. Emotion recognition method compatible with missing modal reasoning [J]. Journal of Computer Applications, 2025, 45(9): 2764-2772.
[9]	Yilin DENG, Fajiang YU. Pseudo random number generator based on LSTM and separable self-attention mechanism [J]. Journal of Computer Applications, 2025, 45(9): 2893-2901.
[10]	Peng PENG, Ziting CAI, Wenling LIU, Caihua CHEN, Wei ZENG, Baolai HUANG. Speech emotion recognition method based on hybrid Siamese network with CNN and bidirectional GRU [J]. Journal of Computer Applications, 2025, 45(8): 2515-2521.
[11]	Shuo ZHANG, Guokai SUN, Yuan ZHUANG, Xiaoyu FENG, Jingzhi WANG. Dynamic detection method of eclipse attacks for blockchain node analysis [J]. Journal of Computer Applications, 2025, 45(8): 2428-2436.
[12]	Jinhao LIN, Chuan LUO, Tianrui LI, Hongmei CHEN. Thoracic disease classification method based on cross-scale attention network [J]. Journal of Computer Applications, 2025, 45(8): 2712-2719.
[13]	Lina GE, Mingyu WANG, Lei TIAN. Review of research on efficiency of federated learning [J]. Journal of Computer Applications, 2025, 45(8): 2387-2398.
[14]	Yimeng XI, Zhen DENG, Qian LIU, Libo LIU. Cross-modal information fusion for video-text retrieval [J]. Journal of Computer Applications, 2025, 45(8): 2448-2456.
[15]	Jin ZHOU, Yuzhi LI, Xu ZHANG, Shuo GAO, Li ZHANG, Jiachuan SHENG. Modulation recognition network for complex electromagnetic environments [J]. Journal of Computer Applications, 2025, 45(8): 2672-2682.