Abstract: Abstract: In the fusion of synthetic aperture radar and visible light images, the existing methods usually face the challenges of large modal differences, information loss and high computational complexity, so a SAR and visible image fusion algorithm based on residual Swin transformer model is proposed. Firstly, Swin transformer was used as the backbone to extract global features, and a full-attention feature coding backbone network was used to model remote dependencies. Secondly, in order to improve the fusion effect, three different fusion strategies were designed: the feature fusion strategy based on the L1 norm of the sequence matrix, the fusion strategy based on the image pyramid and downsampling, and then the final fusion result was obtained by weighting the average of the three results, which effectively adjusted the pixel value and reduced the noise of the SAR image, better retained the clear details and structural information of the visible light image, and fused the feature information of the SAR image and the visible light image at different scales. Finally, a large number of experiments are carried out on the SEN1-2 dataset and the QXS-SARPT dataset, and the results show that compared with other fusion methods, the objective evaluation indexes AG, MI, SD, PSNR, SF and EN of the proposed method are increased by 13.44%, 18.12%, 16.72%, 6.34%, 22.93% and 12.69%, respectively, and the results are better than those of the existing methods. The ablation experiments also proved the effectiveness of the improved fusion strategy, which provided new ideas and methods for further research in the field of fusion of SAR and visible images.

Key words: Keywords: Synthetic Aperture Radar, visible image, image fusion, transformer, deep learning

摘要： 摘 要: 在合成孔径雷达与可见光图像的融合研究中，现有方法通常面临着模态间差异大、信息丢失和计算复杂度高等挑战，因此提出了一种基于残差Swin transformer模型的SAR和可见光图像融合算法。首先，采用Swin transformer作为主干来完成提取全局特征，用一个全注意力特征编码主干网络来建模远程依赖关系；其次，为了提高融合效果，设计了三种不同的融合策略：基于序列矩阵的L1范数的特征融合策略、基于图像金字塔的融合策略及下采样，再将三个结果加权平均得到最终的融合结果，有效地调节像素值并减少SAR图像的噪声，更好地保留可见光图像清晰的细节和结构信息，融合了SAR图像和可见光图像不同尺度的地物特征信息。最后，在SEN1-2数据集以及QXS-SAROPT数据集上进行了大量实验验证，结果表明，与其他融合方法相比，所提方法的客观评价指标AG、MI、SD、PSNR、SF及EN分别平均提升了13.44%、18.12%、16.72%、6.34%、22.93%和12.69%，得到了较现有方法更佳的结果，通过消融实验也证明了改进的融合策略具备有效性，为SAR和可见光图像的融合领域的进一步研究提供了新的思路和方法

关键词: 合成孔径雷达, 可见光图像, 图像融合, transformer, 深度学习