关键词: 合成孔径雷达, 可见光图像, 图像融合, transformer, 深度学习

Abstract: Abstract: In the fusion of synthetic aperture radar and visible light images, the existing methods usually face the challenges of large modal differences, information loss and high computational complexity, so a SAR and visible image fusion algorithm based on residual Swin transformer model is proposed. Firstly, Swin transformer was used as the backbone to extract global features, and a full-attention feature coding backbone network was used to model remote dependencies. Secondly, in order to improve the fusion effect, three different fusion strategies were designed: the feature fusion strategy based on the L1 norm of the sequence matrix, the fusion strategy based on the image pyramid and downsampling, and then the final fusion result was obtained by weighting the average of the three results, which effectively adjusted the pixel value and reduced the noise of the SAR image, better retained the clear details and structural information of the visible light image, and fused the feature information of the SAR image and the visible light image at different scales. Finally, a large number of experiments are carried out on the SEN1-2 dataset and the QXS-SARPT dataset, and the results show that compared with other fusion methods, the objective evaluation indexes AG, MI, SD, PSNR, SF and EN of the proposed method are increased by 13.44%, 18.12%, 16.72%, 6.34%, 22.93% and 12.69%, respectively, and the results are better than those of the existing methods. The ablation experiments also proved the effectiveness of the improved fusion strategy, which provided new ideas and methods for further research in the field of fusion of SAR and visible images.

Key words: Keywords: Synthetic Aperture Radar, visible image, image fusion, transformer, deep learning