关键词: 图像融合, 红外与可见光图像融合, 无监督学习, 门控专家网络, 记忆补偿

Abstract: To address the reliance of existing unsupervised fusion networks on reconstruction constraints and static fusion modules, which prevents them from dynamically coordinating feature-processing paths according to bimodal content differences, a memory-gated expert network for infrared and visible image fusion was proposed. First, a dual-branch multi-scale feature encoding and expert fusion framework was constructed, in which a local spatial expert, a global channel expert, a structure-preserving expert, and a content-aware gating mechanism were used to dynamically coordinate different feature-processing paths. Then, a learnable prototype memory bank was introduced for each expert, and Top-k retrieval and soft-gated compensation were used to enhance degraded features. Finally, self-reconstruction, cross-reconstruction, gating-consistency, and memory regularization constraints were jointly imposed to improve the stability of unsupervised fusion. Experimental results show that, on MSRS, the Entropy (EN) and Structural Similarity Index Measure (SSIM) are 0.67% and 8.76% higher than those of the best-performing comparison network, SFDFuse and ITFuse, respectively. On LLVIP, EN and Standard Deviation (SD) are 0.84% and 7.95% higher than those of TDFusion, respectively. On TNO, SSIM is 3.58% higher than that of URFusion, while the Edge Information Fusion Quality Metric (Qabf) reaches 0.503 and ranks second. The network enhances infrared target saliency while preserving visible textures and structural information, thereby providing fused inputs that retain both target and background information for pedestrian detection in complex road scenes.

Key words: Keywords: image fusion, infrared and visible image fusion, unsupervised learning, gated expert network, memory compensation.