关键词: 多模态图像融合, 时序动态融合, 自适应特征精化, 自适应权重损失函数, 时序适应性

Abstract: Multimodal image fusion aims to generate a fused image that incorporates significant features and detailed textures from various modalities. Most existing methods fail to exploit complementary information across multiple temporal steps and rely on fixed loss weights, leading to sub-optimal fusion quality. To address these issues, we propose a Temporal Dynamic Fusion and Adaptive Feature Refinement Network (TDFuse) for infrared and visible image fusion. The network is collaboratively formed by a Temporal Dynamic Fusion Module (TDFM) — which simulates temporal processing mechanisms through multi-timestep parallel feature extraction — and a Temporally-Aware Adaptive Weighting Loss Function (TA-Loss). The former employs three transformer-encoder branches for different timesteps, whose importance is dynamically weighted and fused via a dynamic weighting network, enabling adaptive feature integration. The latter dynamically adjusts loss weights based on input images to optimize the training process in real-time. Experimental results on the TNO and RoadScene datasets demonstrate that TDFuse outperforms other state-of-the-art methods, particularly in the Structural Similarity (SSIM) metric, where it achieves improvements of 12.2% and 12.3% respectively compared to CDDFuse (Correlation-Driven feature Decomposition Fusion), showcasing its powerful fusion capability and temporal adaptability. Additionally, TDFuse showed outstanding performance in preserving detail information and edge structures, providing a new research direction for multimodal image fusion.

Key words: multi-modal image fusion, temporal dynamic fusion, adaptive feature refinement, adaptive weighting loss function, temporal adaptability