Abstract: Multimodal image fusion aims to generate a fused image that incorporates significant features and detailed textures from various modalities. Most existing methods fail to exploit complementary information across multiple temporal steps and rely on fixed loss weights, leading to sub-optimal fusion quality. To address these issues, we propose a Temporal Dynamic Fusion and Adaptive Feature Refinement Network (TDFuse) for infrared and visible image fusion. The network is collaboratively formed by a Temporal Dynamic Fusion Module (TDFM) — which simulates temporal processing mechanisms through multi-timestep parallel feature extraction — and a Temporally-Aware Adaptive Weighting Loss Function (TA-Loss). The former employs three transformer-encoder branches for different timesteps, whose importance is dynamically weighted and fused via a dynamic weighting network, enabling adaptive feature integration. The latter dynamically adjusts loss weights based on input images to optimize the training process in real-time. Experimental results on the TNO and RoadScene datasets demonstrate that TDFuse outperforms other state-of-the-art methods, particularly in the Structural Similarity (SSIM) metric, where it achieves improvements of 12.2% and 12.3% respectively compared to CDDFuse (Correlation-Driven feature Decomposition Fusion), showcasing its powerful fusion capability and temporal adaptability. Additionally, TDFuse showed outstanding performance in preserving detail information and edge structures, providing a new research direction for multimodal image fusion.

Key words: multi-modal image fusion, temporal dynamic fusion, adaptive feature refinement, adaptive weighting loss function, temporal adaptability

摘要： 多模态图像融合旨在生成兼具各模态显著特征与细节纹理的融合图像。现有方法多数未利用多时间步互补特性，且权重损失固定，导致融合效果不佳。针对这些问题，提出时序动态融合与自适应特征精化网络（TDFuse），用于红外与可见光图像融合。该网络由时序动态融合模块（TDFM）（通过多时间步并行特征提取模拟时序处理机制）与时序感知自适应权重损失函数（TA-Loss）协同构成：前者包含3个时间步Transformer编码分支，各时间步重要性由动态权重网络评估，完成自适应动态特征融合；后者根据输入图像实时调节损失权重，动态优化训练过程。在TNO、RoadScene数据集上的实验结果显示，TDFuse优于其他先进方法，特别是在结构相似性（SSIM）指标上，相较于CDDFuse（Correlation-Driven feature Decomposition Fusion）分别提升12.2%和12.3%，展现了强大的融合能力和时序适应性。此外，TDFuse在保留细节信息和边缘结构方面表现突出，为多模态图像融合提供了新的研究方向。

关键词: 多模态图像融合, 时序动态融合, 自适应特征精化, 自适应权重损失函数, 时序适应性