《计算机应用》唯一官方网站

• •    下一篇

基于光照权重分配和注意力的红外与可见光图像融合深度学习模型

魏文亮,王阳萍,岳彪,王安政,张哲   

  1. 兰州交通大学
  • 收稿日期:2023-07-19 修回日期:2023-10-06 发布日期:2023-10-26 出版日期:2023-10-26
  • 通讯作者: 魏文亮

Deep learning model for infrared and visible image fusion based on illumination weight allocation and attention

  • Received:2023-07-19 Revised:2023-10-06 Online:2023-10-26 Published:2023-10-26

摘要: 针对现有红外与可见光图像融合模型在融合过程中忽略光照因素,使用常规的融合策略,导致融合结果存在细节信息丢失,显著信息不明显等问题,提出了一种基于光照权重分配和注意力的红外与可见光图像融合深度学习模型。首先,设计光照权重分配网络(IWA-Net)来估计光照分布并计算光照权重;其次,引入CM-L1范式融合策略提高像素之间的依赖关系,完成对显著特征的平滑处理;最后,由全卷积层构成解码网络,来完成对融合图像的重构。在公开数据集上融合实验结果表明,所提模型相较于对比模型,所选六种评价指标均有所提高,其中空间频率(SF)和互信息(MI)指标分别平均提高了0.45倍、0.41倍,有效减少边缘模糊,使得融合图像具有较高的清晰度和对比度。该模型的融合结果在主客观方面均优于其他对比模型。

Abstract: Aiming to solve the problems that existing infrared and visible image fusion models which ignored illumination factor in fusion process and used conventional fusion strategy leaded to the fusion results with the loss of detail information and inconspicuous salient information, a deep learning model for infrared and visible image fusion based on illumination weight allocation and attention was proposed. Firstly, an Illumination Weight Allocation Network (IWA-Net) was designed to estimate the illumination distribution and calculated illumination weights. Secondly, a CM-LI-norm fusion strategy was introduced to enhance dependency between pixels and achieved smooth processing of salient features. Finally, a decoding network composed of fully convolutional layers was employed to reconstruct fused images. The results of the fusion experiments on publicly available datasets show that the proposed model outperforms the compared models, with improvements observed in all six selected evaluation metrics. Specifically, the Spatial Frequency (SF) and Mutual Information (MI) metrics have increased by an average of 0.45 and 0.41, respectively, which effectively reduces edge blurring and enhances clarity and contrast of the fused image. The fusion results of this model exhibit superior performance in both subjective and objective aspects compared to other contrastive models.

中图分类号: