《计算机应用》唯一官方网站 ›› 2023, Vol. 43 ›› Issue (11): 3574-3578.DOI: 10.11772/j.issn.1001-9081.2022101553

• 多媒体计算与计算机仿真 • 上一篇    下一篇

保留梯度和轮廓的可见光与红外图像融合

韩林凯, 姚江伟, 王坤峰()   

  1. 北京化工大学 信息科学与技术学院,北京 100029
  • 收稿日期:2022-10-15 修回日期:2023-02-01 接受日期:2023-02-06 发布日期:2023-05-24 出版日期:2023-11-10
  • 通讯作者: 王坤峰
  • 作者简介:韩林凯(1998—),男,山西运城人,硕士研究生,主要研究方向:图像融合、目标检测
    姚江伟(1996—),男,山西太原人,硕士研究生,主要研究方向:图像融合、图像配准
    王坤峰(1982—),男,河南周口人,教授,博士,CCF会员,主要研究方向:计算机视觉、机器学习、多模态人工智能。wangkf@buct.edu.cn
  • 基金资助:
    国家重点研发计划项目(2020YFC2003900);国家自然科学基金资助项目(62076020)

Visible and infrared image fusion by preserving gradients and contours

Linkai HAN, Jiangwei YAO, Kunfeng WANG()   

  1. College of Information Science and Technology,Beijing University of Chemical Technology,Beijing 100029,China
  • Received:2022-10-15 Revised:2023-02-01 Accepted:2023-02-06 Online:2023-05-24 Published:2023-11-10
  • Contact: Kunfeng WANG
  • About author:HAN Linkai, born in 1998, M. S. candidate. His research interests include image fusion, object detection.
    YAO Jiangwei, born in 1996, M. S. candidate. His research interests include image fusion, image registration.
    WANG Kunfeng, born in 1982, Ph. D., professor. His research interests include computer vision, machine learning, multimodal artificial intelligence.
  • Supported by:
    National Key Research and Development Program of China(2020YFC2003900);National Natural Science Foundation of China(62076020)

摘要:

为了解决可见光与红外图像采用基础拉普拉斯融合(Laplacian Blending)时,存在热源物体的轮廓不清晰以及曝光严重区域图像内容缺失的问题,提出一种保留红外轮廓与梯度信息的图像融合方法。首先,对输入图像进行颜色空间转换和自适应形态学去噪,并将两幅图像的梯度对比和红外图像突出目标的轮廓作为像素活动信息的权值;其次,同时分解权值与输入图像,并采用基于相似度的比较调整权重分配;最后,重构图像并转换颜色空间。在主观评价中,所提方法未产生伪影和怪异色彩,图像中的发热目标轮廓清晰;在客观评价指标中,该方法的熵(EN)为7.49,边缘梯度(EI)为74.61,平均梯度(AG)为7.23,与传统多尺度变换方法(包括非下采样轮廓波变换(NSCT)方法和基于非下采样剪切波变换(NSST)多尺度熵方法)和深度学习方法(结合残差网络(ResNet)与零相位分量分析(ZCA)的图像融合方法)相比,它的EN分别提升了0.10、0.58和0.75,EI分别提升了6.65、20.35和37.35,AG分别提升了0.73、2.19和3.55;而且它在Intel i5系列计算机上的处理速度达到5 frame/s,计算复杂度低。

关键词: 图像融合, 图像去噪, 权值分配, 拉普拉斯分解, 轮廓保留

Abstract:

In order to solve the problems of unclear contours of heat source objects and missing image content in severely exposed regions when visible and infrared images are fused by using basic Laplacian blending, an image fusion algorithm that preserves infrared contours and gradient information was proposed. Firstly, the input image was transformed into color space and denoised by adaptive morphology, and the gradient contrast of the two images and the contour of the highlighted object in the infrared image were taken as the weights of pixel activity information. Secondly, the weights and the input images were decomposed simultaneously, and the weight assignment was adjusted by similarity-based comparison. Finally, the image was reconstructed and the color space was transformed. In subjective evaluation, the proposed algorithm does not produce artifacts and strange colors, and the contours of the heat object in the obtained image is clear. In objective evaluation, the proposed algorithm has an ENtropy (EN) of 7.49, an Edge Intensity (EI) of 74.61, and an Average Gradient (AG) of 7.23, compared with the traditional multi-scale transformation methods (including Non-Subsampled Contourlet Transformation (NSCT) method, the method based on Non-Subsampled Shearlet Transform (NSST) multi-scale entropy) and the latest deep learning method (such as the method combining Residual Network (ResNet) and Zero-phase Component Analysis (ZCA)), it improves EN by 0.10, 0.58 and 0.75, EI by 6.65, 20.35 and 37.35, and AG by 0.73, 2.19 and 3.55; it also achieves a processing speed of 5 frame/s on Intel i5 series computers with low computational complexity.

Key words: image fusion, image denoising, weight assignment, Laplacian decomposition, contour preserving

中图分类号: