Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (4): 1172-1178.DOI: 10.11772/j.issn.1001-9081.2020071064

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Improved remote sensing image fusion algorithm based on channel attention feedback network

WU Lei, YANG Xiaomin   

  1. College of Electronics and Information Engineering, Sichuan University, Chengdu Sichuan 610065, China
  • Received:2020-07-21 Revised:2020-09-18 Online:2021-04-10 Published:2020-10-19
  • Supported by:
    This work is partially supported by the National Natural Science Foundation of China (61701327).


吴蕾, 杨晓敏   

  1. 四川大学 电子信息学院, 成都 610065
  • 通讯作者: 杨晓敏
  • 作者简介:吴蕾(1996—),女,湖北黄冈人,硕士研究生,主要研究方向:遥感图像融合;杨晓敏(1980—),女,四川成都人,教授,博士,主要研究方向:图像处理、机器学习。
  • 基金资助:

Abstract: Aiming at the problems of feedforward Convolutional Neural Network(CNN), such as small receptive field, insufficient context information acquirement and that only shallow features can be extracted by the feature extraction convolutional layer of the network, an improved remote sensing image fusion algorithm based on channel attention feedback network was proposed. Firstly, the detail features of PANchromatic(PAN) images and the spectral features of Low-resolution MultiSpectral(LMS) images were initially extracted through two convolutional layers. Secondly, the extracted features were combined with the deep features fed back from the network and inputted to the channel attention mechanism module to obtain the initially refined features. Thirdly, the deep features with stronger characterization capability were generated by feedback module. Finally, High-resolution MultiSpectral(HMS) images were obtained by putting the generated deep features into the reconstruction layer with deconvolution. Experimental results on three different satellite image datasets show that the proposed algorithm can well extract the detail features of PAN images and the spectral features of LMS images, and the HMS images recovered by this algorithm are clearer subjectively and better than the comparison algorithms objectively; at the same time, the Root Mean Square Error(RMSE) index of the proposed method is more than 50% lower than that the traditional methods, and more than 10% lower than that the feedforward convolutional network methods.

Key words: remote sensing image fusion, channel attention, feedback network, Convolutional Neural Network (CNN), deep learning

摘要: 针对前馈卷积神经网络(CNN)感受野较小、获取上下文信息不足、其特征提取卷积层只能提取到浅层特征的问题,提出改进的基于通道注意力反馈网络的遥感图像融合算法。首先,通过两层卷积层分别初步提取全色(PAN)图像的细节特征和低分辨率多光谱(LMS)图像的光谱特征;其次,将提取的特征和网络反馈的深层特征相结合,并将其输入到通道注意力机制模块中以得到初步精细化特征;然后,经过反馈模块生成表征能力更强的深层特征;最后,将生成的深层特征经过含有反卷积的重建层,从而得到高分辨率多光谱(HMS)图像。在三个不同卫星图像数据集上的实验结果表明:所提算法能很好地提取PAN图像的细节特征和LMS图像的光谱特征,同时其恢复出来的HMS图像在主观视觉上更加清晰,并且在客观评价指标上优于对比算法,同时在均方根误差(RMSE)指标上,所提算法比传统算法降低了50%以上,比前馈卷积神经网络算法降低了10%以上。

关键词: 遥感图像融合, 通道注意力, 反馈网络, 卷积神经网络, 深度学习

CLC Number: