Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (9): 2963-2969.DOI: 10.11772/j.issn.1001-9081.2022091458

• Multimedia computing and computer simulation • Previous Articles     Next Articles

Remote sensing image pansharpening by convolutional neural network

Kunting LU, Rongrong FEI, Xuande ZHANG()   

  1. School of Electronic Information and Artificial Intelligence,Shaanxi University of Science and Technology,Xi’an Shaanxi 710021,China
  • Received:2022-10-08 Revised:2022-12-01 Accepted:2022-12-06 Online:2023-02-22 Published:2023-09-10
  • Contact: Xuande ZHANG
  • About author:LU Kunting, born in 1999, M. S. candidate. Her research interests include remote sensing image fusion.
    FEI Rongrong, born in 1991, Ph. D., lecturer. Her research interests include remote sensing image processing, computer vision.
  • Supported by:
    National Natural Science Foundation of China(61871260);Natural Science Basic Research Program of Shaanxi Province(2020JQ-346)

融合卷积神经网络的遥感图像全色锐化

路琨婷, 费蓉蓉, 张选德()   

  1. 陕西科技大学 电子信息与人工智能学院,西安 710021
  • 通讯作者: 张选德
  • 作者简介:路琨婷(1999—),女,陕西渭南人,硕士研究生,主要研究方向:遥感图像融合
    费蓉蓉(1991—),女,陕西咸阳人,讲师,博士,主要研究方向:遥感图像处理、计算机视觉;
  • 基金资助:
    国家自然科学基金资助项目(61871260);陕西省自然科学基础研究计划项目(2020JQ-346)

Abstract:

In the linear injection models of traditional Component Substitution (CS) and Multi-Resolution Analysis (MRA) in remote sensing image pansharpening, the relative spectral response of the sensor used for pansharpening is not considered. At the same time, the insufficient feature extraction of the original images caused by the deep learning-based methods results in the loss of spectral and spatial information in the fusion results. Aiming at the above problems, a pansharpening method combining traditional and deep learning methods was proposed, namely CMRNet. Firstly, CS and MRA were combined with Convolutional Neural Network (CNN) to achieve nonlinearity and improve the performance of pansharpening method. Secondly, the Residual Channel (RC) blocks were designed to realize the fusion and extraction of multi-scale feature information, and the feature maps of different channels were assigned different weights by using Channel Attention (CA) adaptively to learn more effective information. Experiments were conducted on QuickBird and GF1 satellite datasets. Experimental results show that on downscale QuickBird and GF1 datasets, compared with the classic method PanNet, CMRNet has the Peak Signal-to-Noise Ratio (PSNR) increased by 5.48% and 9.62% respectively, and other indexes also improved significantly, which verifies that CMRNet can achieve a better pansharpening effect.

Key words: Convolutional Neural Network (CNN), remote sensing image processing, pansharpening, component substitution, Multi-Resolution Analysis (MRA)

摘要:

在遥感图像全色锐化中,传统的成分替换(CS)和多分辨率分析(MRA)方法的线性注入模型没有考虑用于全色锐化传感器的相对光谱响应,而基于深度学习的方法对原图像特征的提取不足会导致融合结果中的光谱和空间信息的丢失。针对以上问题,提出一种结合传统与深度学习方法的全色锐化方法CMRNet。首先,将CS和MRA与卷积神经网络(CNN)相结合以实现非线性从而提高全色锐化方法性能;其次,设计残差通道(RC)块实现多尺度特征信息的融合提取,并利用通道注意力(CA)自适应地为不同通道的特征图分配不同的权值,从而学习更有效的信息。在QuickBird和GF1卫星数据集上对CMRNet进行训练和测试,实验结果表明,在降尺度QuickBird和GF1数据集上,与经典方法PanNet相比,CMRNet的峰值信噪比(PSNR)分别提高了5.48%和9.62%,其他指标也均有显著提高。可见,CMRNet能实现较好的全色锐化效果。

关键词: 卷积神经网络, 遥感图像处理, 全色锐化, 成分替换, 多分辨率分析

CLC Number: