Remote sensing image fusion enhancement model for power facilities based on deep convolutional network

doi:10.11772/j.issn.1001-9081.2023121794

Journal of Computer Applications ›› 0, Vol. ›› Issue (): 212-216.DOI: 10.11772/j.issn.1001-9081.2023121794

• Multimedia computing and computer simulation • Previous Articles Next Articles

Remote sensing image fusion enhancement model for power facilities based on deep convolutional network

Fangrong ZHOU(), Yifan WANG, Yi MA, Gang WEN, Guofang WANG, Yutang MA, Hao GENG

Joint Laboratory of Power Remote Sensing Technology，Electric Power Science Research Institute of Yunnan Power Grid Company Limited，Kunming Yunnan 650217，China

Received:2023-12-26 Revised:2024-05-16 Accepted:2024-05-17 Online:2025-01-24 Published:2024-12-31
Contact: Fangrong ZHOU

基于深度卷积网络的电力设施遥感图像融合增强模型

周仿荣(), 王一帆, 马仪, 文刚, 王国芳, 马御棠, 耿浩

云南电网有限责任公司电力科学研究院电力遥感技术联合实验室，昆明 650217

通讯作者: 周仿荣
作者简介:周仿荣（1982—），男，贵州贵阳人，正高级工程师，博士，主要研究方向：电网防灾减灾
王一帆（1994—），男，云南昆明人，工程师，博士，主要研究方向：电网防灾减灾
马仪（1969—），男，云南昆明人，正高级工程师，硕士，主要研究方向：电网防灾减灾
文刚（1992—），男，湖北武汉人，工程师，硕士，主要研究方向：电网防灾减灾
王国芳（1982—），男，云南昆明人，高级工程师，硕士，主要研究方向：电网防灾减灾
马御棠（1986—），男，四川成都人，高级工程师，博士，主要研究方向：电网防灾减灾
耿浩（1995—），男，云南昆明人，工程师，硕士，主要研究方向：电网防灾减灾。
基金资助:
云南省重大科技专项(202202AD080010);南方电网重点科技项目(056200KK52220011)

Abstract

Abstract:

To meet the demand of high spatio-temporal resolution remote sensing images for power facilities safety monitoring and emergency management， a deep convolutional network-based remote sensing image fusion enhancement model for power facilities was proposed. Firstly， a deep convolutional network was designed， including encoder， Residual Attention （RA） mechanism block， substitution attention mechanism block and decoder. Secondly， the two-layer convolution and the residual block of fusion channel attention mechanism were improved to increase the network's attention to details and key features of images， and enhance the feature extraction capability of the network. Thirdly， the multi-channel substitution attention block was improved to make the network paying more attention to the details of images. As the result， the performance of high-resolution image fusion reconstruction was improved. Finally， the loss function composition of the model was improved， and the composite loss function consisting of content loss and visual loss was adopted to improve training effect of the model. Experimental results indicate that the proposed model has the performance of image fusion reconstruction better than other fusion models significantly， and the detail textures of predicted image closer to those of the real image. Compared with Multi-stage Feature Compensation NET （MFCNET） model， the proposed model has the Correlation Coefficient （CC） improved by 1.6%. and the SSIM （Structure Similarity Index Measure） improved by 18.4%. It can be seen that the proposed model provides a basis for remote sensing image processing， especially for high-resolution reconstruction of small target remote sensing images.

Key words: remote sensing image, deep convolutional network, fusion enhancement, power facility, attention mechanism

摘要：

针对电力设施安全监测及应急管理对于高时空分辨率遥感图像需求，提出一种基于深度卷积网络的电力设施遥感图像融合增强模型。首先，设计包含编码器、残差注意力机制（RA）模块、置换注意力机制模块和解码器的深度卷积网络；其次，改进双层卷积与融合通道注意力机制残差模块，以提高网络对于图像细节及关键特征的关注度，并增强网络的特征提取能力；再次，改进多通道置换注意力模块，使得网络能够更加关注图像细节，从而提升高分辨图像融合重建的性能；最后，改进深度学习网络的损失函数组成，采用由内容损失以及视觉损失组成的复合损失函数，从而提高模型的训练效果。实验结果表明，所提模型的图像融合重建效果明显优于其他融合模型，预测图像在细节纹理上更接近真实图像，与多级特征补偿网络（MFCNET）模型相比，所提模型的重建图像的相关系数（CC）提升了1.6%，结构相似性指数（SSIM）提升了18.4%。可见，所提模型为遥感图像处理，特别是小目标遥感图像高分辨重建提供了基础。

关键词: 遥感图像, 深度卷积网络, 融合增强, 电力设施, 注意力机制

CLC Number:

TP75

Fangrong ZHOU, Yifan WANG, Yi MA, Gang WEN, Guofang WANG, Yutang MA, Hao GENG. Remote sensing image fusion enhancement model for power facilities based on deep convolutional network[J]. Journal of Computer Applications, 0, (): 212-216.

周仿荣, 王一帆, 马仪, 文刚, 王国芳, 马御棠, 耿浩. 基于深度卷积网络的电力设施遥感图像融合增强模型[J]. 《计算机应用》唯一官方网站, 0, (): 212-216.

Figures/Tables 7

Baseline	RA	loss	SA	ERGA	CC	SSIM	PSNR/dB
$√$				6.353	0.742	0.775	31.524
$√$	$√$			4.655	0.749	0.784	31.756
$√$		$√$		5.916	0.743	0.779	32.012
$√$			$√$	5.217	0.751	0.790	32.452
$√$			$√$	5.696	0.746	0.792	32.631
$√$	$√$	$√$	$√$	4.311	0.758	0.798	32.894

Baseline	RA	loss	SA	ERGA	CC	SSIM	PSNR/dB
$√$				6.353	0.742	0.775	31.524
$√$	$√$			4.655	0.749	0.784	31.756
$√$		$√$		5.916	0.743	0.779	32.012
$√$			$√$	5.217	0.751	0.790	32.452
$√$			$√$	5.696	0.746	0.792	32.631
$√$	$√$	$√$	$√$	4.311	0.758	0.798	32.894

模型	ERGA	CC	SSIM
DCSTFN	12.364	0.615	0.584
EDCSTFN	7.125	0.741	0.569
STARFM	23.651	0.465	0.432
MFCNET	5.654	0.746	0.674
本文模型	4.311	0.758	0.798

References 18

1	王权，尤淑撑. 陆地卫星遥感监测体系及应用前景［J］. 测绘学报， 2022， 51（4）：534-543.
2	张睿卓. 基于多源数据的林区电力走廊安全风险评估方法［J］. 测绘学报， 2022， 51（5）：784-784.
3	方明，孙晓敏，黄然，等. 面向高分辨率卫星遥感的电力杆塔自动检测［J］. 航天返回与遥感， 2021， 42（5）：118-126.
4	高振宇，刘亮，潘浩，等. 基于改进语义分割网络的输电走廊遥感变化检测［J］. 航天返回与遥感， 2021， 42（4）：120-128.
5	罗梦，杨知，费香泽，等. 输电通道卫星遥感巡视特征选择与关键技术研究［J］. 电力信息与通信技术， 2021， 19（4）：55-63.
6	秦登达，万里，何佩恩，等. 结合数据融合与特征选择的遥感影像尺度多样目标检测［J］. 遥感学报， 2022， 26（8）：1662-1673.
7	张弢，王艺霖，田喆文. 基于特征增强和筛选机制的遥感电力目标检测［J］. 激光杂志， 2023， 44（3）：87-94.
8	BELGIU M， STEIN A. Spatiotemporal image fusion in remote sensing［J］. Remote Sensing， 2019， 11（7）： No.818.
9	HUANG B， SONG H. Spatiotemporal reflectance fusion via sparse representation［J］. IEEE Transactions on Geoscience and Remote Sensing， 2012， 50（10）： 3707-3716.
10	SONG H， LIU Q， WANG G， et al. Spatiotemporal satellite image fusion using deep convolutional neural networks［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2018， 11（3）： 821-829.
11	TAN Z， YUE P， DI L， et al. Deriving high spatiotemporal remote sensing images using deep convolutional network［J］. Remote Sensing， 2018， 10（7）： No.1066.
12	LIU X， DENG C， CHANUSSOT J， et al. StfNet： a two-stream convolutional neural network for spatiotemporal image fusion ［J］. IEEE Transactions on Geoscience and Remote Sensing， 2019， 57（9）： 6552-6564.
13	SHAO Z， CAl J. Remote sensing image fusion with deep convolutional neural network ［J］. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing， 2018， 11（5）： 1656-1669.
14	刘文杰，李雨珈，白梦浩，等. 基于多级特征补偿的遥感图像时空融合方法［J］. 太赫兹科学与电子信息学报， 2023， 21（7）：939-951.
15	谭振宇. 基于深度卷积网络的MODIS与Landsat影像时空融合研究［D］. 武汉：武汉大学， 2019： 41-72.
16	XU C， HUANG J， WANG S P， et al. A novel parameterized activation function in visual geometry group［C］// Proceedings of the 2nd International Conference on Data Science and Business Analytics. Piscataway： IEEE， 2018： 386-389.
17	ZHANG H， WU C， ZHANG Z， et al. ResNeSt： split-attention networks［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Piscataway： IEEE， 2022： 2735-2745.
18	KANCHARLA P， CHANNAPPAYYA S S. Improving the visual quality of Generative Adversarial Network （GAN）-generated images using the multi-scale structural similarity index ［C］// Proceedings of the 25th IEEE International Conference on Image Processing. Piscataway： IEEE， 2018： 3908-3912.

[1]	Yilin DENG, Fajiang YU. Pseudo random number generator based on LSTM and separable self-attention mechanism [J]. Journal of Computer Applications, 2025, 45(9): 2893-2901.
[2]	Jinggang LYU, Shaorui PENG, Shuo GAO, Jin ZHOU. Speech enhancement network driven by complex frequency attention and multi-scale frequency enhancement [J]. Journal of Computer Applications, 2025, 45(9): 2957-2965.
[3]	Weigang LI, Jiale SHAO, Zhiqiang TIAN. Point cloud classification and segmentation network based on dual attention mechanism and multi-scale fusion [J]. Journal of Computer Applications, 2025, 45(9): 3003-3010.
[4]	Xiang WANG, Zhixiang CHEN, Guojun MAO. Multivariate time series prediction method combining local and global correlation [J]. Journal of Computer Applications, 2025, 45(9): 2806-2816.
[5]	Jin ZHOU, Yuzhi LI, Xu ZHANG, Shuo GAO, Li ZHANG, Jiachuan SHENG. Modulation recognition network for complex electromagnetic environments [J]. Journal of Computer Applications, 2025, 45(8): 2672-2682.
[6]	Haifeng WU, Liqing TAO, Yusheng CHENG. Partial label regression algorithm integrating feature attention and residual connection [J]. Journal of Computer Applications, 2025, 45(8): 2530-2536.
[7]	Chao JING, Yutao QUAN, Yan CHEN. Improved multi-layer perceptron and attention model-based power consumption prediction algorithm [J]. Journal of Computer Applications, 2025, 45(8): 2646-2655.
[8]	Jinhao LIN, Chuan LUO, Tianrui LI, Hongmei CHEN. Thoracic disease classification method based on cross-scale attention network [J]. Journal of Computer Applications, 2025, 45(8): 2712-2719.
[9]	Chen LIANG, Yisen WANG, Qiang WEI, Jiang DU. Source code vulnerability detection method based on Transformer-GCN [J]. Journal of Computer Applications, 2025, 45(7): 2296-2303.
[10]	Yihan WANG, Chong LU, Zhongyuan CHEN. Multimodal sentiment analysis model with cross-modal text information enhancement [J]. Journal of Computer Applications, 2025, 45(7): 2237-2244.
[11]	Haoyu LIU, Pengwei KONG, Yaoli WANG, Qing CHANG. Pedestrian detection algorithm based on multi-view information [J]. Journal of Computer Applications, 2025, 45(7): 2325-2332.
[12]	Xiaoqiang ZHAO, Yongyong LIU, Yongyong HUI, Kai LIU. Batch process quality prediction model using improved time-domain convolutional network with multi-head self-attention mechanism [J]. Journal of Computer Applications, 2025, 45(7): 2245-2252.
[13]	Huibin WANG, Zhan’ao HU, Jie HU, Yuanwei XU, Bo WEN. Time series forecasting model based on segmented attention mechanism [J]. Journal of Computer Applications, 2025, 45(7): 2262-2268.
[14]	Wenjing YAN, Ruidong WANG, Min ZUO, Qingchuan ZHANG. Recipe recommendation model based on hierarchical learning of flavor embedding heterogeneous graph [J]. Journal of Computer Applications, 2025, 45(6): 1869-1878.
[15]	Haijie WANG, Guangxin ZHANG, Hai SHI, Shu CHEN. Document-level relation extraction based on entity representation enhancement [J]. Journal of Computer Applications, 2025, 45(6): 1809-1816.

Remote sensing image fusion enhancement model for power facilities based on deep convolutional network

基于深度卷积网络的电力设施遥感图像融合增强模型

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 18

Related Articles 15

Recommended Articles

Metrics