Remote sensing image classification based on sample incremental learning

doi:10.11772/j.issn.1001-9081.2023030366

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (3): 732-736.DOI: 10.11772/j.issn.1001-9081.2023030366

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Remote sensing image classification based on sample incremental learning

Xue LI¹^,², Guangle YAO¹^,²(), Honghui WANG¹^,², Jun LI², Haoran ZHOU³^,⁴, Shaoze YE⁴

^1.State Key Laboratory of Geohazard Prevention and Geoenvironment Protection （Chengdu University of Technology），Chengdu Sichuan 610059，China
^2.College of Computer Science and Cyber Security，Chengdu University of Technology，Chengdu Sichuan 610059，China
^3.Key Laboratory of Natural Disaster Monitoring，Early Warning and Assessment of Jiangxi Province （Jiangxi Normal University），Nanchang Jiangxi 330022，China
^4.Shenzhen Sensing Data Technology Company Limited，Shenzhen Guangdong 518052，China

Received:2023-04-06 Revised:2023-05-15 Accepted:2023-05-18 Online:2023-06-05 Published:2024-03-10
Contact: Guangle YAO
About author:LI Xue， born in 1998， M. S. candidate. Her research interests include incremental learning， remote sensing image intelligent processing.
WANG Honghui， born in 1985， Ph. D.， professor. His research interests include deep learning， geological disaster monitoring and early warning.
LI Jun， born in 1973， Ph. D.， professor. His research interests include big data analysis.
ZHOU Haoran， born in 1997. His research interests include information processing， intelligent mapping.
YE Shaoze， born in 1993， M. S. His research interests include deep learning， intelligent mapping.
Supported by:
Key Research and Development Project of Sichuan Province(2021YFG0298);Basic Research Project of Sichuan Province(2021YJ0086);Geomathematics Key laboratory of Sichuan Province Open Foundation(SCSXDZ2020YB04)

基于样本增量学习的遥感影像分类

李雪¹^,², 姚光乐¹^,²(), 王洪辉¹^,², 李军², 周皓然³^,⁴, 叶绍泽⁴

^1.地质灾害防治与地质环境保护国家重点实验室(成都理工大学), 成都 610059
^2.成都理工大学计算机与网络安全学院, 成都 610059
^3.江西省自然灾害监测预警与评估重点实验室(江西师范大学), 南昌 330022
^4.深圳市森歌数据技术有限公司, 广东深圳 518052

通讯作者: 姚光乐
作者简介:李雪（1998—），女，四川中江人，硕士研究生，主要研究方向：增量学习、遥感图像智能处理
王洪辉（1985—），男，湖北孝感人，教授，博士，主要研究方向：深度学习、地质灾害监测预警
李军（1973—），男，湖北潜江人，教授，博士，主要研究方向：大数据分析
周皓然（1997—），男，广东广州人，主要研究方向：信息处理、智能测绘
叶绍泽（1993—），男，广东深圳人，硕士，主要研究方向：深度学习、智能测绘。
基金资助:
四川省重点研发项目(2021YFG0298);四川省基础研究项目(2021YJ0086);数学地质四川省重点实验室开放基金资助项目(SCSXDZ2020YB04)

Abstract

Abstract:

Deep learning models have achieved remarkable results in remote sensing image classification. With the continuous collection of new remote sensing images， when the remote sensing image classification models based on deep learning train new data to learn new knowledge， their recognition performance of old data will decline， that is， old knowledge forgetting. In order to help remote sensing image classification model consolidate old knowledge and learn new knowledge， a remote sensing image classification model based on sample incremental learning， namely ICLKM （Incremental Collaborative Learning Knowledge Model） was proposed. The model consisted of two knowledge networks. The first network mitigated knowledge forgetting by retaining the output of the old model through knowledge distillation. The second network took the output of new data as the learning objective of the first network and effectively learned new knowledge by maintaining the consistency of the dual network models. Finally， two networks learned together to generate more accurate model through knowledge collaboration strategy. Experimental results on two remote sensing datasets NWPU-RESISC45 and AID show that， ICLKM has the accuracy improved by 3.53 and 6.70 percentage points respectively compared with FT （Fine-Tuning） method. It can be seen that ICLKM can effectively solve the knowledge forgetting problem of remote sensing image classification and continuously improve the recognition accuracy of known remote sensing images.

Key words: remote sensing image classification, incremental learning, Knowledge Distillation (KD), collaborative learning, Fine-Tuning (FT)

摘要：

深度学习模型在遥感影像分类中取得了显著的成绩。随着新的遥感数据不断被采集，基于深度学习的遥感影像分类模型在训练新数据、学习新知识时，对旧数据的识别性能会下降，即旧知识遗忘。为帮助遥感影像分类模型巩固旧知识和学习新知识，提出一种基于样本增量学习的遥感影像分类模型——增量协同学习知识模型（ICLKM）。该模型由两个知识网络组成，第一个网络通过知识蒸馏保留旧模型的输出，缓解知识遗忘问题；第二个网络将新数据的输出作为第一个网络的学习目标，通过维护双网络模型的一致性有效地学习新知识。最后两个网络共同学习，通过知识协同策略生成更精确的模型。在两个遥感数据集NWPU-RESISC45和AID上的实验结果表明，相较于微调训练（FT）方法，ICLKM的准确率分别提升了3.53和6.70个百分点。可见ICLKM能够有效解决遥感影像分类的知识遗忘问题，不断提高对已知遥感影像的识别准确率。

关键词: 遥感影像分类, 增量学习, 知识蒸馏, 协同学习, 微调

CLC Number:

TP391.4

Xue LI, Guangle YAO, Honghui WANG, Jun LI, Haoran ZHOU, Shaoze YE. Remote sensing image classification based on sample incremental learning[J]. Journal of Computer Applications, 2024, 44(3): 732-736.

李雪, 姚光乐, 王洪辉, 李军, 周皓然, 叶绍泽. 基于样本增量学习的遥感影像分类[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 732-736.

Figures/Tables 7

Fig. 1 Basic flow of sample incremental learning for remote sensing images

Fig. 2 Framework of ICLKM

Tab. 1 Dataset information

数据集

样本数

基础训练集

样本数（1组）

增量训练集

样本数（4组）

Tab. 2 Comparison of experimental results on AID and NWPU datasets

数据集	方法	增量学习阶段准确率/%
数据集	方法	$D 0$	$D 1$	$D 2$	$D 3$	$D 4$
AID	Joint	69.10	75.60	78.50	79.00	79.90
	FT	69.10	64.50	66.10	67.60	70.70
	LwF^*	69.10	69.30	71.20	72.10	74.40
	EWC^*	69.10	68.30	68.00	70.00	70.60
	ICLKM	69.10	69.40	72.10	74.90	77.40
NWPU	Joint	65.02	72.31	77.04	80.28	82.17
	FT	65.02	69.15	72.17	74.86	77.00
	LwF^*	65.02	69.93	72.82	74.93	76.91
	EWC^*	65.02	67.88	71.28	74.40	76.11
	ICLKM	65.02	71.62	75.48	78.20	80.53

Tab. 2 Comparison of experimental results on AID and NWPU datasets

数据集	方法	增量学习阶段准确率/%
数据集	方法	$D 0$	$D 1$	$D 2$	$D 3$	$D 4$
AID	Joint	69.10	75.60	78.50	79.00	79.90
	FT	69.10	64.50	66.10	67.60	70.70
	LwF^*	69.10	69.30	71.20	72.10	74.40
	EWC^*	69.10	68.30	68.00	70.00	70.60
	ICLKM	69.10	69.40	72.10	74.90	77.40
NWPU	Joint	65.02	72.31	77.04	80.28	82.17
	FT	65.02	69.15	72.17	74.86	77.00
	LwF^*	65.02	69.93	72.82	74.93	76.91
	EWC^*	65.02	67.88	71.28	74.40	76.11
	ICLKM	65.02	71.62	75.48	78.20	80.53

Fig. 3 Forgetting curve

Fig. 4 Classification results of Mt和Mt'

Tab. 3 Results of ablation experiments

方法	准确率
FT	70.70
FT+KD	74.40
FT+KCON	73.90
ICLKM	77.40

References 16

1	AI B， MA C， ZHAO J， et al. The impact of rapid urban expansion on coastal mangroves： a case study in Guangdong Province， China［J］. Frontiers of Earth Science， 2020， 14： 37-49. 10.1007/s11707-019-0768-6
2	PAN B， XU X， SHI Z， et al. DSSNet： a simple dilated semantic segmentation network for hyperspectral imagery classification ［J］. IEEE Geoscience and Remote Sensing Letters， 2020， 17（11）： 1968-1972. 10.1109/lgrs.2019.2960528
3	LI Y， PENG C， CHEN Y， et al. A deep learning method for change detection in synthetic aperture radar images ［J］. IEEE Transactions on Geoscience and Remote Sensing， 2019， 57（8）： 5751-5763. 10.1109/tgrs.2019.2901945
4	LOWE D G. Distinctive image features from scale-invariant keypoints ［J］. International Journal of Computer Vision， 2004， 60： 91-110. 10.1023/b:visi.0000029664.99615.94
5	YANG Y， NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification ［C］// Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York： ACM， 2010： 270-279. 10.1145/1869790.1869829
6	DAI X， WU X， WANG B， et al. Semisupervised scene classification for remote sensing images： a method based on convolutional neural networks and ensemble learning ［J］. IEEE Geoscience and Remote Sensing Letters， 2019， 16（6）： 869-873. 10.1109/lgrs.2018.2886534
7	许夙晖，慕晓冬，赵鹏，等. 利用多尺度特征与深度网络对遥感影像进行场景分类［J］. 测绘学报， 2016， 45（7）： 834-840. 10.11947/j.AGCS.2016.20150623
	XU S H， MU X D， ZHAO P， et al. Scene classification of remote sensing image based on multi-scale feature and deep neural network［J］. Acta Geodaetica et Cartographica Sinica， 2016， 45（7）： 834-840. 10.11947/j.AGCS.2016.20150623
8	边小勇，费雄君，穆楠. 基于尺度注意力网络的遥感图像场景分类［J］. 计算机应用， 2020， 40（3）： 872-877.
	BIAN X Y， FEI X J， MU N. Remote sensing image scene classification based on scale-attention network ［J］. Journal of Computer Applications， 2020， 40（3）： 872-877.
9	宋中山，梁家锐，郑禄，等. 基于双向门控尺度特征融合的遥感场景分类［J］. 计算机应用， 2021， 41（9）： 2726-2735. 10.11772/j.issn.1001-9081.2020111778
	SONG Z S， LIANG J R， ZHENG L， et al. Remote sensing scene classification based on bidirectional gated scale feature fusion ［J］. Journal of Computer Applications， 2021， 41（9）： 2726-2735. 10.11772/j.issn.1001-9081.2020111778
10	屈震，李堃婷，冯志玺. 基于有效通道注意力的遥感图像场景分类［J］. 计算机应用， 2022， 42（5）： 1431-1439.
	QU Z， LI K T， FENG Z X. Remote sensing image scene classification based on effective channel attention ［J］. Journal of Computer Applications， 2022， 42（5）：1431-1439.
11	HINTON G， VINYALS O， DEAN J. Distilling the knowledge in a neural network ［EB/OL］. ［2023-03-01］. .
12	XIA G-S， HU J， HU F， et al. AID： a benchmark data set for performance evaluation of aerial scene classification ［J］. IEEE Transactions on Geoscience and Remote Sensing， 2017， 55（7）： 3965-3981. 10.1109/tgrs.2017.2685945
13	GONG C， HAN J， LU X. Remote sensing image scene classification： benchmark and state of the art ［J］. Proceedings of the IEEE， 2017， 105（10）： 1865-1883. 10.1109/jproc.2017.2675998
14	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 770-778. 10.1109/cvpr.2016.90
15	LI Z， HOIEM D. Learning without forgetting ［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2018， 40（12）： 2935-2947. 10.1109/tpami.2017.2773081
16	KIRKPATRICK J， PASCANU R， RABINOWITZ N， et al. Overcoming catastrophic forgetting in neural networks ［J］. Proceedings of the National Academy of Sciences of the United States of America， 2017， 114（13）： 3521-3526. 10.1073/pnas.1611835114

[1]	Jianli DING, Hui HUANG, Weidong CAO. Dynamic monitoring method of flight chain operation status [J]. Journal of Computer Applications, 2024, 44(12): 3941-3948.
[2]	Huaiqing HE, Jianqing YAN, Kanghua HUI. Lightweight face recognition method based on deep residual network [J]. Journal of Computer Applications, 2022, 42(7): 2030-2036.
[3]	Le WANG, Meng HAN, Xiaojuan LI, Ni ZHANG, Haodong CHENG. Ensemble classification algorithm based on dynamic weighting function [J]. Journal of Computer Applications, 2022, 42(4): 1137-1147.
[4]	Jing LIU, Zhihong DONG, Zheyu ZHANG, Zhigang SUN, Haipeng JI. Data sharing method of industrial internet of things based on federal incremental learning [J]. Journal of Computer Applications, 2022, 42(4): 1235-1243.
[5]	Chao LIU, Lei WANG, Wen YANG, Qiangqiang ZHONG, Min LI. Incremental attribute reduction method for set-valued decision information system with variable attribute sets [J]. Journal of Computer Applications, 2022, 42(2): 463-468.
[6]	LUO Changyin, CHEN Xuebin, MA Chundi, WANG Junyu. Online federated incremental learning algorithm for blockchain [J]. Journal of Computer Applications, 2021, 41(2): 363-371.
[7]	DENG Wenhan, ZHANG Ming, WANG Lijin, ZHONG Yiwen. Hybrid population-based incremental learning algorithm for solving closed-loop layout problem [J]. Journal of Computer Applications, 2021, 41(1): 95-102.
[8]	LIU Dan, YAO Lishuang, WANG Yunfeng, PEI Zuofei. Classification model for class imbalanced traffic data [J]. Journal of Computer Applications, 2020, 40(8): 2327-2333.
[9]	Keyun LUO, Baoliu YE, Bin TANG, Feng MEI, Wenda LU. Incremental learning based proactive caching mechanism for RocksDB key-value system [J]. Journal of Computer Applications, 2020, 40(2): 321-327.
[10]	YANG Liangdong, YANG Zhixia. Incremental robust non-negative matrix factorization with sparseness constraints and its application [J]. Journal of Computer Applications, 2019, 39(5): 1275-1281.
[11]	FENG Guozheng, XU Jindong, FAN Baode, ZHAO Tianyu, ZHU Meng, SUN Xiao. Remote sensing image classification via semi-supervised fuzzy C-means algorithm [J]. Journal of Computer Applications, 2019, 39(11): 3227-3232.
[12]	CHEN Ping, GONG Xun. Face alignment method based on scale self-adaption and incremental learning [J]. Journal of Computer Applications, 2018, 38(7): 2064-2069.
[13]	HU Lisha, WANG Suzhen, CHEN Yiqiang, HU Chunyu, JIANG Xinlong, CHEN Zhenyu, GAO Xingyu. Objective equilibrium measurement based kernelized incremental learning method for fall detection [J]. Journal of Computer Applications, 2018, 38(4): 928-934.
[14]	LIU Yutong, LI Zhiqing, YANG Xiaoling. Application of improved convolution neural network in remote sensing image classification [J]. Journal of Computer Applications, 2018, 38(4): 949-954.
[15]	WANG Jintao, CAO Yudong, SUN Fuming. Incremental learning algorithm based on graph regularized non-negative matrix factorization with sparseness constraints [J]. Journal of Computer Applications, 2017, 37(4): 1071-1074.

Remote sensing image classification based on sample incremental learning

基于样本增量学习的遥感影像分类

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 7

References 16

Related Articles 15

Recommended Articles

Metrics