Remote sensing scene classification based on bidirectional gated scale feature fusion

doi:10.11772/j.issn.1001-9081.2020111778

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (9): 2726-2735.DOI: 10.11772/j.issn.1001-9081.2020111778

Special Issue: 多媒体计算与计算机仿真

• Multimedia computing and computer simulation • Previous Articles Next Articles

Remote sensing scene classification based on bidirectional gated scale feature fusion

SONG Zhongshan^1,2, LIANG Jiarui^1,2, ZHENG Lu^1,2, LIU Zhenyu³, TIE Jun^1,2

1. College of Computer Science, South-Central University for Nationalities, Wuhan Hubei 430074, China;
2. Hubei Provincial Engineering Research Center for Intelligent Management of Manufacturing Enterprises(South-Central University for Nationalities), Wuhan Hubei 430074, China;
3. College of Resources and Environmental Science, South-Central University for Nationalities, Wuhan Hubei 430074, China

Received:2020-11-13 Revised:2021-02-01 Online:2021-09-10 Published:2021-05-08
Supported by:
This work is partially supported by the Technology Innovation Special Program of Hubei Province (Major Project) (2019ABA101), the Application Basic Frontier Project of Wuhan Science and Technology Plan (2020020601012267), the Graduate Academic Innovation Fund Project of South Central University for Nationalities (3212020sycxjj130).

基于双向门控尺度特征融合的遥感场景分类

宋中山^1,2, 梁家锐^1,2, 郑禄^1,2, 刘振宇³, 帖军^1,2

1. 中南民族大学计算机科学学院, 武汉 430074;
2. 湖北省制造企业智能管理工程技术研究中心(中南民族大学), 武汉 430074;
3. 中南民族大学资源与环境学院, 武汉 430074

通讯作者: 郑禄
作者简介:宋中山(1963-),男,湖北仙桃人,副教授,硕士,主要研究方向:深度学习;梁家锐(1995-),男,广西贵港人,硕士研究生,主要研究方向:计算机视觉;郑禄(1989-),男,内蒙古乌兰察布人,讲师,硕士,CCF会员,主要研究方向:图像处理、模式识别;刘振宇(1983-),男,山东临沂人,讲师,博士,主要研究方向:遥感图像处理;帖军(1976-),男,河南社旗人,教授,博士,CCF会员,主要研究方向:机器感知、模式识别。
基金资助:
湖北省技术创新专项（重大项目）（2019ABA101）；武汉市科技计划应用基础前沿项目（2020020601012267）；中南民族大学研究生学术创新基金资助项目（3212020sycxjj130）。

Abstract

Abstract: There are large differences in shape, texture and color of images in remote sensing image datasets, and the classification accuracy of remote sensing scenes is low due to the scale differences cased by different shooting heights and angles. Therefore, a Feature Aggregation Compensation Convolution Neural Network (FAC-CNN) was proposed, which used active rotation aggregation to fuse features of different scales and improved the complementarity between bottom features and top features through bidirectional gated method. In the network, the image pyramid was used to generate images of different scales and input them into the branch network to extract multi-scale features, and the active rotation aggregation method was proposed to fuse features of different scales, so that the fused features have directional information, which improved the generalization ability of the model to different scale inputs and different rotation inputs, and improved the classification accuracy of the model. On NorthWestern Polytechnical University REmote Sensing Image Scene Classification (NWPU-RESISC) dataset, the accuracy of FAC-CNN was increased by 2.05 percentage points and 2.69 percentage points respectively compared to those of Attention Recurrent Convolutional Network based on VGGNet (ARCNet-VGGNet) and Gated Bidirectional Network (GBNet); and on Aerial Image Dataset (AID), the accuracy of FAC-CNN was increased by 3.24 percentage points and 0.86 percentage points respectively compared to those of the two comparison networks. Experimental results show that FAC-CNN can effectively solve the problems in remote sensing image datasets and improve the accuracy of remote sensing scene classification.

Key words: remote sensing image, scene classification, bidirectional gated method, Convolutional Neural Network (CNN), active rotation aggregation

摘要： 针对遥感影像数据集的图像在形状、纹理和颜色上存在较大差别，以及因拍摄高度和角度不同存在的尺度差异导致遥感场景分类精度不高的问题，提出利用主动旋转聚合来融合不同尺度的特征，并通过双向门控提高底层特征与顶层特征互补性的特征融合补偿卷积神经网络（FAC-CNN）。该网络利用图像金字塔为原始图像生成不同尺度图像后将其输入到分支网络中来提取多尺度特征，并提出主动旋转聚合的方式来融合不同尺度的特征，使融合后的特征具有方向信息，从而提高模型对不同尺度输入以及不同旋转输入的泛化能力，实现模型分类精度的提升。FAC-CNN比基于VGGNet的注意循环卷积网络（ARCNet-VGGNet）和门控双向网络（GBNet）在西北工业大学遥感场景图像分类数据集（NWPU-RESISC）上准确率分别提升了2.05个百分点与2.69个百分点，在航空影像数据集（AID）上准确率分别提升了3.24个百分点与0.86个百分点。实验结果表明，FAC-CNN能有效解决遥感影像数据集存在的问题，提高遥感场景分类的精度。

关键词: 遥感图像, 场景分类, 双向门控方法, 卷积神经网络, 主动旋转聚合

CLC Number:

TP751.1

SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion[J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.

宋中山, 梁家锐, 郑禄, 刘振宇, 帖军. 基于双向门控尺度特征融合的遥感场景分类[J]. 计算机应用, 2021, 41(9): 2726-2735.

References

[1] LI J,BENEDIKTSSON J A,ZHANG B,et al. Spatial technology and social media in remote sensing:a survey[J]. Proceedings of the IEEE,2017,105(10):1855-1864.
[2] LUO F L,HUANG H,DUAN Y L,et al. Local geometric structure feature for dimensionality reduction of hyperspectral imagery[J]. Remote Sensing,2017,9(8):No. 790.
[3] SWAIN M J,BALLARD D H. Color indexing[J]. International Journal of Computer Vision,1991,7(1):11-32.
[4] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision,2004,60(2):91-110.
[5] HOTELLING H. Analysis of a complex of statistical variables into principal components[J]. Journal of Educational Psychology, 1933,24(6):417-441.
[6] OLSHAUSEN B A, FIELD D J. Sparse coding with an overcomplete basis set:a strategy employed by V1?[J]. Vision Research,1997,37(23):3311-3325.
[7] HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science,2006,313(5786):504-507.
[8] KRIZHECSKY A, SUTSKEVER I, HINTON G. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook, NY:Curran Associates Inc.,2012:1097-1105.
[9] SIMONYAN K,ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2015-04-10)[2020-08-22]. https://arxiv.org/pdf/1409.1556.pdf.
[10] SZEGEDY C, LIU W, JIA Y Q, et al. Going deeper with convolutions[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE, 2015:1-9.
[11] HE K M,ZHANG X Y,REN S Q,et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE,2015:770-778.
[12] CHENG G,MA C C,ZHOU P C,et al. Scene classification of high resolution remote sensing images using convolutional neural networks[C]//Proceedings of the 2016 IEEE International Geoscience and Remote Sensing Symposium. Piscataway:IEEE, 2016:767-770.
[13] LIANG Y L,MONTEIRO S T,SABER E S. Transfer learning for high resolution aerial image classification[C]//Proceedings of the 2016 IEEE Applied Imagery Pattern Recognition Workshop. Piscataway:IEEE,2016:1-8.
[14] 王鑫, 李可, 宁晨, 等. 基于深度卷积神经网络和多核学习的遥感图像分类方法[J]. 电子与信息学报,2019,41(5):1098-1105.(WANG X,LI K,NING C,et al. Remote sensing image classification method based on deep convolution neural network and multi-kernel learning[J]. Journal of Electronics and Information Technology,2019,41(5):1098-1105.)
[15] 赵春晖, 马博博. 基于中、高层特征融合的高分辨率遥感图像场景分类[J]. 沈阳大学学报(自然科学版),2020,32(3):224-232.(ZHAO C H,MA B B. Scene classification of high resolution remote sensing images based on middle and high level features fusion[J]. Journal of Shenyang University (Natural Science), 2020,32(3):224-232.)
[16] WANG B Q,LU X Q,ZHENG X T,et al. Semantic descriptions of high-resolution remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters,2019,16(8):1274-1278.
[17] 陈雅琼, 强振平, 陈旭, 等. 利用微调卷积神经网络的土地利用场景分类[J]. 遥感信息,2019,34(3):70-77.(CHEN Y Q, QIANG Z P,CHEN X,et al. Classification of land use scenarios based on fine-tuning convolution neural network[J]. Remote Sensing Information,2019,34(3):70-77.)
[18] YANG S F,RAMANAN D. Multi-scale recognition with DAGCNNs[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway:IEEE,2015:1215-1223.
[19] WANG Q,LIU S T,CHANUSSOT J,et al. Scene classification with recurrent attention of VHR remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing,2019,57(2):1155-1167.
[20] SUN H,LI S Y,ZHENG X T,et al. Remote sensing scene classification by gated bidirectional network[J]. IEEE Transactions on Geoscience and Remote Sensing,2020,58(1):82-96.
[21] LIU W,ANGUELOV D,ERHAN D,et al. SSD:single shot multibox detector[C]//Proceedings of the 2016 European Conference on Computer Vision,LNCS 9905. Cham:Springer, 2016:21-37.
[22] ZENG X Y,OUYANG W L,YAN J J,et al. Crafting GBD-Net for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2018,40(9):2109-2123.
[23] LIN M,CHEN Q,YAN S C. Network in network[EB/OL]. (2014-03-04)[2020-11-06]. https://arxiv.org/pdf/1312.4400.pdf.
[24] LIU W,RABINOVICH A,BERG A C. ParseNet:looking wider to see better[EB/OL]. (2015-11-09)[2020-12-11]. https://arxiv.org/pdf/1506.04579.pdf.
[25] YANG Y, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York:ACM, 2010:270-279.
[26] CHENG G,HAN J W,LU X Q. Remote sensing image scene classification:benchmark and state of the art[J]. Proceedings of the IEEE,2017,105(10):1865-1883.

Remote sensing scene classification based on bidirectional gated scale feature fusion

基于双向门控尺度特征融合的遥感场景分类

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[2]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[3]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[4]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[5]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[6]	SUN Xiao, XU Jindong. Remote sensing image dehazing method based on cascaded generative adversarial network [J]. Journal of Computer Applications, 2021, 41(8): 2440-2444.
[7]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[8]	REN Qiuru, YANG Wenzhong, WANG Chuanjian, WEI Wenyu, QIAN Yunyun. Review of remote sensing image change detection [J]. Journal of Computer Applications, 2021, 41(8): 2294-2305.
[9]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[10]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[11]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[12]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[13]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.
[14]	TAN Daoqiang, ZENG Cheng, QIAO Jinxia, ZHANG Jun. Shadow detection method based on hybrid attention model [J]. Journal of Computer Applications, 2021, 41(7): 2076-2081.
[15]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.