Hyperspectral image classification method based on spatial-spectral fusion network

doi:10.11772/j.issn.1001-9081.2017122905

Journal of Computer Applications ›› 2018, Vol. 38 ›› Issue (7): 1888-1892.DOI: 10.11772/j.issn.1001-9081.2017122905

Previous Articles Next Articles

Hyperspectral image classification method based on spatial-spectral fusion network

OUYANG Ning^1,2, ZHU Ting², LIN Leping^1,2

1. Key Laboratory of Cognitive Radio and Information Processing, Ministry of Education(Guilin University of Electronic Technology), Guilin Guangxi 541004, China;
2. School of Information and Communication, Guilin University of Electronic Technology, Guilin Guangxi 541004, China

Received:2017-12-12 Revised:2018-03-01 Online:2018-07-10 Published:2018-07-12
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61661017,61362021), the Natural Science Foundation of Guangxi (2017GXNSFBA198212, 2014GXNSFDA118035, 2016GXNSFAA38014), the Scientific and Technological Innovation Ability and Condition Construction Plan of Guangxi (GUIKENENG 1598025-21), the Scientific and Technological Development Program of Guilin (20150103-6), the China Postdoctoral Science Foundation (2016M602923XB), the Innovation Project for Graduate Education in Guilin University of Electronic Technology (2016YJCXB02), the Fund of the Key Laboratory of Cognitive Radio and Information Processing of Ministry of Education (CRKL160104, CRKL150103, 2011KF11).

基于空谱融合网络的高光谱图像分类方法

欧阳宁^1,2, 朱婷², 林乐平^1,2

1. 认知无线电与信息处理省部共建教育部重点实验室(桂林电子科技大学), 广西桂林 541004;
2. 桂林电子科技大学信息与通信学院, 广西桂林 541004

通讯作者: 林乐平
作者简介:欧阳宁(1972-),男,湖南宁远人,教授,硕士,主要研究方向:数字图像处理、智能信息处理;朱婷(1991-),女,江苏宿迁人,硕士研究生,主要研究方向:模式识别、深度学习;林乐平(1980-),女,广西桂平人,副教授,博士,主要研究方向:机器学习、智能信息处理、图像处理。
基金资助:
国家自然科学基金资助项目（61661017，61362021）；广西自然科学基金资助项目（2017GXNSFBA198212，2014GXNSFDA118035，2016GXNSFAA38014）；广西科技创新能力与条件建设计划项目（桂科能1598025-21）；桂林科技开发项目（20150103-6）；中国博士后科学基金资助项目（2016M602923XB）；桂林电子科技大学研究生教育创新计划资助项目（2016YJCXB02）；认知无线电与信息处理重点实验室基金资助项目（CRKL160104，CRKL150103，2011KF11）。

Abstract

Abstract: Concerning the problem that the extracted spatial-spectral features have the shortcoming of weak representation ability and high dimensionality, a HyperSpectral Image (HSI) classification method based on Spatial-Spectral Fusion Network (SSF-Net) was proposed. Firstly, a Two-channel Convolutional Neural Network (Two-CNN) was used to extract the features of spectral domain and spatial domain for HSI respectively. Secondly, Multimodal Compact Bilinear pooling (MCB) was employed to project the outer product of extracted multimodal features to the low dimensional space for producing jointly spatial-spectral features. The fusion network could not only analyze the complex relationship between elements in spectral and spatial eigenvectors, but also avoid directly computing the outer product of spectral and spatial vectors, resulting in high dimension and high computation times. The experimental results show that, compared with the state-of-the-art neural network based classification methods, the proposed algorithm can obtain higher pixel classification accuracy. It indicates that the spatial-spectral joint vector extracted by the proposed network has stronger representation ability.

Key words: Spatial-Spectral Fusion Network (SSF-Net), multimodal compact bilinear pooling, feature fusion, outer product, HyperSpectral Image (HSI) classification

摘要： 针对高光谱图像分类中提取的空-谱特征表达能力弱及维数较高的问题，提出一种基于空-谱融合网络（SSF-Net）的高光谱图像分类方法。首先，利用双通道卷积神经网络（Two-CNN）同时提取高光谱图像的光谱和空间特征；其次，使用多模态压缩双线性池化（MCB）将所提取的多模态特征向量的外积投射到低维空间，以此产生空-谱联合特征。该特征融合网络，既可以分析光谱特征和空间特征向量中元素之间的复杂关系，同时也避免对光谱和空间向量直接进行外积计算，造成维数过高、计算困难的问题。最终实验表明，与现有基于神经网络的分类方法相比，所提出的高光谱图像分类算法能够获得更高的像元分类精度，表明该网络所提取的空-谱联合向量对高光谱图像具有更强的特征表达能力。

关键词: 空-谱融合网络, 多模态压缩双线性池化, 特征融合, 外积, 高光谱图像分类

CLC Number:

TP753
TP18

OUYANG Ning, ZHU Ting, LIN Leping. Hyperspectral image classification method based on spatial-spectral fusion network[J]. Journal of Computer Applications, 2018, 38(7): 1888-1892.

欧阳宁, 朱婷, 林乐平. 基于空谱融合网络的高光谱图像分类方法[J]. 计算机应用, 2018, 38(7): 1888-1892.

References

[1] 童庆禧,张兵,张立福.中国高光谱遥感的前沿进展[J].遥感学报,2016,20(5):689-707.(TONG Q X, ZHANG B, ZHANG L F. Current progress of hyperspectral remote sensing in China[J]. Journal of Remote Sensing, 2016, 20(5):689-707.)
[2] LI J, MARPU P R, PLAZA A, et al. Generalized composite kernel framework for hyperspectral image classification[J]. IEEE Transactions on Geoscience & Remote Sensing, 2013, 51(9):4816-4829.
[3] NASRABADI N M. Hyperspectral target detection:an overview of current and future challenges[J]. IEEE Signal Processing Magazine, 2013, 31(1):34-44.
[4] LI W, DU Q, ZHANG B. Combined sparse and collaborative representation for hyperspectral target detection[J]. Pattern Recognition, 2015, 48(12):3904-3916.
[5] SAMANIEGO L, BÁRDOSSY A, SCHULZ K. Supervised classification of remotely sensed imagery using a modified k-NN technique[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(1):2112-2125.
[6] CRAWFORD M M, MA L, KIM W. Exploring nonlinear manifold learning for classification of hyperspectral data[M]//Optical Remote Sensing. Berlin:Springer, 2011:207-234.
[7] ZHANG L, ZHANG L, TAO D, et al. On combining multiple features for hyperspectral remote sensing image classification[J]. IEEE Transactions on Geoscience & Remote Sensing, 2012, 50(3):879-893.
[8] ZHANG L, ZHANG L, DU B. Deep learning for remote sensing data:a technical tutorial on the state of the art[J]. IEEE Geoscience & Remote Sensing Magazine, 2016, 4(2):22-40.
[9] ZHAO W, DU S. Spectral-spatial feature extraction for hyperspectral image classification:a dimension reduction and deep learning approach[J]. IEEE Transactions on Geoscience & Remote Sensing, 2016, 54(8):4544-4554.
[10] YUE J, ZHAO W, MAO S, et al. Spectral-spatial classification of hyperspectral images using deep convolutional neural networks[J]. Remote Sensing Letters, 2015, 6(6):468-477.
[11] MAKANTASIS K, KARANTZALOS K, DOULAMIS A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]//Proceedings of the 2015 Geoscience and Remote Sensing Symposium. Piscataway, NJ:IEEE, 2015:4959-4962.
[12] CHEN Y, JIANG H, LI C, et al. Deep feature extraction and classification of hyperspectral images based on convolutional neural networks[J]. IEEE Transactions on Geoscience & Remote Sensing, 2016, 54(10):6232-6251.
[13] LIU Q, ZHOU F, HANG R, et al. Bidirectional-convolutional LSTM based spectral-spatial feature learning for hyperspectral image classification[J]. Remote Sensing, 2017, 9(12):1330.
[14] YANG J, ZHAO Y, CHAN C W, et al. Hyperspectral image classification using two-channel deep convolutional neural network[C]//Proceedings of the 2016 Geoscience and Remote Sensing Symposium. Piscataway, NJ:IEEE, 2016:5079-5082.
[15] ZHANG H, LI Y, ZHANG Y, et al. Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network[J]. Remote Sensing Letters, 2017, 8(5):438-447.
[16] LIN T Y, ROYCHOWDHURY A, MAJI S. Bilinear CNN models for fine-grained visual recognition[C]//ICCV 2015:Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway, NJ:IEEE, 2015:1449-1457.
[17] GAO Y, BEIJBOM O, ZHANG N, et al. Compact bilinear pooling[C]//Proceedings of the 2016 Computer Vision and Pattern Recognition. Washington, DC:IEEE Computer Society, 2016:317-326.
[18] TENENBAUM J B, FREEMAN W T. Separating style and content with bilinear models[J]. Neural Computation, 2014, 12(6):1247-1283.
[19] PHAM N, PAGH R. Fast and scalable polynomial kernels via explicit feature maps[C]//Proceedings of the 2013 ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2013:239-247.
[20] JIA Y, SHELHAMER E, DONAHUE J, et al. Caffe:convolutional architecture for fast feature embedding[C]//MM'14:Proceedings of the 22nd ACM International Conference on Multimedia. New York:ACM, 2014:675-678.

Hyperspectral image classification method based on spatial-spectral fusion network

基于空谱融合网络的高光谱图像分类方法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	WANG Wei, ZHAO Erping, CUI Zhiyuan, SUN Hao. Disambiguation method of multi-feature fusion based on HowNet sememe and Word2vec word embedding representation [J]. Journal of Computer Applications, 2021, 41(8): 2193-2198.
[2]	ZHOU Xianbing, FAN Xiaochao, REN Ge, YANG Yong. Automated English essay scoring method based on multi-level semantic features [J]. Journal of Computer Applications, 2021, 41(8): 2205-2211.
[3]	WU Lidan, XUE Yuyang, TONG Tong, DU Min, GAO Qinquan. Image colorization algorithm based on foreground semantic information [J]. Journal of Computer Applications, 2021, 41(7): 2048-2053.
[4]	DU Yan, LYU Liangfu, JIAO Yichen. Fuzzy prototype network based on fuzzy reasoning [J]. Journal of Computer Applications, 2021, 41(7): 1885-1890.
[5]	ZHANG Sun, YIN Chunyong. Sequential multimodal sentiment analysis model based on multi-task learning [J]. Journal of Computer Applications, 2021, 41(6): 1631-1639.
[6]	LAI Xuemei, TANG Hong, CHEN Hongyu, LI Shanshan. Multimodal sentiment analysis based on feature fusion of attention mechanism-bidirectional gated recurrent unit [J]. Journal of Computer Applications, 2021, 41(5): 1268-1274.
[7]	BIAN Pengcheng, ZHENG Zhonglong, LI Minglu, HE Yiran, WANG Tianxiang, ZHANG Dawei, CHEN Liyuan. Attention fusion network based video super-resolution reconstruction [J]. Journal of Computer Applications, 2021, 41(4): 1012-1019.
[8]	HU Yishan, QIN Pinle, ZENG Jianchao, CHAI Rui, WANG Lifang. Ultrasound thyroid segmentation network based on feature fusion and dynamic multi-scale dilated convolution [J]. Journal of Computer Applications, 2021, 41(3): 891-897.
[9]	JIANG Qianyu, WANG Fengying, JIA Lipeng. Malware detection method based on perceptual hash algorithm and feature fusion [J]. Journal of Computer Applications, 2021, 41(3): 780-785.
[10]	HOU Yunlong, ZHU Lei, CHEN Qin, LYU Suidong. Salient object detection based on difference of Gaussian feature network [J]. Journal of Computer Applications, 2021, 41(3): 706-713.
[11]	LIU Ziyan, ZHU Mingcheng, YUAN Lei, MA Shanshan, CHEN Lingzhouting. Video person re-identification based on non-local attention and multi-feature fusion [J]. Journal of Computer Applications, 2021, 41(2): 530-536.
[12]	CHANG Zheng, LUO Ping, YANG Bo, ZHANG Xiaoxiao. Respiratory sound recognition of chronic obstructive pulmonary disease patients based on HHT-MFCC and short-term energy [J]. Journal of Computer Applications, 2021, 41(2): 598-603.
[13]	MENG Xiangrui, YANG Wenzhong, WANG Ting. Survey of sentiment analysis based on image and text fusion [J]. Journal of Computer Applications, 2021, 41(2): 307-317.
[14]	DU Peide, YAN Hua. Crowd counting network based on multi-scale spatial attention feature fusion [J]. Journal of Computer Applications, 2021, 41(2): 537-543.
[15]	QIU Ningjia, WANG Xiaoxia, WANG Peng, WANG Yanchun. Analysis of double-channel Chinese sentiment model integrating grammar rules [J]. Journal of Computer Applications, 2021, 41(2): 318-323.