Abstract: Concerning the problem of convolutional neural network having poor classification performance to time-series remote sensing images with abnormal data, an end-to-end network based on the integration of multi-mode and multi-single-mode architecture was introduced. Firstly, multi-scale features of the multi-dimensional time-series were extracted by the multivariate time-series model and the univariate time-series model. Then, the spatio-temporal sequence feature construction was completed by automatic coding based on the pixel spatial coordinate information. Finally, the classification was implemented by fully connected layer and the softmax function. In the case of data anomaly (data loss and data distortion), the proposed algorithm was compared with commonly used time-series remote sensing image classification algorithms such as 1D Convolutional Neural Network (1D-CNN), Multi-Channels Deep Neural Network (MCDNN), Time Series Convolutional Neural Networks (TSCNN) and Long Short-Term Memory (LSTM) network. Experimental results showed that the proposed network using the end-to-end multi-mode and multi-single-mode architecture fusion had the highest classification accuracy in the case of data anomaly, and the F1 value reached 93.40%.

Key words: remote sensing image, time serials, Convolutional Neural Network (CNN), classification, data anomaly

摘要： 针对时序遥感图像数据异常时卷积神经网络对其分类性能较差的问题，提出了一种端到端的多模式与多单模架构相结合的网络结构。首先，通过多元时序模型和单变量时间序列模型对多维时间序列进行多尺度特征提取；然后，基于像素空间坐标信息，通过自动编码形式完成遥感图像的时空序列特征的构建；最后，通过全连接层和softmax函数实现分类。在数据异常（数据缺失和数据扭曲）的情况下，提出的算法和一维卷积神经网络（1D-CNN）、多通道深度神经网络（MCDNN）、时序卷积神经网络（TSCNN）和长短期记忆（LSTM）网络等通用时间序列遥感影像分类算法进行分析比较。实验结果表明，所提的利用端到端的多模式与多单模式架构融合的网络在数据异常的情况下分类精度最高，F1值达到了93.40%。

关键词: 遥感影像, 时序数据, 卷积神经网络, 分类, 数据异常