Abstract:

The multi-scale features of time series contain abundant category information which has different importance for classification. However， the existing univariate time series classification models conventionally extract series features by convolutions with a fixed kernel size， resulting in being unable to acquire and focus on important multi-scale features effectively. In order to solve the above problem， a Multi-scale Convolution and Attention mechanism （MCA） based Long Short-Term Memory （LSTM） model （MCA-LSTM） was proposed， which was capable of concentrating and fusing important multi-scale features to achieve more accurate classification effect. In this structure， by using LSTM， the transmission of series information was controlled through memory cells and gate mechanism， and the correlation information of time series was extracted fully； by using Multi-scale Convolution Module （MCM）， the multi-scale features of the series were extracted through Convolutional Neural Networks （CNNs） with different kernel sizes； by using Attention Module （AM）， the channel information was fused to obtain the importance of features and assign attention weights， which enabled the network to focus on important time series features. Experimental results on 65 univariate time series datasets of UCR archive show that compared with the state-of-the-art time series classification methods： Unsupervised Scalable Representation Learning-FordA （USRL-FordA）， Unsupervised Scalable Representation Learning-Combined （1-Nearest Neighbor） （USRL-Combined （1-NN））， Omni-Scale Convolutional Neural Network （OS-CNN）， Inception-Time and Robust Temporal Feature Network for time series classification （RTFN），MCA-LSTM has the Mean Error （ME） reduced by 7.48， 9.92， 2.43， 2.09 and 0.82 percentage points， respectively； and achieved the highest Arithmetic Mean Rank （AMR） and Geometric Mean Rank （GMR）， which are 2.14 and 3.23 respectively. These results fully demonstrate the effectiveness of MCA-LSTM in the classification of univariate time series.

Key words: time series classification, deep neural network, Long Short-Term Memory (LSTM) network, multi-scale convolution, attention mechanism

摘要：

时间序列的多尺度特征包含丰富的类别信息，且这些信息对分类具有不同的重要程度，然而现有的单变量时间序列分类模型通常以固定大小的卷积核提取序列特征，导致不能有效地获取并聚焦重要的多尺度特征。针对上述问题，提出一种基于多尺度卷积和注意力机制（MCA）的长短时记忆（LSTM）模型（MCA-LSTM），它能够关注并融合重要的多尺度特征，从而实现更准确的分类。其中，LSTM使用记忆细胞和门机制控制序列信息的传递，并充分提取时间序列的相关性信息；多尺度卷积模块（MCM）使用具有不同卷积核的卷积神经网络（CNN）提取序列的多尺度特征；注意力模块（AM）融合通道信息获取特征的重要性并分配注意力权重，从而使网络关注重要的时间序列特征。在UCR档案的65个单变量时间序列数据集上的实验结果表明，对比当前最先进的基于深度学习的时间序列分类模型：USRL-FordA（Unsupervised Scalable Representation Learning-FordA）、USRL-Combined （1-NN） （Unsupervised Scalable Representation Learning-Combined （1-Nearest Neighbor）） OS-CNN（Omni-Scale Convolutional Neural Network）、Inception-Time和RTFN（Robust Temporal Feature Network for time series classification），MCA-LSTM在平均错误率（ME）上分别降低了7.48、9.92、2.43、2.09和0.82个百分点，并取得了最高的算术平均排名（AMR）和几何平均排名（GMR），分别为2.14和3.23，这些充分体现了MCA-LSTM模型在单变量时间序列分类中的有效性。

关键词: 时间序列分类, 深度神经网络, 长短时记忆网络, 多尺度卷积, 注意力机制