Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (6): 1842-1847.DOI: 10.11772/j.issn.1001-9081.2023060731

Special Issue: 数据科学与技术

• Data science and technology • Previous Articles     Next Articles

Time series classification method based on multi-scale cross-attention fusion in time-frequency domain

Mei WANG1,2, Xuesong SU1, Jia LIU1, Ruonan YIN2(), Shan HUANG1,3   

  1. 1.Shengli Oilfield Technical Testing Center,China Petroleum and Chemical Corporation,Dongying Shandong 257000,China
    2.School of Computer Science and Technology,China University of Petroleum (East China),Qingdao Shandong 266000,China
    3.College of Pipeline and Civil Engineering,China University of Petroleum (East China),Qingdao Shandong 266000,China
  • Received:2023-06-07 Revised:2023-09-08 Accepted:2023-09-14 Online:2023-09-27 Published:2024-06-10
  • Contact: Ruonan YIN
  • About author:WANG Mei, born in 1981, Ph. D. candidate, senior engineer. Her research interests include artificial intelligence, time series analysis and application.
    SU Xuesong, born in 1989, Ph. D. His research interests include oilfield data mining, artificial intelligence.
    LIU Jia, born in 1981, M. S., senior engineer. His research interests include oilfield data mining.
    HUANG Shan, born in 1989, senior engineer. Her research interests include standard informatization in oilfield.
  • Supported by:
    SINOPEC Scientific Research Project(323016);Scientific Research Project of Shengli Oilfield Branch(YG2208)

时频域多尺度交叉注意力融合的时间序列分类方法

王美1,2, 苏雪松1, 刘佳1, 殷若南2(), 黄珊1,3   

  1. 1.中国石油化工股份有限公司 胜利油田技术检测中心, 山东 东营 257000
    2.中国石油大学(华东) 计算机科学与技术学院, 山东 青岛 266000
    3.中国石油大学(华东) 储运与建筑工程学院, 山东 青岛 266000
  • 通讯作者: 殷若南
  • 作者简介:王美(1981—),女,山东威海人,高级工程师,博士研究生,主要研究方向:人工智能、时间序列分析及应用
    苏雪松(1989—),男,山东东营人,博士,主要研究方向:油田数据挖掘、人工智能
    刘佳(1981—),男,山东威海人,高级工程师,硕士,主要研究方向:油田数据挖掘
    黄珊(1989—),女,河北沧州人,高级工程师,主要研究方向:石油领域标准信息化。
  • 基金资助:
    中国石化股份公司科研项目(323016);胜利油田分公司科研项目(YG2208)

Abstract:

To address the problem of low classification accuracy caused by insufficient potential information interaction between time series subsequences, a time series classification method based on multi-scale cross-attention fusion in time-frequency domain called TFFormer (Time-Frequency Transformer) was proposed. First, time and frequency spectrums of the original time series were divided into subsequences with the same length respectively, and the point-value coupling problem was solved by adding positional embedding after linear projection. Then, the long-term time series dependency problem was solved because the model was made to focus on more important time series features by Improved Multi-Head self-Attention (IMHA) mechanism. Finally, a multi-scale Cross-Modality Attention (CMA) module was proposed to enhance the interaction between the time domain and frequency domain, so that the model could further mine the frequency information of the time series. The experimental results show that compared with Fully Convolutional Network (FCN), the classification accuracy of the proposed method on Trace, StarLightCurves and UWaveGestureLibraryAll datasets increased by 0.3, 0.9 and 1.4 percentage points. It is proved that by enhancing the information interaction between time domain and frequency domain of the time series, the model convergence speed and classification accuracy can be improved.

Key words: time series, attention mechanism, positional embedding, deep neural network, multi-scale fusion

摘要:

针对时间序列子序列间的潜在信息交互不足导致分类准确率低的问题,提出时频域多尺度交叉注意力融合的时间序列分类方法TFFormer(Time-Frequency Transformer)。首先,将原始时间序列的时频域谱分别划分为等长子序列,经线性投影后加入位置信息解决时间序列的点值耦合问题;其次,通过改进的多头自注意力(IMHA)模块使模型关注更重要的序列特征,解决长时间序列的前后依赖问题;最后,构造多尺度时频域交叉注意力(CMA)模块增强时间序列在时域和频域之间的信息交互,使模型进一步挖掘序列的频域信息。实验结果表明,在Trace、StarLightCurves和UWaveGestureLibraryAll数据集上,相较于全卷积网络(FCN),所提方法的分类准确率分别提高了0.3、0.9和1.4个百分点,验证了通过增强时间序列时域和频域间的信息交互,可以提高模型收敛速度和分类精度。

关键词: 时间序列, 注意力机制, 位置编码, 深度神经网络, 多尺度融合

CLC Number: