Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2732-2738.DOI: 10.11772/j.issn.1001-9081.2023091301

• Data science and technology • Previous Articles     Next Articles

Multivariate time series prediction model based on decoupled attention mechanism

Liting LI1, Bei HUA1(), Ruozhou HE1, Kuang XU2   

  1. 1.School of Computer Science and Technology,University of Science and Technology of China,Hefei Anhui 230027,China
    2.USTC Sinovate Software Company Limited,Hefei Anhui 230088,China
  • Received:2023-09-20 Revised:2023-12-05 Accepted:2023-12-11 Online:2024-02-07 Published:2024-09-10
  • Contact: Bei HUA
  • About author:LI Liting, born in 1999, M. S. candidate. His research interests include deep neural network, data mining, time series prediction.
    HE Ruozhou, born in 2000, M. S. candidate. His research interests include deep neural network, intelligent transportation system, spatio-temporal prediction.
    XU Kuang, born in 1982, M. S. His research interests include intelligent network orchestration and scheduling.
  • Supported by:
    National Key Research and Development Program of China(2018AAA0101200)

基于解耦注意力机制的多变量时序预测模型

李力铤1, 华蓓1(), 贺若舟1, 徐况2   

  1. 1.中国科学技术大学 计算机科学与技术学院,合肥 230027
    2.科大国创软件股份有限公司,合肥 230088
  • 通讯作者: 华蓓
  • 作者简介:李力铤(1999—),男,浙江宁波人,硕士研究生,主要研究方向:深度神经网络、数据挖掘、时序预测
    华蓓(1966—),女,江苏无锡人,教授,博士,CCF会员,主要研究方向:高性能计算、网络系统、时序预测
    贺若舟(2000—),男,广东惠州人,硕士研究生,主要研究方向:深度神经网络、智能交通系统、时空预测
    徐况(1982—),男,安徽蚌埠人,硕士,主要研究方向:智能网络编排与调度。
  • 基金资助:
    国家重点研发计划(2018AAA0101200)

Abstract:

Aiming at the problem that it is difficult to fully utilize the sequence contextual semantic information and the implicit correlation information among variables in multivariate time-series prediction, a model based on decoupled attention mechanism — Decformer was proposed for multivariate time-series prediction. Firstly, a novel decoupled attention mechanism was proposed to fully utilize the embedded semantic information, thereby improving the accuracy of attention weight allocation. Secondly, a pattern correlation mining method without relying on explicit variable relationships was proposed to mine and utilize implicit pattern correlation information among variables. On three different types of real datasets (TTV, ECL and PeMS-Bay), including traffic volume of call, electricity consumption and traffic, Decformer achieves the highest prediction accuracy over all prediction time lengths compared with excellent open-source multivariate time-series prediction models such as Long- and Short-term Time-series Network (LSTNet), Transformer and FEDformer. Compared with LSTNet, Decformer has the Mean Absolute Error (MAE) reduced by 17.73%-27.32%, 10.89%-17.01%, and 13.03%-19.64% on TTV, ECL and PeMS-Bay datasets, respectively, and the Mean Squared Error (MSE) reduced by 23.53%-58.96%, 16.36%-23.56% and 15.91%-26.30% on TTV, ECL and PeMS-Bay datasets, respectively. Experimental results indicate that Decformer can enhance the accuracy of multivariate time series prediction significantly.

Key words: multivariate time series prediction, self-attention mechanism, pattern correlation, temporal correlation, embedding mechanism

摘要:

针对多变量时序预测难以充分利用序列上下文语义信息及变量间隐含关联信息的问题,提出一种基于解耦注意力机制的多变量时序预测模型Decformer。首先,提出一种解耦注意力机制,从而充分利用嵌入的语义信息提升注意力权值分配的准确度;其次,提出一种不依赖于显式变量关系的模式关联挖掘方法,以挖掘并利用变量间隐含的模式关联信息。在话务量、电力消耗和交通3种不同类型的真实数据集(TTV、ECL和PeMS-Bay)上,与长短期时间序列网络(LSTNet)、Transformer、FEDformer等优秀的开源多变量时序预测模型相比,Decformer在所有预测时间长度上都取得了最高的预测精度。相较于LSTNet,Decformer在TTV、ECL和PeMS-Bay数据集上的平均绝对误差(MAE)分别降低了17.73%~27.32%、10.89%~17.01%和13.03%~19.64%;均方误差(MSE)分别降低了23.53%~58.96%、16.36%~23.56%和15.91%~26.30%。实验结果表明,Decformer能够有效提升多变量时序预测的精度。

关键词: 多变量时序预测, 自注意力机制, 模式关联, 时间关联, 嵌入机制

CLC Number: