Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2739-2746.DOI: 10.11772/j.issn.1001-9081.2023091320

• Data science and technology • Previous Articles     Next Articles

Multivariate long-term series forecasting method with DFT-based frequency-sensitive dual-branch Transformer

Liehong REN1, Lyuwen HUANG1(), Xu TIAN1, Fei DUAN2   

  1. 1.College of Information Engineering,Northwest A&F University,Xianyang Shaanxi 712100,China
    2.Ningbo Industrial Internet Institute,Ningbo Zhejiang 315000,China
  • Received:2023-09-26 Revised:2023-11-14 Accepted:2023-11-20 Online:2023-12-01 Published:2024-09-10
  • Contact: Lyuwen HUANG
  • About author:REN Liehong, born in 1997, M. S. candidate. His research interests include time series prediction, deep learning.
    TIAN Xu, born in 1998, M. S. candidate. His research interests include embedded system, non-destructive detection of fruit quality.
    DUAN Fei, born in 1978, Ph. D. His research interests include MEMS sensor, AI sensing.
  • Supported by:
    National Key Research and Development Program of China(2020YFD1100601)

基于DFT的频率敏感双分支Transformer多变量长时间序列预测方法

任烈弘1, 黄铝文1(), 田旭1, 段飞2   

  1. 1.西北农林科技大学 信息工程学院,陕西 咸阳 712100
    2.宁波工业互联网研究院,浙江 宁波 315000
  • 通讯作者: 黄铝文
  • 作者简介:任烈弘(1997—),男,山西吕梁人,硕士研究生,CCF会员,主要研究方向:时间序列预测、深度学习
    黄铝文(1976—),男,湖南湘乡人,副教授,博士,CCF会员,主要研究方向:生物图像处理、机器人
    田旭(1998—),男,甘肃天水人,硕士研究生,主要研究方向:嵌入式系统、水果品质无损检测
    段飞(1978—),男,湖南湘乡人,博士,主要研究方向:MEMS传感器、AI感知。
  • 基金资助:
    国家重点研发计划项目(2020YFD1100601)

Abstract:

In multivariate long-term time series forecasting, only relying on time domain analysis often falls to capture long time-series dependencies, leading to insufficient information utilization and not high enough prediction accuracy. To solve these problems, combined with time and frequency domain analyses, a Frequency-Sensitive Dual-branch Transformer with Discrete Fourier Transform (DFT) for multivariate long-term series forecasting (FSDformer) method was proposed. Firstly, by utilizing DFT, the transformation between time and frequency was accomplished, allowing the decomposition of complex time-series data into three structurally simple components: low-frequency trend item, medium-frequency seasonal item, and high-frequency residual item. Then, a dual-branch structure was adopted: one branch dedicated to predict medium- and high-frequency components, with an Encoder-Decoder structure applied to design a periodic enhancement attention mechanism, and another dedicated forecast to low-frequency trend components, with a MultiLayer Perceptron (MLP) structure. Finally, the prediction results from both branches were aggregated to obtain the final multivariate long-term time series forecasting results. FSDformer was compared with five classical algorithms on two datasets. On the Electricity dataset, when the historical sequence length is 96 and the predicted sequence length is 336, compared to the comparison algorithms such as Autoformer, FSDformer decreases the Mean Absolute Error (MAE) by 11.5%-29.1%, and decreases the Mean Square Error (MSE) by 20.9%-43.7%, reaching the optimal prediction accuracy. Experimental results show that, FSDformer can capture the dependencies within long-term time series data efficiently, and can improve the prediction stability of model while enhancing prediction accuracy and computational efficiency.

Key words: Discrete Fourier Transform (DFT), frequency-sensitive, time-series prediction, series decomposition, Transformer, periodic enhancement attention

摘要:

在进行多变量长时间序列预测时,仅利用时域分析通常无法充分捕捉长时间序列依赖,而这会导致信息利用率不足、预测精度不够高。因此,结合频域时域分析,提出一种基于离散傅里叶变换(DFT)的频率敏感双分支多变量长时间序列预测(FSDformer)方法。首先,通过DFT实现时间和频率的相互转换,从而将复杂的时间序列数据分解为结构简单的低频趋势项、中频季节项和高频余项3个分量;其次,采用双分支结构,针对中高频分量预测,应用Encoder-Decoder结构,设计了周期性增强注意力机制;针对低频趋势分量预测,采用多层感知机(MLP)结构;最后将中高频分量与低频分量预测结果相加,得到多变量长时间序列的最终预测结果。在2个数据集上把FSDformer与其他5个经典算法进行了对比分析,在Electricity数据集上,当历史序列长度为96,预测序列长度为336时,相较于Autoformer等对比算法,FSDformer的平均绝对误差(MAE)下降了11.5%~29.1%,均方误差(MSE)下降了20.9%~43.7%,达到了最优预测精度。实验结果表明,FSDformer能有效捕捉长时间序列的相关依赖,在提升预测精度和计算效率的同时,增强了模型预测的稳定性。

关键词: 离散傅里叶变换, 频率敏感, 时间序列预测, 序列分解, Transformer, 周期性增强注意力

CLC Number: