Multivariate long-term series forecasting model based on decomposition and frequency domain feature extraction

doi:10.11772/j.issn.1001-9081.2023111684

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (11): 3442-3448.DOI: 10.11772/j.issn.1001-9081.2023111684

• Data science and technology • Previous Articles Next Articles

Multivariate long-term series forecasting model based on decomposition and frequency domain feature extraction

Yiyang FAN¹^,², Yang ZHANG¹^,²^,³(), Shang ZENG¹^,², Yu ZENG¹^,², Maoli FU¹^,²^,³

^1.Chengdu Institute of Computer Application，Chinese Academy of Sciences，Chengdu Sichuan 610213，China
^2.School of Computer Science and Technology，University of Chinese Academy of Sciences，Beijing 100049，China
^3.Shenzhen CBPM?KEXIN Banking Technology Company Limited，Shenzhen Guangdong 518206，China

Received:2023-12-08 Revised:2024-03-08 Accepted:2024-03-12 Online:2024-03-22 Published:2024-11-10
Contact: Yang ZHANG
About author:FAN Yiyang， born in 1998， M. S. candidate. His research interests include time series analysis， data mining.
ZENG Shang， born in 1995， Ph. D. candidate. His research interests include big data analysis， data mining.
ZENG Yu， born in 1999， M. S. candidate. His research interests include time series analysis， data mining.
FU Maoli， born in 1988， Ph. D. candidate， engineer. His research interests include artificial intelligence， image processing， pattern recognition.
Supported by:
Sichuan Province Science Program(2023YFG0113)

基于分解和频域特征提取的多变量长时间序列预测模型

范艺扬¹^,², 张洋¹^,²^,³(), 曾尚¹^,², 曾渝¹^,², 付茂栗¹^,²^,³

^1.中国科学院成都计算机应用研究所，成都 610213
^2.中国科学院大学计算机科学与技术学院，北京 100049
^3.深圳市中钞科信金融科技有限公司，广东深圳 518206

通讯作者: 张洋
作者简介:范艺扬（1998—），男，四川成都人，硕士研究生，主要研究方向：时间序列分析、数据挖掘
曾尚（1995—），男，湖北荆门人，博士研究生，主要研究方向：大数据分析、数据挖掘
曾渝（1999—），男，重庆人，硕士研究生，主要研究方向：时间序列分析、数据挖掘
付茂栗（1988—），男，四川遂宁人，工程师，博士研究生，主要研究方向：人工智能、图像处理、模式识别。
基金资助:
四川省科技计划项目(2023YFG0113)

Abstract

Abstract:

In response to the problems that the existing Transformer-based Multivariate Long-Term Series Forecasting （MLTSF） models mainly extract features from the time domain， and it is difficult to find out reliable dependencies directly from the dispersed time points of the long-term series， a new decomposition and frequency domain feature extraction model was proposed. Firstly， a periodic term-trend term decomposition method based on the frequency domain was proposed， which reduced the time complexity of the decomposition process. Then， based on the extraction of trend features using periodic term-trend term decomposition， a Transformer network performing frequency domain feature extraction based on Gabor transform was utilized to capture periodic dependencies， which enhanced the stability and robustness of forecasting. Experimental results on five benchmark datasets show that compared with the current state-of-the-art methods， the proposed model has the Mean Squared Error （MSE） in MLTSF is reduced by an average of 7.6% with a maximum reduction of 18.9%， which demonstrates that the proposed model improves forecasting accuracy effectively.

Key words: multivariate long-term series forecasting, frequency domain feature extraction, Gabor transform, Transformer, time series, deep learning

摘要：

针对现有基于Transformer的多变量长时间序列预测（MLTSF）模型主要从时域中提取特征，难以直接从长时间序列分散的时间点中找出可靠依赖关系的问题，提出一种新的基于分解和频域特征提取的模型。首先，提出基于频域的周期项-趋势项的分解方法，以降低分解过程的时间复杂度；其次，在利用周期项-趋势项分解提取序列趋势性特征的基础上，利用基于Gabor变换进行频域特征提取的Transformer网络捕捉周期性的依赖，提高预测的稳定性和鲁棒性。在5个基准数据集上的实验结果显示，与现有的先进方法相比，所提模型在MLTSF上的均方误差（MSE）平均减小了7.6%，最多减小了18.9%，有效提升了预测精度。

关键词: 多变量长时间序列预测, 频域特征提取, Gabor变换, Transformer, 时间序列, 深度学习

CLC Number:

TP183

Yiyang FAN, Yang ZHANG, Shang ZENG, Yu ZENG, Maoli FU. Multivariate long-term series forecasting model based on decomposition and frequency domain feature extraction[J]. Journal of Computer Applications, 2024, 44(11): 3442-3448.

范艺扬, 张洋, 曾尚, 曾渝, 付茂栗. 基于分解和频域特征提取的多变量长时间序列预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3442-3448.

Figures/Tables 9

References 24

1	ZHOU H， ZHANG S， PENG J， et al. Informer： beyond efficient Transformer for long sequence time-series forecasting［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2021： 11106-11115.
2	郭建平.农业气象灾害监测预测技术研究进展［J］.应用气象学报，2016，27（5）：620-630.
	GUO J P. Research progress on agricultural meteorological disaster monitoring and forecasting［J］. Journal of Applied Meteorological Science， 2016， 27（5）： 620-630.
3	YIN Y， SHANG P. Forecasting traffic time series with multivariate predicting method［J］. Applied Mathematics and Computation， 2016， 291： 266-278.
4	WU H， HU T， LIU Y， et al. TimesNet： temporal 2D-variation modeling for general time series analysis［EB/OL］. （2023-04-12）［2023-10-19］..
5	ZENG A， CHEN M， ZHANG L， et al. Are Transformers effective for time series forecasting？［C］// Proceedings of the 37th AAAI Conference on Artificial Intelligence. Palo Alto， CA ： AAAI Press， 2023：11121-11128.
6	ZHOU T， MA Z， WEN Q， et al. FEDformer： frequency enhanced decomposed Transformer for long-term series forecasting［C］// Proceedings of the 39th International Conference on Machine Learning. New York： JMLR.org， 2022： 27268-27286.
7	JONES D L， PARKS T W. Time-frequency window leakage in the short-time Fourier transform［J］. Circuits， Systems and Signal Processing， 1987， 6（3）： 263-286.
8	GOTTLIEB D， SHU C W. On the Gibbs phenomenon and its resolution［J］. SIAM Review， 1997， 39（4）： 644-668.
9	WANG H， PENG J， HUANG F， et al. MICN： multi-scale local and global context modeling for long-term series forecasting［EB/OL］. ［2023-12-22］..
10	GU G F， ZHOU W X. Detrending moving average algorithm for multifractals［J］. Physical Review. E， Statistical， Nonlinear， and Soft Matter Physics， 2010， 82（1）： No.011136.
11	HUNTER J S. The exponentially weighted moving average［J］. Journal of Quality Technology， 1986， 18（4）： 203-210.
12	SHUMWAY R H， STOFFER D S. ARIMA models［M］// Time series Analysis and Its Applications： With R Examples. Cham： Springer， 2017： 75-163.
13	WEST M. Time series decomposition［J］. Biometrika， 1997， 84（2）： 489-494.
14	QIAN S， CHEN D. Discrete Gabor transform［J］. IEEE Transactions on Signal Processing， 1993， 41（7）： 2429-2438.
15	BRACEWELL R， KAHN P B. The Fourier transform and its applications［J］. American Journal of Physics， 1966， 34（8）： 712-712.
16	SELESNICK I W， BURRUS C S. Generalized digital Butterworth filter design［J］. IEEE Transactions on Signal Processing， 1998， 46（6）： 1688-1694.
17	YOUNG I T， VAN VLIET L J. Recursive implementation of the Gaussian filter［J］. Signal Processing， 1995， 44（2）： 139-151.
18	WU H， XU J， WANG J， et al. Autoformer： decomposition Transformers with auto-correlation for long-term series forecasting［C］// Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2021： 22419-22430.
19	TRINDADE A. ElectricityLoadDiagrams20112014 in UCI Machine Learning Repository［DS/OL］. ［2023-10-19］..
20	LAI G， CHANG W C， YANG Y， et al. Modeling long- and short-term temporal patterns with deep neural networks［C］// Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. New York： ACM， 2018： 95-104.
21	CHOLLET F. Deep learning with Python［M］. 2nd ed. Shelter Island， NY： Manning Publications， 2021： 45-52
22	KITAEV N， KAISER Ł， LEVSKAYA A. Reformer： the efficient Transformer［EB/OL］. ［2023-11-08］..
23	NIE X， ZHOU X， LI Z， et al. LogTrans： providing efficient local-global fusion with Transformer and CNN parallel network for biomedical image segmentation［C］// Proceedings of the 24th IEEE International Conference on High Performance Computing and Communications/ 8th IEEE International Conference on Data Science and Systems/ 20th IEEE International Conference on Smart City/ 8th IEEE International Conference on Dependability in Sensor， Cloud and Big Data Systems and Application. Piscataway： IEEE， 2022： 769-776.
24	HEWAGE P， BEHERA A， TROVATI M， et al. Temporal Convolutional Neural （TCN） network for an effective weather forecasting using time-series data from the local weather station［J］. Soft Computing， 2020， 24（21）： 16453-16482.

数据集	长度	维度	采集频率/min
ETT1/ETT2	69 680	8	15 / 60
Electricity	26 304	322	60
Exchange	7 588	9	1 440
Weather	52 696	22	10
Illness	966	8	10 080

数据集	长度	维度	采集频率/min
ETT1/ETT2	69 680	8	15 / 60
Electricity	26 304	322	60
Exchange	7 588	9	1 440
Weather	52 696	22	10
Illness	966	8	10 080

数据集	预测长度	本文模型		FEDformer		Autoformer		Informer		LogTrans		Reformer		LSTNet		LSTM		TCN
数据集	预测长度	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm2	96	0.188	0.281	0.203	0.287	0.255	0.339	0.365	0.453	0.768	0.642	0.658	0.619	3.142	1.365	2.041	1.073	3.041	1.330
	192	0.252	0.322	0.269	0.328	0.281	0.340	0.533	0.563	0.989	0.757	1.078	0.827	1.154	1.369	2.249	1.112	3.072	1.339
	336	0.315	0.363	0.325	0.366	0.339	0.372	1.363	0.887	1.334	0.872	1.549	0.972	3.160	1.369	2.568	1.238	3.105	1.348
	720	0.411	0.410	0.421	0.415	0.422	0.419	3.379	1.388	3.048	1.328	2.631	1.242	3.171	1.368	2.720	1.278	3.135	1.354
Electricity	96	0.189	0.304	0.193	0.308	0.201	0.317	0.274	0.368	0.258	0.357	0.312	0.402	0.680	0.645	0.375	0.437	0.985	0.813
	192	0.197	0.313	0.201	0.315	0.222	0.334	0.296	0.386	0.266	0.368	0.348	0.433	0.725	0.676	0.442	0.473	0.996	0.821
	336	0.207	0.324	0.214	0.329	0.231	0.338	0.300	0.394	0.280	0.380	0.350	0.433	0.828	0.727	0.439	0.473	1.000	0.824
	720	0.238	0.349	0.246	0.355	0.254	0.361	0.373	0.439	0.283	0.376	0.340	0.420	0.957	0.811	0.980	0.814	1.438	0.784
Exchange	96	0.120	0.250	0.148	0.278	0.197	0.323	0.847	0.752	0.968	0.812	1.065	0.829	1.551	1.058	1.453	1.049	3.004	1.432
	192	0.226	0.344	0.271	0.380	0.300	0.369	1.204	0.895	1.040	0.851	1.188	0.906	1.477	1.028	1.846	1.179	3.048	1.444
	336	0.384	0.452	0.460	0.500	0.509	0.524	1.672	1.036	1.659	1.081	1.350	0.976	1.507	1.031	2.136	1.231	3.113	1.459
	720	1.139	0.813	1.195	0.841	1.447	0.941	2.478	1.310	1.941	1.127	1.510	1.016	2.285	1.243	2.984	1.427	3.150	1.458
Weather	96	0.212	0.294	0.217	0.296	0.266	0.336	0.300	0.384	0.458	0.490	0.689	0.596	0.594	0.587	0.369	0.406	0.615	0.589
	192	0.261	0.329	0.276	0.336	0.307	0.367	0.598	0.544	0.658	0.589	0.752	0.638	0.560	0.565	0.416	0.435	0.629	0.600
	336	0.312	0.364	0.339	0.380	0.359	0.395	0.578	0.523	0.797	0.652	0.639	0.596	0.597	0.587	0.455	0.454	0.639	0.608
	720	0.399	0.413	0.403	0.428	0.419	0.428	1.059	0.741	0.869	0.675	1.130	0.782	0.618	0.599	0.535	0.520	0.639	0.610
Illness	24	2.623	1.060	3.228	1.260	3.483	1.287	5.764	1.677	4.480	1.444	4.000	1.382	6.026	1.770	5.914	1.734	6.624	1.830
	36	2.399	0.981	2.679	1.080	3.103	1.148	4.755	1.467	4.799	1.467	4.783	1.448	5.340	1.668	6.631	1.845	6.858	1.879
	48	2.367	0.980	2.622	1.078	2.669	1.085	4.763	1.469	4.800	1.468	4.832	1.465	6.080	1.787	6.736	1.857	6.968	1.892
	60	2.575	1.056	2.857	1.157	2.770	1.125	5.264	1.564	5.278	1.560	4.882	1.483	5.548	1.720	6.870	1.879	7.127	1.918

数据集	预测长度	本文模型		FEDformer		Autoformer		Informer		LogTrans		Reformer		LSTNet		LSTM		TCN
数据集	预测长度	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm2	96	0.188	0.281	0.203	0.287	0.255	0.339	0.365	0.453	0.768	0.642	0.658	0.619	3.142	1.365	2.041	1.073	3.041	1.330
	192	0.252	0.322	0.269	0.328	0.281	0.340	0.533	0.563	0.989	0.757	1.078	0.827	1.154	1.369	2.249	1.112	3.072	1.339
	336	0.315	0.363	0.325	0.366	0.339	0.372	1.363	0.887	1.334	0.872	1.549	0.972	3.160	1.369	2.568	1.238	3.105	1.348
	720	0.411	0.410	0.421	0.415	0.422	0.419	3.379	1.388	3.048	1.328	2.631	1.242	3.171	1.368	2.720	1.278	3.135	1.354
Electricity	96	0.189	0.304	0.193	0.308	0.201	0.317	0.274	0.368	0.258	0.357	0.312	0.402	0.680	0.645	0.375	0.437	0.985	0.813
	192	0.197	0.313	0.201	0.315	0.222	0.334	0.296	0.386	0.266	0.368	0.348	0.433	0.725	0.676	0.442	0.473	0.996	0.821
	336	0.207	0.324	0.214	0.329	0.231	0.338	0.300	0.394	0.280	0.380	0.350	0.433	0.828	0.727	0.439	0.473	1.000	0.824
	720	0.238	0.349	0.246	0.355	0.254	0.361	0.373	0.439	0.283	0.376	0.340	0.420	0.957	0.811	0.980	0.814	1.438	0.784
Exchange	96	0.120	0.250	0.148	0.278	0.197	0.323	0.847	0.752	0.968	0.812	1.065	0.829	1.551	1.058	1.453	1.049	3.004	1.432
	192	0.226	0.344	0.271	0.380	0.300	0.369	1.204	0.895	1.040	0.851	1.188	0.906	1.477	1.028	1.846	1.179	3.048	1.444
	336	0.384	0.452	0.460	0.500	0.509	0.524	1.672	1.036	1.659	1.081	1.350	0.976	1.507	1.031	2.136	1.231	3.113	1.459
	720	1.139	0.813	1.195	0.841	1.447	0.941	2.478	1.310	1.941	1.127	1.510	1.016	2.285	1.243	2.984	1.427	3.150	1.458
Weather	96	0.212	0.294	0.217	0.296	0.266	0.336	0.300	0.384	0.458	0.490	0.689	0.596	0.594	0.587	0.369	0.406	0.615	0.589
	192	0.261	0.329	0.276	0.336	0.307	0.367	0.598	0.544	0.658	0.589	0.752	0.638	0.560	0.565	0.416	0.435	0.629	0.600
	336	0.312	0.364	0.339	0.380	0.359	0.395	0.578	0.523	0.797	0.652	0.639	0.596	0.597	0.587	0.455	0.454	0.639	0.608
	720	0.399	0.413	0.403	0.428	0.419	0.428	1.059	0.741	0.869	0.675	1.130	0.782	0.618	0.599	0.535	0.520	0.639	0.610
Illness	24	2.623	1.060	3.228	1.260	3.483	1.287	5.764	1.677	4.480	1.444	4.000	1.382	6.026	1.770	5.914	1.734	6.624	1.830
	36	2.399	0.981	2.679	1.080	3.103	1.148	4.755	1.467	4.799	1.467	4.783	1.448	5.340	1.668	6.631	1.845	6.858	1.879
	48	2.367	0.980	2.622	1.078	2.669	1.085	4.763	1.469	4.800	1.468	4.832	1.465	6.080	1.787	6.736	1.857	6.968	1.892
	60	2.575	1.056	2.857	1.157	2.770	1.125	5.264	1.564	5.278	1.560	4.882	1.483	5.548	1.720	6.870	1.879	7.127	1.918

数据集	预测长度	本文模型		版本1		版本2		版本3		FEDformer		Autoformer
数据集	预测长度	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTh1	96	0.362	0.408	0.366	0.410	0.413	0.449	0.375	0.413	0.395	0.419	0.449	0.459
	192	0.405	0.434	0.410	0.437	0.468	0.483	0.419	0.442	0.420	0.445	0.500	0.482
	336	0.414	0.440	0.438	0.453	0.637	0.587	0.438	0.454	0.459	0.465	0.521	0.496
	720	0.490	0.498	0.492	0.499	0.664	0.614	0.496	0.500	0.506	0.507	0.544	0.514
ETTm2	96	0.188	0.281	0.190	0.283	0.203	0.304	0.192	0.286	0.203	0.287	0.255	0.339
	192	0.252	0.322	0.258	0.326	0.268	0.347	0.264	0.333	0.269	0.328	0.281	0.340
	336	0.315	0.363	0.322	0.368	0.317	0.384	0.324	0.370	0.325	0.366	0.339	0.372
	720	0.411	0.410	0.419	0.417	0.428	0.425	0.421	0.419	0.421	0.415	0.422	0.419

Multivariate long-term series forecasting model based on decomposition and frequency domain feature extraction

基于分解和频域特征提取的多变量长时间序列预测模型

RichHTML

PDF

PDF (Mobile)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 24

Related Articles 15

Recommended Articles

Metrics

[1]	Shunyong LI, Shiyi LI, Rui XU, Xingwang ZHAO. Incomplete multi-view clustering algorithm based on self-attention fusion [J]. Journal of Computer Applications, 2024, 44(9): 2696-2703.
[2]	Liehong REN, Lyuwen HUANG, Xu TIAN, Fei DUAN. Multivariate long-term series forecasting method with DFT-based frequency-sensitive dual-branch Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2739-2746.
[3]	Yunchuan HUANG, Yongquan JIANG, Juntao HUANG, Yan YANG. Molecular toxicity prediction based on meta graph isomorphism network [J]. Journal of Computer Applications, 2024, 44(9): 2964-2969.
[4]	Xin YANG, Xueni CHEN, Chunjiang WU, Shijie ZHOU. Short-term traffic flow prediction of urban highway based on variant residual model and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2947-2951.
[5]	Yexin PAN, Zhe YANG. Optimization model for small object detection based on multi-level feature bidirectional fusion [J]. Journal of Computer Applications, 2024, 44(9): 2871-2877.
[6]	Jinjin LI, Guoming SANG, Yijia ZHANG. Multi-domain fake news detection model enhanced by APK-CNN and Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2674-2682.
[7]	Jieru JIA, Jianchao YANG, Shuorui ZHANG, Tao YAN, Bin CHEN. Unsupervised person re-identification based on self-distilled vision Transformer [J]. Journal of Computer Applications, 2024, 44(9): 2893-2902.
[8]	Qinzhuang ZHAO, Hongye TAN. Time series causal inference method based on adaptive threshold learning [J]. Journal of Computer Applications, 2024, 44(9): 2660-2666.
[9]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[10]	Xiyuan WANG, Zhancheng ZHANG, Shaokang XU, Baocheng ZHANG, Xiaoqing LUO, Fuyuan HU. Unsupervised cross-domain transfer network for 3D/2D registration in surgical navigation [J]. Journal of Computer Applications, 2024, 44(9): 2911-2918.
[11]	Lilin FAN, Fukang CAO, Wanting WANG, Kai YANG, Zhaoyu SONG. Intermittent demand forecasting method based on adaptive matching of demand patterns [J]. Journal of Computer Applications, 2024, 44(9): 2747-2755.
[12]	Jiepo FANG, Chongben TAO. Hybrid internet of vehicles intrusion detection system for zero-day attacks [J]. Journal of Computer Applications, 2024, 44(9): 2763-2769.
[13]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[14]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[15]	Yuwei DING, Hongbo SHI, Jie LI, Min LIANG. Image denoising network based on local and global feature decoupling [J]. Journal of Computer Applications, 2024, 44(8): 2571-2579.