Shorter long-sequence time series forecasting model

doi:10.11772/j.issn.1001-9081.2023060799

Abstract

Abstract:

Aiming at the problem that most of the existing researches study short-sequence time series forecasting and long-sequence time series forecasting separately， which leads to the poor forecasting accuracy of the model in the shorter long-sequence time series， a Shorter Long-sequence Time Series Forecasting Model （SLTSFM） was proposed. Firstly， a Sequence-to-Sequence （Seq2Seq） structure was constructed using Convolutional Neural Network （CNN） and PBUSM （Probsparse Based on Uniform Selection Mechanism） self-attention mechanism， which was used to extract the features of the long-sequence input. Secondly， “far light， near heavy” strategy was designed to apply to reallocate the features of each time period extracted from multiple Long Short-Term Memory （LSTM） modules， which were more capable of short-sequence input feature extraction. Finally， the reallocated features were used to enhance the extracted long-sequence input features to improve the forecasting accuracy and realize the time series forecasting. Four publicly available time series datasets were utilized to verify the effectiveness of the proposed model. The experimental results demonstrate that， compared with the suboptimal comprehensive performing model Gated Recurrent Unit （GRU）， the Mean Absolute Error （MAE） metrics of SLTSFM were reduced by 61.54%， 13.48%， 0.92% and 19.58% for univariate time series forecasting， and were reduced by 17.01%， 18.13%， 3.24% and 6.73% for multivariate time series forecasting on the four datasets. It’s verified that SLTSFM is effective in improving the accuracy of shorter long-sequence time series forecasting.

Key words: shorter long-sequence time series forecasting, Sequence-to-Sequence (Seq2Seq), Long Short-Term Memory (LSTM), self-attention mechanism, feature reallocation

摘要：

针对现有的研究大多将短序列时间序列预测和长序列时间序列预测分开研究而导致模型在较短的长序列时序预测时精度较低的问题，提出一种较短的长序列时间序列预测模型（SLTSFM）。首先，利用卷积神经网络（CNN）和PBUSM（Probsparse Based on Uniform Selection Mechanism）自注意力机制搭建一个序列到序列（Seq2Seq）结构，用于提取长序列输入的特征；其次，设计“远轻近重”策略将多个短序列输入特征提取能力较强的长短时记忆（LSTM）模块提取的各时段数据特征进行重分配；最后，用重分配的特征增强提取的长序列输入特征，提高预测精度并实现时序预测。利用4个公开的时间序列数据集验证模型的有效性。实验结果表明，与综合表现次优的对比模型循环门单元（GRU）相比，SLTSFM的平均绝对误差（MAE）指标在4个数据集上的单变量时序预测分别减小了61.54%、13.48%、0.92%和19.58%，多变量时序预测分别减小了17.01%、18.13%、3.24%和6.73%。由此可见SLTSFM在提升较短的长序列时序预测精度方面的有效性。

关键词: 较短的长序列时间序列预测, 序列到序列, 长短期记忆, 自注意力机制, 特征重分配

CLC Number:

TP183

Zexin XU, Lei YANG, Kangshun LI. Shorter long-sequence time series forecasting model[J]. Journal of Computer Applications, 2024, 44(6): 1824-1831.

徐泽鑫, 杨磊, 李康顺. 较短的长序列时间序列预测模型[J]. 《计算机应用》唯一官方网站, 2024, 44(6): 1824-1831.

Figures/Tables 12

References 20

1	徐晓芳，管瑞.基于神经网络集成学习算法的金融时间序列预测［J］. 计算机系统应用， 2022， 31（6）： 29-37.
	XU X F， GUAN R. Financial time series forecasting based on neural network ensemble learning algorithms［J］. Computer Systems & Applications， 2022， 31（6）： 29-37.
2	李毅，彭晋卿，廖维，等.一种基于时间序列的集成电力负荷预测方法研究［J］.建筑科学， 2022， 38（10）： 190-197.
	LI Y， PENG J Q， LIAO W， et al. Research on an integrated model for electrical load forecasting based on time series ［J］. Building Science， 2022， 38（10）： 190-197.
3	夏进，王正群，朱世明.基于时间序列分解的交通流量预测模型［J］. 计算机应用， 2023， 43（4）： 1129-1135.
	XIA J， WANG Z Q， ZHU S M. Traffic flow prediction model based on time series decomposition［J］. Journal of Computer Applications， 2023， 43（4）： 1129-1135.
4	王海起，王志海，李留珂，等.基于网格划分的城市短时交通流量时空预测模型［J］. 计算机应用， 2022， 42（7）： 2274-2280.
	WANG H Q， WANG Z H， LI L K， et al. Spatial-temporal prediction model of urban short-term traffic flow based on grid division［J］. Journal of Computer Applications， 2022， 42（7）： 2274-2280.
5	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780.
6	CHO K， VAN MERRIËNBOER B， BAHDANAU D， et al. On the properties of neural machine translation： encoder-decoder approaches ［EB/OL］. （2014-10-07）［2023-06-20］. .
7	BOX G E P， JENKINS G M， REINSEL G C， et al. Time Series Analysis： Forecasting and Control［M］. 5th ed. Hoboken： John Wiley & Sons， 2015： 88-94.
8	DRUCKER H， BURGES C J C， KAUFMAN L， et al. Support vector regression machines ［C］// Proceedings of the 9th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 1997： 155-161.
9	LeCUN Y， BOSER B， DENKER J S， et al. Backpropagation applied to handwritten zip code recognition［J］. Neural Computation， 1989， 1（4）： 541-551.
10	SUTSKEVER I， VINYALS O， LE Q V. Sequence to sequence learning with neural networks［C］// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge： MIT Press， 2014： 3104-3112.
11	BAHDANAU D， CHO K， BENGIO Y. Neural machine translation by jointly learning to align and translate ［EB/OL］. （2016-05-19）［2023-06-20］. .
12	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need ［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook： Curran Associates， 2017： 6000-6010.
13	HOPFIELD J J. Neural networks and physical systems with emergent collective computational abilities［J］. Proceedings of the National Academy of Sciences of the United States of America， 1982， 79（8）： 2554-2558.
14	ZHOU H， ZHANG S， PENG J， et al. Informer： beyond efficient Transformer for long sequence time-series forecasting［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Menlo Park： AAAI Press， 2021： 11106-11115.
15	LIANG X， ZOU T， GUO B， et al. Assessing Beijing’s PM_2.5 pollution： severity， weather impact， APEC and winter heating［J］. Proceedings of the Royal Society A： Mathematical， Physical and Engineering Sciences， 2015， 471： 20150257.
16	ZHENG Y， YI X， LI M， et al. Forecasting fine-grained air quality based on big data［C］// Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2015： 2267-2276.
17	BAI S， KOLTER J Z， KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling ［EB/OL］. （2018-04-19）［2023-06-20］. .
18	HE K， ZHANG X， REN S， et al. Deep residual learning for image recognition ［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016：770-778.
19	SHEN Z， ZHANG Y， LU J， et al. A novel time series forecasting model with deep learning［J］. Neurocomputing， 2020， 396： 302-313.
20	YI S， LIU H， CHEN T， et al. A deep LSTM-CNN based on self-attention mechanism with input data reduction for short-term load forecasting［J］. IET Generation， Transmission & Distribution， 2023， 17（7）： 1538-1552.

预测步长	输入长度	标签长度
24	96	48
48	128	64
72	152	86
96	176	110
168	256	192

预测步长	输入长度	标签长度
24	96	48
48	128	64
72	152	86
96	176	110
168	256	192

数据集	类型	SLTSFM		SLTSFM-		LSTM+		GRU		TCN		SeriesNet		LCS
数据集	类型	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETT	U	0.107	0.255	0.181	0.351	0.584	0.663	0.584	0.663	0.591	0.668	0.586	0.664	0.583	0.663
ETT	M	0.586	0.556	1.006	0.760	0.861	0.676	0.855	0.670	0.972	0.739	0.846	0.667	0.957	0.742
PM2.5A	U	0.817	0.629	1.058	0.765	0.914	0.730	0.906	0.727	1.025	0.754	0.964	0.738	0.935	0.727
PM2.5A	M	0.587	0.429	1.376	0.746	0.709	0.528	0.699	0.524	0.822	0.697	0.748	0.538	0.811	0.568
PM2.5B	U	0.696	0.647	0.888	0.790	0.701	0.661	0.687	0.653	0.752	0.675	0.719	0.663	0.748	0.671
PM2.5B	M	1.048	0.716	1.581	1.007	1.007	0.747	0.993	0.740	1.182	0.816	1.026	0.743	1.124	0.797
WTH	U	0.306	0.423	0.907	0.786	0.491	0.521	0.499	0.526	0.807	0.722	0.657	0.626	0.767	0.694
WTH	M	0.535	0.513	0.935	0.749	0.598	0.551	0.608	0.550	0.820	0.697	0.671	0.599	0.732	0.643

数据集	类型	SLTSFM		SLTSFM-		LSTM+		GRU		TCN		SeriesNet		LCS
数据集	类型	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETT	U	0.107	0.255	0.181	0.351	0.584	0.663	0.584	0.663	0.591	0.668	0.586	0.664	0.583	0.663
ETT	M	0.586	0.556	1.006	0.760	0.861	0.676	0.855	0.670	0.972	0.739	0.846	0.667	0.957	0.742
PM2.5A	U	0.817	0.629	1.058	0.765	0.914	0.730	0.906	0.727	1.025	0.754	0.964	0.738	0.935	0.727
PM2.5A	M	0.587	0.429	1.376	0.746	0.709	0.528	0.699	0.524	0.822	0.697	0.748	0.538	0.811	0.568
PM2.5B	U	0.696	0.647	0.888	0.790	0.701	0.661	0.687	0.653	0.752	0.675	0.719	0.663	0.748	0.671
PM2.5B	M	1.048	0.716	1.581	1.007	1.007	0.747	0.993	0.740	1.182	0.816	1.026	0.743	1.124	0.797
WTH	U	0.306	0.423	0.907	0.786	0.491	0.521	0.499	0.526	0.807	0.722	0.657	0.626	0.767	0.694
WTH	M	0.535	0.513	0.935	0.749	0.598	0.551	0.608	0.550	0.820	0.697	0.671	0.599	0.732	0.643

模型	ETT				PM2.5A				PM2.5B				WTH				平均排名
	单变量		多变量		单变量		多变量		单变量		多变量		单变量		多变量
	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
SLTSFM	1	1	1	1	1	1	1	1	2	1	4	1	1	1	1	1	1.25
SLTSFM-	2	2	7	7	7	7	7	7	7	7	7	7	7	7	7	7	6.38
LSTM+	5	5	4	4	3	4	3	3	3	3	2	4	2	2	2	3	3.25
GRU	4	4	3	3	2	2	2	2	1	2	1	2	3	3	3	2	2.44
TCN	7	7	6	5	6	6	6	6	6	6	6	6	6	6	6	6	6.06
SeriesNet	6	6	2	2	5	5	4	4	4	4	3	3	4	4	4	4	4.00
LCS	3	3	5	6	4	3	5	5	5	5	5	5	5	5	5	5	4.63