Time series prediction algorithm based on multi-scale gated dilated convolutional network

doi:10.11772/j.issn.1001-9081.2023111583

Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (11): 3427-3434.DOI: 10.11772/j.issn.1001-9081.2023111583

• Data science and technology • Previous Articles Next Articles

Time series prediction algorithm based on multi-scale gated dilated convolutional network

Yu ZENG¹^,², Yang ZHANG¹^,²^,³(), Shang ZENG¹^,², Maoli FU¹^,²^,³, Qixue HE¹^,², Linlong ZENG¹^,²

^1.Chengdu Institute of Computer Application，Chinese Academy of Sciences，Chengdu Sichuan 610213，China
^2.School of Computer Science and Technology，University of Chinese Academy of Sciences，Beijing 100049，China
^3.Shenzhen CBPM?KEXIN Banking Technology Company Limited，Shenzhen Guangdong 518206，China

Received:2023-11-20 Revised:2024-01-15 Accepted:2024-02-05 Online:2024-02-29 Published:2024-11-10
Contact: Yang ZHANG
About author:ZENG Yu， born in 1999， M. S. candidate. His research interests include time series analysis， data mining.
ZENG Shang， born in 1995， Ph. D. candidate. His research interests include big data analysis， data mining.
FU Maoli， born in 1988， Ph. D. candidate， engineer. His research interests include artificial intelligence， image processing， pattern recognition.
HE Qixue， born in 1978， senior engineer. His research interests include data mining， artificial intelligence.
ZENG Linlong， born in 1998， M. S. candidate. His research interests include machine vision， artificial intelligence.
Supported by:
Sichuan Province Science Program(2023YFG0113)

基于多尺度门控膨胀卷积网络的时间序列预测算法

曾渝¹^,², 张洋¹^,²^,³(), 曾尚¹^,², 付茂栗¹^,²^,³, 何启学¹^,², 曾林隆¹^,²

^1.中国科学院成都计算机应用研究所，成都 610213
^2.中国科学院大学计算机科学与技术学院，北京 100049
^3.深圳市中钞科信金融科技有限公司，广东深圳 518206

通讯作者: 张洋
作者简介:曾渝（1999—），男，重庆人，硕士研究生，主要研究方向：时间序列分析、数据挖掘
曾尚（1995—），男，湖北荆门人，博士研究生，主要研究方向：大数据分析、数据挖掘
付茂栗（1988—），男，四川遂宁人，工程师，博士研究生，主要研究方向：人工智能、图像处理、模式识别
何启学（1978—），男，四川安顺人，高级工程师，主要研究方向：数据挖掘、人工智能
曾林隆（1998—），男，四川内江人，硕士研究生，主要研究方向：机器视觉、人工智能。
基金资助:
四川省科技计划项目(2023YFG0113)

Abstract

Abstract:

Addressing challenges in time series prediction tasks， such as high-dimensional features， large-scale data， and the demand for high prediction accuracy， a multi-scale trend-period decomposition model based on a multi-head gated dilated convolutional network was proposed. A multi-scale decomposition approach was employed to decompose the original covariate sequence and the prediction variable sequence into their respective periodic terms and trend terms， thereby enabling independent prediction. For the periodic terms， the multi-head gated dilated convolutional network encoder was introduced to extract respective periodic information； in the decoder stage， channel information interaction and fusion were performed through the utilization of a cross-attention mechanism， and after sampling and aligning the periodic information of the prediction variables， the periodic prediction was performed through time attention and channel fusion information. The trend terms prediction was executed by using an autoregressive approach. Finally， the prediction sequence was obtained by incorporating the trend prediction results with the periodic prediction results. Compared with multiple mainstream benchmark models such as Long Short-Term Memory （LSTM） and Informer， on five datasets including ETTm1 and ETTh1， a reduction in Mean Squared Error （MSE） is observed， ranging from 19.2% to 52.8% on average， a decrease in Mean Absolute Error （MAE） is noted， ranging from 12.1% to 33.8% on average. Ablation experiments confirm that the proposed multi-scale decomposition module， multi-head gated dilation convolution， and time attention module can enhance the accuracy of time series prediction.

Key words: time series prediction, sequence decomposition, dilated convolution, encoder, decoder, attention mechanism

摘要：

针对当前时间序列预测任务存在的高维特征、大规模数据以及对预测准确性高要求等问题，提出一种基于多尺度趋势-周期分解的多头门控膨胀卷积网络模型。该模型采用多尺度分解方法，将原始协变量序列和预测变量序列分解为各自的周期项和趋势项，从而实现独立的预测。对于周期项，引入多头门控膨胀卷积网络的编码器，以提取各自的周期信息；在解码器阶段，使用交叉注意力机制进行通道信息的交互融合，并将预测变量的周期信息采样对齐后通过时间注意力与通道融合信息进行周期预测。对趋势项则采用自回归方式进行趋势预测。最后将趋势预测与周期预测的结果相加得到预测序列。与长短期记忆（LSTM）、Informer等多个主流基准模型进行比较，所提模型在ETTm1、ETTh1等5个数据集上的均方误差（MSE）平均下降了19.2%~52.8%，平均绝对误差（MAE）平均下降了12.1%~33.8%。通过消融实验验证了所提出的多尺度分解模块、多头门控膨胀卷积以及时间注意力模块能提升时序预测的准确度。

关键词: 时间序列预测, 序列分解, 膨胀卷积, 编码器, 解码器, 注意力机制

CLC Number:

TP399

Yu ZENG, Yang ZHANG, Shang ZENG, Maoli FU, Qixue HE, Linlong ZENG. Time series prediction algorithm based on multi-scale gated dilated convolutional network[J]. Journal of Computer Applications, 2024, 44(11): 3427-3434.

曾渝, 张洋, 曾尚, 付茂栗, 何启学, 曾林隆. 基于多尺度门控膨胀卷积网络的时间序列预测算法[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3427-3434.

Figures/Tables 9

References 19

1	LI J， YANG C， LI Y， et al. A context-aware enhanced GRU network with feature-temporal attention for prediction of silicon content in hot metal［J］. IEEE Transactions on Industrial Informatics， 2022， 18（10）： 6631-6641.
2	BI K， XIE L， ZHANG H， et al. Accurate medium-range global weather forecasting with 3D neural networks［J］. Nature， 2023， 619（7970）： 533-538.
3	ZHOU X， PAN Z， HU G， et al. Stock market prediction on high-frequency data using generative adversarial nets［J］. Mathematical Problems in Engineering， 2018， 2018： No.4907423.
4	李志强，张香燕，李鸿飞，等.基于GM-ARIMA模型的遥测数据中长期预测［J］.计算机仿真，2023，40（5）：106-111.
	LI Z Q， ZHANG X Y， LI H F， et al. Long-term prediction method of telemetry data based on GM-ARIMA model［J］. Computer Simulation， 2023， 40（5）：106-111.
5	徐耀松，王德兴.基于加权灰色关联投影的Bagging-Blending多模型融合短期电力负荷预测［J］.信息与控制，2022，51（2）：188-200.
	XU Y S， WANG D X. Bagging-Blending multi-model fusion short-term power load forecasting based on weighted gray relational projection［J］. Information and Control， 2022， 51（2）：188-200.
6	BAI S， KOLTER J Z， KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling［EB/OL］. （2018-04-19）［2024-01-15］..
7	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 6000-6010.
8	HAN L， YE H J， ZHAN D C. The capacity and robustness trade-off： revisiting the channel independent strategy for multivariate time series forecasting［J］. IEEE Transactions on Knowledge and Data Engineering， 2024（Early Access）： 1-14.
9	毛远宏，孙琛琛，徐鲁豫，等.基于深度学习的时间序列预测方法综述［J］.微电子学与计算机，2023，40（4）：8-17.
	MAO Y H， SUN C C， XU L Y， et al. A survey of time series forecasting methods based on deep learning［J］. Microelectronics and Computer， 2023， 40（4）：8-17.
10	DU D， SU B， WEI Z. Preformer： predictive Transformer with multi-scale segment-wise correlations for long-term time series forecasting［C］// Proceedings of the 2023 IEEE International Conference on Acoustics， Speech and Signal Processing. Piscataway： IEEE， 2023： 1-5.
11	LIU M， ZENG A， CHEN M， et al. SCINet： time series modeling and forecasting with sample convolution and interaction［C/OL］// Proceedings of the 36th International Conference on Neural Information Processing Systems. San Diego， CA： NeurIPS， 2022 ［2024-01-15］..
12	ZHOU H， ZHANG S， PENG J， et al. Informer： beyond efficient Transformer for long sequence time-series forecasting［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2021： 11106-11115.
13	ZENG A， CHEN M， ZHANG L， et al. Are Transformers effective for time series forecasting？［C］// Proceedings of the 37th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2023： 11121-11128.
14	NIE Y， NGUYEN N H， SINTHONG P， et al. A time series is worth 64 words： long-term forecasting with Transformers［EB/OL］. ［2024-01-10］..
15	ZHANG Y， YAN J. Crossformer： Transformer utilizing cross-dimension dependency for multivariate time series forecasting［EB/OL］. ［2023-12-16］..
16	CHEN S A， LI C L， YODER N C， et al. TSMixer： an all-MLP architecture for time series forecasting［EB/OL］. ［2024-01-19］..
17	KIM T， KIM J， TAE Y， et al. Reversible instance normalization for accurate time-series forecasting against distribution shift［EB/OL］. ［2024-03-21］..
18	KINGMA D P， BA J L. Adam： a method for stochastic optimization［EB/OL］. ［2023-12-29］..
19	ZHOU W， XU C， GE T， et al. BERT loses patience： fast and robust inference with early exit［C/OL］// Proceedings of the 34th International Conference on Neural Information Processing Systems. San Diego， CA： NeurIPS， 2020 ［2024-01-15］..

数据集	输入长度	预测长度	本文模型		LSTM		TCN		Informer		NLinear		PatchTST		Crossformer		TSMixer
数据集	输入长度	预测长度	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	96	0.104	0.230	0.286	0.440	0.165	0.310	0.195	0.335	0.115	0.238	0.171	0.302	0.146	0.282	0.121	0.258
		192	0.172	0.309	0.412	0.482	0.292	0.431	0.456	0.550	0.184	0.318	0.201	0.315	0.250	0.392	0.174	0.315
		336	0.231	0.360	0.523	0.588	0.650	0.662	0.554	0.615	0.267	0.392	0.277	0.405	0.382	0.455	0.284	0.414
		720	0.317	0.431	0.936	0.773	0.614	0.609	0.768	0.717	0.420	0.513	0.323	0.445	0.510	0.568	0.404	0.508
ETTh1	48	24	0.061	0.185	0.077	0.210	0.092	0.236	0.122	0.280	0.132	0.293	0.121	0.281	0.066	0.192	0.125	0.290
		48	0.084	0.218	0.142	0.296	0.128	0.281	0.171	0.332	0.172	0.348	0.137	0.295	0.125	0.271	0.146	0.308
		96	0.115	0.263	0.287	0.444	0.192	0.350	0.180	0.339	0.211	0.372	0.201	0.365	0.295	0.426	0.204	0.375
		192	0.145	0.295	0.328	0.470	0.198	0.366	0.208	0.372	0.532	0.628	0.468	0.512	0.372	0.483	0.564	0.640
Weather	144	72	0.062	0.178	0.125	0.257	0.127	0.279	0.099	0.240	0.068	0.189	0.062	0.186	0.064	0.187	0.069	0.195
		144	0.099	0.234	0.138	0.278	0.142	0.298	0.131	0.275	0.098	0.238	0.099	0.238	0.127	0.270	0.101	0.234
		288	0.161	0.296	0.224	0.356	0.218	0.352	0.186	0.332	0.161	0.297	0.169	0.315	0.163	0.312	0.181	0.299
		432	0.180	0.313	0.273	0.380	0.262	0.403	0.238	0.367	0.214	0.333	0.194	0.325	0.229	0.355	0.209	0.335
ILI	104	52	0.546	0.463	1.531	1.057	1.675	1.060	1.618	1.194	1.850	1.200	1.980	1.198	0.700	0.664	1.989	1.285
		104	1.139	0.864	2.351	1.231	2.579	1.198	2.054	1.033	2.846	1.431	2.040	1.090	1.816	1.032	2.441	1.389
		208	2.644	1.278	3.868	1.549	3.871	1.536	3.278	1.910	5.363	2.083	3.835	1.520	3.097	1.395	3.016	1.490
Electricity	72	24	0.268	0.393	0.308	0.424	0.509	0.538	0.364	0.450	0.289	0.417	0.280	0.407	0.309	0.428	0.291	0.415
		72	0.385	0.474	0.425	0.496	0.597	0.579	0.498	0.523	0.468	0.516	0.425	0.480	0.422	0.497	0.395	0.485
		168	0.393	0.479	0.492	0.539	0.615	0.620	0.569	0.576	0.551	0.571	0.455	0.500	0.518	0.571	0.425	0.493
		240	0.389	0.463	0.499	0.555	0.642	0.616	0.669	0.677	0.573	0.574	0.482	0.516	0.547	0.601	0.449	0.513

数据集	输入长度	预测长度	本文模型		LSTM		TCN		Informer		NLinear		PatchTST		Crossformer		TSMixer
数据集	输入长度	预测长度	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE	MSE	MAE
ETTm1	96	96	0.104	0.230	0.286	0.440	0.165	0.310	0.195	0.335	0.115	0.238	0.171	0.302	0.146	0.282	0.121	0.258
		192	0.172	0.309	0.412	0.482	0.292	0.431	0.456	0.550	0.184	0.318	0.201	0.315	0.250	0.392	0.174	0.315
		336	0.231	0.360	0.523	0.588	0.650	0.662	0.554	0.615	0.267	0.392	0.277	0.405	0.382	0.455	0.284	0.414
		720	0.317	0.431	0.936	0.773	0.614	0.609	0.768	0.717	0.420	0.513	0.323	0.445	0.510	0.568	0.404	0.508
ETTh1	48	24	0.061	0.185	0.077	0.210	0.092	0.236	0.122	0.280	0.132	0.293	0.121	0.281	0.066	0.192	0.125	0.290
		48	0.084	0.218	0.142	0.296	0.128	0.281	0.171	0.332	0.172	0.348	0.137	0.295	0.125	0.271	0.146	0.308
		96	0.115	0.263	0.287	0.444	0.192	0.350	0.180	0.339	0.211	0.372	0.201	0.365	0.295	0.426	0.204	0.375
		192	0.145	0.295	0.328	0.470	0.198	0.366	0.208	0.372	0.532	0.628	0.468	0.512	0.372	0.483	0.564	0.640
Weather	144	72	0.062	0.178	0.125	0.257	0.127	0.279	0.099	0.240	0.068	0.189	0.062	0.186	0.064	0.187	0.069	0.195
		144	0.099	0.234	0.138	0.278	0.142	0.298	0.131	0.275	0.098	0.238	0.099	0.238	0.127	0.270	0.101	0.234
		288	0.161	0.296	0.224	0.356	0.218	0.352	0.186	0.332	0.161	0.297	0.169	0.315	0.163	0.312	0.181	0.299
		432	0.180	0.313	0.273	0.380	0.262	0.403	0.238	0.367	0.214	0.333	0.194	0.325	0.229	0.355	0.209	0.335
ILI	104	52	0.546	0.463	1.531	1.057	1.675	1.060	1.618	1.194	1.850	1.200	1.980	1.198	0.700	0.664	1.989	1.285
		104	1.139	0.864	2.351	1.231	2.579	1.198	2.054	1.033	2.846	1.431	2.040	1.090	1.816	1.032	2.441	1.389
		208	2.644	1.278	3.868	1.549	3.871	1.536	3.278	1.910	5.363	2.083	3.835	1.520	3.097	1.395	3.016	1.490
Electricity	72	24	0.268	0.393	0.308	0.424	0.509	0.538	0.364	0.450	0.289	0.417	0.280	0.407	0.309	0.428	0.291	0.415
		72	0.385	0.474	0.425	0.496	0.597	0.579	0.498	0.523	0.468	0.516	0.425	0.480	0.422	0.497	0.395	0.485
		168	0.393	0.479	0.492	0.539	0.615	0.620	0.569	0.576	0.551	0.571	0.455	0.500	0.518	0.571	0.425	0.493
		240	0.389	0.463	0.499	0.555	0.642	0.616	0.669	0.677	0.573	0.574	0.482	0.516	0.547	0.601	0.449	0.513

数据集	输入长度	MSE	数据集	输入长度	MSE	数据集	输入长度	MSE
ETTm1	24	0.130	Weather	24	0.144	Electricity	24	0.613
	48	0.112		72	0.122		72	0.252
	96	0.109		144	0.110		144	0.270
	192	0.127		288	0.117		168	0.427
	336	0.113		432	0.141		240	0.402

数据集	输入长度	MSE	数据集	输入长度	MSE	数据集	输入长度	MSE
ETTm1	24	0.130	Weather	24	0.144	Electricity	24	0.613
	48	0.112		72	0.122		72	0.252
	96	0.109		144	0.110		144	0.270
	192	0.127		288	0.117		168	0.427
	336	0.113		432	0.141		240	0.402

模型	MSE	模型	MSE
本文模型	0.108	Model-2	0.112
Model-1	0.127	Model-3	0.120

Time series prediction algorithm based on multi-scale gated dilated convolutional network

基于多尺度门控膨胀卷积网络的时间序列预测算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 19

Related Articles 15

Recommended Articles

Metrics

[1]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[2]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[3]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[4]	Zhigang XU, Chuang ZHANG. Multi-level color restoration of mural image based on gated positional encoding [J]. Journal of Computer Applications, 2024, 44(9): 2931-2937.
[5]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[6]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[7]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[8]	Kaili DENG, Weibo WEI, Zhenkuan PAN. Industrial defect detection method with improved masked autoencoder [J]. Journal of Computer Applications, 2024, 44(8): 2595-2603.
[9]	Chenqian LI, Jun LIU. Ultrasound carotid plaque segmentation method based on semi-supervision and multi-scale cascaded attention [J]. Journal of Computer Applications, 2024, 44(8): 2604-2610.
[10]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[11]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[12]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[13]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[14]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.
[15]	Song XU, Wenbo ZHANG, Yifan WANG. Lightweight video salient object detection network based on spatiotemporal information [J]. Journal of Computer Applications, 2024, 44(7): 2192-2199.