Multi-site wind speed prediction based on graph dynamic attention network

doi:10.11772/j.issn.1001-9081.2022111749

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (11): 3616-3624.DOI: 10.11772/j.issn.1001-9081.2022111749

Special Issue: 前沿与综合应用

• Frontier and comprehensive applications • Previous Articles Next Articles

Multi-site wind speed prediction based on graph dynamic attention network

Bolu LI, Li WU, Xiaoying WANG(), Jianqiang HUANG, Tengfei CAO

Department of Computer Technology and Applications，Qinghai University，Xining Qinghai 810016，China

Received:2022-11-24 Revised:2023-03-10 Accepted:2023-03-17 Online:2023-04-03 Published:2023-11-10
Contact: Xiaoying WANG
About author:LI Bolu， born in 1997， M. S. candidate. His research interests include artificial intelligence， spatio-temporal meteorological prediction.
WU Li， born in 1992， M. S.， teaching assistant. Her research interests include artificial intelligence， high performance computing.
WANG Xiaoying， born in 1982， Ph. D.， professor. Her research interests include smart grid， high performance computing， computer architecture.
HUANG Jianqiang， born in 1985， Ph. D.， professor. His research interests include high performance computing， big data processing.
CAO Tengfei， born in 1987， Ph. D.， associate professor. His research interests include intelligent network optimization， network attack and defense.
Supported by:
National Natural Science Foundation of China(62162053);Project of Tsinghua-Ningxia Yinchuan Joint Institute of Internet of Waters on Digital Water Governance(SKL-IOW-2020TC2004-01);Application Basic Research Project of Science and Technology Department of Qinghai Province(2022-ZJ-701)

基于图动态注意力网络的多站点风速预测

李博录, 吴利, 王晓英(), 黄建强, 曹腾飞

青海大学计算机技术与应用系，西宁 810016

通讯作者: 王晓英
作者简介:李博录（1997—），男，甘肃天水人，硕士研究生，主要研究方向：人工智能、时空气象预测
吴利（1992—），女，安徽铜陵人，助教，硕士，主要研究方向：人工智能、高性能计算
王晓英（1982—），女，吉林大安人，教授，博士，主要研究方向：智能电网、高性能计算、计算机体系结构 wxy_cta@qhu.edu.cn
黄建强（1985—），男，陕西西安人，教授，博士，主要研究方向：高性能计算、大数据处理
曹腾飞（1987—），男，湖北钟祥人，副教授，博士，主要研究方向：智能网络优化、网络攻防。
基金资助:
国家自然科学基金资助项目(62162053);清华大学—宁夏银川水联网数字治水联合研究院横向课题(SKL?IOW?2020TC2004?01);青海省科技厅应用基础研究项目(2022?ZJ?701)

Abstract

Abstract:

The task of spatio-temporal sequence prediction has a wide range of applications in the fields such as transportation， meteorology and smart city. It is necessary to learn the spatio-temporal characteristics of different data with the combination of external factors such as precipitation and temperature when making station wind speed predictions， which is one of the main tasks in meteorological forecasting. The irregular distribution of meteorological stations and the inherent intermittency of the wind itself bring the challenge of achieving wind speed prediction with high accuracy. In order to consider the influence of multi-site spatial distribution on wind speed to obtain accurate and reliable prediction results， a Graph-based Dynamic Switch-Attention Network （Graph-DSAN） wind speed prediction model was proposed. Firstly， the distances between different sites were used to reconstruct the connection of them. Secondly， the process of local sampling was used to model adjacency matrices of different sampling sizes to achieve the aggregation and transmission of the information between neighbor nodes during the graph convolution process. Thirdly， the results of the graph convolution processed by Spatio-Temporal Position Encoding （STPE） were fed into the Dynamic Attention Encoder （DAE） and Switch-Attention Decoder （SAD） for dynamic attention computation to extract the spatio-temporal correlations. Finally， a multi-step prediction was formed by using autoregression. In experiments on wind speed prediction on 15 sites data in New York State， the designed model was compared with ConvLSTM， Graph Multi-Attention Network （GMAN）， Spatio-Temporal Graph Convolutional Network （STGCN）， Dynamic Switch-Attention Network （DSAN） and Spatial-Temporal Dynamic Network （STDN）. The results show that the Root Mean Square Error （RMSE） of 12 h prediction of Graph-DSAN model is reduced by 28.2%， 6.9%， 27.7%， 14.4% and 8.9% respectively， verifying the accuracy of Graph-DSAN in wind speed prediction.

Key words: wind speed prediction, dynamic attention network, graph convolution, attention mechanism

摘要：

时空序列预测任务在交通、气象、智慧城市等领域有着广泛应用。站点风速预测作为气象预测中的主要任务之一，需要结合降水、气温等外部因素，学习不同数据的时空特征。气象站点的不规则分布和风本身的固有间歇性成为实现高精度风速预测的挑战。为考虑多站点空间分布对风速的影响以获得准确可靠的预测结果，提出一种基于图的动态转换注意力网络（Graph-DSAN）风速预测模型。首先，利用不同站点之间的距离重新构建它们的连接；其次，使用局部采样的过程建模不同采样大小的邻接矩阵，实现图卷积过程中邻居节点信息的聚合与传递；接着，将时空位置编码（STPE）处理后的图卷积结果加入动态注意力编码器（DAE）和转换注意力解码器（SAD）以实现动态注意力计算，从而提取时空相关性；最后，利用自回归的方式形成多步预测。在纽约州15个站点的风速预测实验中，将所设计模型与ConvLSTM、图多注意力网络（GMAN）、时空图卷积网络（STGCN）、动态转换注意力网络（DSAN）和时空动态网络（STDN）进行比较，Graph-DSAN的12 h预测均方根误差（RMSE）分别降低了28.2%、6.9%、27.7%、14.4%和8.9%，验证了Graph-DSAN风速预测的准确性。

关键词: 风速预测, 动态注意力网络, 图卷积, 注意力机制

CLC Number:

TP181

Bolu LI, Li WU, Xiaoying WANG, Jianqiang HUANG, Tengfei CAO. Multi-site wind speed prediction based on graph dynamic attention network[J]. Journal of Computer Applications, 2023, 43(11): 3616-3624.

李博录, 吴利, 王晓英, 黄建强, 曹腾飞. 基于图动态注意力网络的多站点风速预测[J]. 《计算机应用》唯一官方网站, 2023, 43(11): 3616-3624.

Figures/Tables 13

References 38

1	刘苇航，叶涛，史培军，等. 气候变化对粮食生产风险的影响研究进展［J］. 自然灾害学报， 2022， 31（4）：1-11.
	LIU W H， YE T， SHI P J， et al. Advances in the study of climate change impact on crop producing risk［J］. Journal of Natural Disasters， 2022， 31（4）： 1-11.
2	孟鑫禹，王睿涵，张喜平，等. 基于经验模态分解与多分支神经网络的超短期风功率预测［J］. 计算机应用， 2021， 41（1）： 237-242. 10.11772/j.issn.1001-9081.2020060930
	MENG X Y， WANG R H， ZHANG X P， et al. Ultra-short-term wind power prediction based on empirical mode decomposition and multi-branch neural network［J］. Journal of Computer Applications， 2021， 41（1）： 237-242. 10.11772/j.issn.1001-9081.2020060930
3	PIOTROWSKI P， BACZYŃSKI D， KOPYT M， et al. Analysis of forecasted meteorological data （NWP） for efficient spatial forecasting of wind power generation［J］. Electric Power Systems Research， 2019， 175： No.105891. 10.1016/j.epsr.2019.105891
4	AGRAWAL S， BARRINGTON L， BROMBERG C， et al. Machine learning for precipitation nowcasting from radar images［EB/OL］. （2019-12-11）［2022-11-08］..
5	孙丽华，严军峰，徐健锋. 基于多机器学习竞争策略的短时雷电预报［J］. 计算机应用， 2016， 36（9）：2555-2559. 10.11772/j.issn.1001-9081.2016.09.2555
	SUN L H， YAN J F， XU J F. Short-term lightning prediction based on multi-machine learning competitive strategy［J］. Journal of Computer Applications， 2016， 36（9）： 2555-2559. 10.11772/j.issn.1001-9081.2016.09.2555
6	RAVURI S， LENC K， WILLSON M， et al. Skilful precipitation nowcasting using deep generative models of radar［J］. Nature， 2021， 597（7878）： 672-677. 10.1038/s41586-021-03854-z
7	石峰，楼文高，张博. 基于灰狼群智能最优化的神经网络PM_2.5浓度预测［J］. 计算机应用， 2017， 37（10）：2854-2860. 10.11772/j.issn.1001-9081.2017.10.2854
	SHI F， LOU W G， ZHANG B. Neural network model for PM_2.5 concentration prediction by grey wolf optimizer algorithm［J］. Journal of Computer Applications， 2017， 37（10）： 2854-2860. 10.11772/j.issn.1001-9081.2017.10.2854
8	王军，费凯，程勇. 基于改进Adaboost-BP模型在降水中的预测［J］. 计算机应用， 2017， 37（9）：2689-2693. 10.11772/j.issn.1001-9081.2017.09.2689
	WANG J， FEI K， CHENG Y. Prediction of rainfall based on improved Adaboost-BP model［J］. Journal of Computer Applications， 2017， 37（9）： 2689-2693. 10.11772/j.issn.1001-9081.2017.09.2689
9	SCHER S. Toward data‐driven weather and climate forecasting： approximating a simple general circulation model with deep learning［J］. Geophysical Research Letters， 2018， 45（22）： 12616-12622. 10.1029/2018gl080704
10	SALMAN A G， KANIGORO B， HERYADI Y. Weather forecasting using deep learning techniques［C］// Proceedings of the 2015 International Conference on Advanced Computer Science and Information Systems. Piscataway： IEEE， 2015： 281-285. 10.1109/icacsis.2015.7415154
11	MEHRKANOON S. Deep shared representation learning for weather elements forecasting［J］. Knowledge-Based Systems， 2019， 179： 120-128. 10.1016/j.knosys.2019.05.009
12	TREBING K， MEHRKANOON S. Wind speed prediction using multidimensional convolutional neural networks［C］// Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence. Piscataway： IEEE， 2020： 713-720. 10.1109/ssci47803.2020.9308323
13	ZHANG S， TONG H， XU J， et al. Graph convolutional networks： a comprehensive review［J］. Computational Social Networks， 2019， 6： No.11. 10.1186/s40649-019-0069-y
14	STAŃCZYK T， MEHRKANOON S. Deep graph convolutional networks for wind speed prediction［C/OL］// Proceedings of the 29th European Symposium on Artificial Neural Networks， Computational Intelligence and Machine Learning ［2022-11-08］.. 10.14428/esann/2021.es2021-25
15	YU B， YIN H， ZHU Z. Spatio-temporal graph convolutional networks： a deep learning framework for traffic forecasting［C］// Proceedings of the 27th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2018： 3634-3640. 10.24963/ijcai.2018/505
16	LIN H， BAI R， JIA W， et al. Preserving dynamic attention for long-term spatial-temporal prediction［C］// Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2020： 36-46. 10.1145/3394486.3403046
17	HOCHREITER S， SCHMIDHUBER J. Long short-term memory［J］. Neural Computation， 1997， 9（8）： 1735-1780. 10.1162/neco.1997.9.8.1735
18	SHI X， CHEN Z， WANG H， et al. Convolutional LSTM network： a machine learning approach for precipitation nowcasting［C］// Proceedings of the 28th International Conference on Neural Information Processing Systems — Volume 1. Cambridge： MIT Press， 2015： 802-810.
19	WANG Y， LONG M， WANG J， et al. PredRNN： recurrent neural networks for predictive learning using spatiotemporal LSTMs［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 879-888.
20	TREBING K， STAǸCZYK T， MEHRKANOON S. SmaAt-UNet： precipitation nowcasting using a small attention-UNet architecture［J］. Pattern Recognition Letters， 2021， 145： 178-186. 10.1016/j.patrec.2021.01.036
21	RONNEBERGER O， FISCHER P， BROX T. U-net： convolutional networks for biomedical image segmentation［C］// Proceedings of the 2015 International Conference on Medical Image Computing and Computer-Assisted Intervention， LNCS 9351. Cham： Springer， 2015： 234-241.
22	廖雪超，伍杰平，陈才圣. 结合注意力机制与LSTM的短期风电功率预测模型［J］. 计算机工程， 2022， 48（9）：286-297， 304. 10.19678/j.issn.1000-3428.0062059
	LIAO X C， WU J P， CHEN C S. Short-term wind power prediction model combining attention mechanism and LSTM［J］. Computer Engineering， 2022， 48（9）： 286-297， 304. 10.19678/j.issn.1000-3428.0062059
23	WILSON Y， TAN P N， LUO L. A low rank weighted graph convolutional approach to weather prediction［C］// Proceedings of the 2018 IEEE International Conference on Data Mining. Piscataway： IEEE， 2018： 627-636. 10.1109/icdm.2018.00078
24	SEO S， MOHEGH A， BAN-WEISS G. with recurrent neural networks for spatiotemporal forecasting［C］// Proceedings of the 7th International Workshop on Climate Informatics. Boulder， CO： National Center for Atmospheric Research in Boulder， 2017： 85-88.
25	祁柏林，郭昆鹏，杨彬，等. 基于GCN-LSTM的空气质量预测［J］. 计算机系统应用， 2021， 30（3）： 208-213. 10.15888/j.cnki.csa.007815
	QI B L， GUO K P， YANG B， et al. Air quality prediction based on GCN-LSTM［J］. Computer Systems and Applications， 2021， 30（3）： 208-213. 10.15888/j.cnki.csa.007815
26	XU K， HU W， LESKOVEC J， et al. How powerful are graph neural networks？［EB/OL］. （2019-02-22）［2022-11-11］..
27	KIPF T N， WEWLLING M. Semi-supervised classification with graph convolutional networks［EB/OL］. （2017-02-22）［2022-11-11］.. 10.48550/arXiv.1609.02907
28	GAO H， WANG Z， JI S. Large-scale learnable graph convolutional networks［C］// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York： ACM， 2018： 1416-1424. 10.1145/3219819.3219947
29	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 6000-6010.
30	VELIČKOVIĆ P， CUCURULL G， CASANOVA A， et al. Graph attention networks［EB/OL］. （2018-02-04）［2021-11-12］..
31	BRONSTEIN M M， BRUNA J， LeCUN Y， et al. Geometric deep learning： going beyond Euclidean data［J］. IEEE Signal Processing Magazine， 2017， 34（4）： 18-42. 10.1109/msp.2017.2693418
32	ZHOU J， CUI G， HU S， et al. Graph neural networks： a review of methods and applications［J］. AI Open， 2020， 1： 57-81. 10.1016/j.aiopen.2021.01.001
33	NIEPERT M， AHMED M， KUTZKOV K. Learning convolutional neural networks for graphs［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： JMLR.org， 2016： 2014-2023.
34	BRUNA J， ZAREMBA W， SZLAM A， et al. Spectral networks and locally connected networks on graphs［EB/OL］. （2014-03-21）［2022-11-12］.. 10.1017/cbo9780511761942.003
35	DEFFERRARD M， BRESSON X， VANDERGHEYNST P. Convolutional neural networks on graphs with fast localized spectral filtering［C］// Proceedings of the 30th International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2016： 3844-3852.
36	Meteorological Development Laboratory/Office of Science and Technology/National Weather Service/NOAA/U.S. Department of Commerce. TDL U.S. and Canada surface hourly observations［DB/OL］. ［2022-11-12］..
37	YAO H， TANG X， WEI H， et al. Revisiting spatial-temporal similarity： a deep learning framework for traffic prediction［C］// AAAI’19/IAAI’19/EAAI’19： Proceedings of the 33rd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2019： 5668-5675. 10.1609/aaai.v33i01.33015668
38	ZHENG C， FAN X， WANG C， et al. GMAN： a graph multi-attention network for traffic prediction［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 1234-1241. 10.1609/aaai.v34i01.5477

网络模块	模块层数
网络模块	1	2	3	4	5	6
n_Decoder	1.38	1.37	1.23	1.40	1.55	1.57
n_Encoder	1.51	1.48	1.23	3.48	2.80	1.66
n_Layer	1.46	1.30	1.23	1.42	1.34	1.70
网络模块	注意力头数
网络模块	1	2	4	8	16	32
n_Head	1.43	1.33	1.34	1.23	1.31	1.37

网络模块	模块层数
网络模块	1	2	3	4	5	6
n_Decoder	1.38	1.37	1.23	1.40	1.55	1.57
n_Encoder	1.51	1.48	1.23	3.48	2.80	1.66
n_Layer	1.46	1.30	1.23	1.42	1.34	1.70
网络模块	注意力头数
网络模块	1	2	4	8	16	32
n_Head	1.43	1.33	1.34	1.23	1.31	1.37

时长/h	模型	RMSE	MAE	时长/h	模型	RMSE	MAE	时长/h	模型	RMSE	MAE
1	ConvLSTM	1.73	1.36	6	ConvLSTM	2.18	1.71	12	ConvLSTM	2.97	2.31
	STDN	1.70	1.31		STDN	2.12	1.70		STDN	2.34	1.76
	GMAN	1.74	1.38		GMAN	2.08	1.57		GMAN	2.29	1.75
	STGCN	1.50	1.16		STGCN	2.13	1.70		STGCN	2.95	2.30
	Xgboost	1.36	1.04		Xgboost	1.93	1.48		Xgboost	2.19	1.72
	DSAN	1.28	0.95		DSAN	2.07	1.52		DSAN	2.49	1.97
	Graph⁃DSAN	1.23	0.89		Graph⁃DSAN	1.85	1.43		Graph⁃DSAN	2.13	1.71

时长/h	模型	RMSE	MAE	时长/h	模型	RMSE	MAE	时长/h	模型	RMSE	MAE
1	ConvLSTM	1.73	1.36	6	ConvLSTM	2.18	1.71	12	ConvLSTM	2.97	2.31
	STDN	1.70	1.31		STDN	2.12	1.70		STDN	2.34	1.76
	GMAN	1.74	1.38		GMAN	2.08	1.57		GMAN	2.29	1.75
	STGCN	1.50	1.16		STGCN	2.13	1.70		STGCN	2.95	2.30
	Xgboost	1.36	1.04		Xgboost	1.93	1.48		Xgboost	2.19	1.72
	DSAN	1.28	0.95		DSAN	2.07	1.52		DSAN	2.49	1.97
	Graph⁃DSAN	1.23	0.89		Graph⁃DSAN	1.85	1.43		Graph⁃DSAN	2.13	1.71

方法	预测时长为1 h		预测时长为12 h
方法	RMSE	MAE	RMSE	MAE
DSAN-NS	1.31	0.98	2.42	1.92
DSAN-SS	1.41	1.06	2.60	2.00
DSAN-NE	1.48	1.10	2.45	1.97
DSAN-ND	1.42	1.07	2.74	2.15
Graph⁃DSAN	1.23	0.89	2.13	1.71

Multi-site wind speed prediction based on graph dynamic attention network

基于图动态注意力网络的多站点风速预测

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 13

References 38

Related Articles 15

Recommended Articles

Metrics

[1]	Guixiang XUE, Hui WANG, Weifeng ZHOU, Yu LIU, Yan LI. Port traffic flow prediction based on knowledge graph and spatio-temporal diffusion graph convolutional network [J]. Journal of Computer Applications, 2024, 44(9): 2952-2957.
[2]	Jing QIN, Zhiguang QIN, Fali LI, Yueheng PENG. Diagnosis of major depressive disorder based on probabilistic sparse self-attention neural network [J]. Journal of Computer Applications, 2024, 44(9): 2970-2974.
[3]	Liting LI, Bei HUA, Ruozhou HE, Kuang XU. Multivariate time series prediction model based on decoupled attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2732-2738.
[4]	Zhiqiang ZHAO, Peihong MA, Xinhong HEI. Crowd counting method based on dual attention mechanism [J]. Journal of Computer Applications, 2024, 44(9): 2886-2892.
[5]	Chuanlin PANG, Rui TANG, Ruizhi ZHANG, Chuan LIU, Jia LIU, Shibo YUE. Distributed power allocation algorithm based on graph convolutional network for D2D communication systems [J]. Journal of Computer Applications, 2024, 44(9): 2855-2862.
[6]	Chunxue ZHANG, Liqing QIU, Cheng’ai SUN, Caixia JING. Purchase behavior prediction model based on two-stage dynamic interest recognition [J]. Journal of Computer Applications, 2024, 44(8): 2365-2371.
[7]	Kaipeng XUE, Tao XU, Chunjie LIAO. Multimodal sentiment analysis network with self-supervision and multi-layer cross attention [J]. Journal of Computer Applications, 2024, 44(8): 2387-2392.
[8]	Pengqi GAO, Heming HUANG, Yonghong FAN. Fusion of coordinate and multi-head attention mechanisms for interactive speech emotion recognition [J]. Journal of Computer Applications, 2024, 44(8): 2400-2406.
[9]	Yuhan LIU, Genlin JI, Hongping ZHANG. Video pedestrian anomaly detection method based on skeleton graph and mixed attention [J]. Journal of Computer Applications, 2024, 44(8): 2551-2557.
[10]	Zhonghua LI, Yunqi BAI, Xuejin WANG, Leilei HUANG, Chujun LIN, Shiyu LIAO. Low illumination face detection based on image enhancement [J]. Journal of Computer Applications, 2024, 44(8): 2588-2594.
[11]	Shangbin MO, Wenjun WANG, Ling DONG, Shengxiang GAO, Zhengtao YU. Single-channel speech enhancement based on multi-channel information aggregation and collaborative decoding [J]. Journal of Computer Applications, 2024, 44(8): 2611-2617.
[12]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[13]	Huanhuan LI, Tianqiang HUANG, Xuemei DING, Haifeng LUO, Liqing HUANG. Public traffic demand prediction based on multi-scale spatial-temporal graph convolutional network [J]. Journal of Computer Applications, 2024, 44(7): 2065-2072.
[14]	Dianhui MAO, Xuebo LI, Junling LIU, Denghui ZHANG, Wenjing YAN. Chinese entity and relation extraction model based on parallel heterogeneous graph and sequential attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2018-2025.
[15]	Li LIU, Haijin HOU, Anhong WANG, Tao ZHANG. Generative data hiding algorithm based on multi-scale attention [J]. Journal of Computer Applications, 2024, 44(7): 2102-2109.