Journal of Computer Applications ›› 2026, Vol. 46 ›› Issue (5): 1499-1506.DOI: 10.11772/j.issn.1001-9081.2025050628

• Data science and technology • Previous Articles    

Long time series prediction based on hybrid self-attention and differentiated normalization

Ruirui SONG, Leichun WANG(), Yunping HE, Jinxiang WEI, Xiangfeng LU, Xiaomeng LIU   

  1. School of Computer Science,Huber University,Wuhan Hubei 430062,China
  • Received:2025-06-06 Revised:2025-10-21 Accepted:2025-11-04 Online:2025-11-12 Published:2026-05-10
  • Contact: Leichun WANG
  • About author:SONG Ruirui, born in 1999, M. S. candidate. Her research interests include time series prediction, deep learning.
    HE Yunping, born in 2000, M. S. candidate. His research interests include multimodal sentiment analysis, time series.
    WEI Jinxiang, born in 2000, M. S. candidate. Her research interests include deep learning, spatio-temporal data prediction.
    LU Xiangfeng, born in 2000, M. S. candidate. Her research interests include multimodal classification detection, deep learning.
    LIU Xiaomeng, born in 2001, M. S. candidate. Her research interests include deep learning, multimodality.
  • Supported by:
    National Natural Science Foundation of China(62106069);National Social Science Foundation of China(24BTQ019)

基于混合自注意力和差异归一化的长时间序列预测

宋芮芮, 王雷春(), 何运平, 魏金香, 卢祥凤, 刘小萌   

  1. 湖北大学 计算机学院,武汉 430062
  • 通讯作者: 王雷春
  • 作者简介:宋芮芮(1999—),女,山东枣庄人,硕士研究生,主要研究方向:时间序列预测、深度学习
    何运平(2000—),男,湖北荆州人,硕士研究生,主要研究方向:多模态情感分析、时间序列
    魏金香(2000—),女,安徽阜阳人,硕士研究生,主要研究方向:深度学习、时空数据预测
    卢祥凤(2000—),女,山东临沂人,硕士研究生,主要研究方向:多模态分类检测、深度学习
    刘小萌(2001—),女,山东枣庄人,硕士研究生,主要研究方向:深度学习、多模态。
  • 基金资助:
    国家自然科学基金资助项目(62106069);国家自然科学基金资助项目(62102136);国家社会科学基金资助项目(24BTQ019)

Abstract:

Aiming at the problems of error accumulation, modeling difficulty, and low computational efficiency in long time series prediction, a long time series prediction model based on hybrid self-attention and differentiated normalization, namely HSADN (Hybrid Self-Attention and Differentiated Normalization), was proposed. Firstly, the model used a stacked multi-head self-attention mechanism in the encoder to capture long-distance dependencies in time series, thereby reducing computational complexity, and used a multi-layer sparse self-attention mechanism in the decoder to dynamically adjust the generation strategy. Secondly, in the encoder, Batch Channel Normalization (BCN) was used to extract, fuse, and reconstruct the features, while in the decoder, Layer Normalization (LN) was adopted to alleviate the gradient vanishing and improve the training stability, generating predicted sequence values. Experimental results show that compared with CALF (Cross-modAl Large Language Model Fine-tuning) model, HSADN has the Mean Squared Error (MSE) and Mean Absolute Error (MAE) of univariate prediction reduced by 6.2% and 6.9% on ECL-960, respectively, and by 13.1% and 2.9% on ETTh-720, respectively; the MSE and MAE of multivariate prediction reduced by 3.5% and 2.6% on ETTm-672, respectively, and by 1.8% and 0.9% on Weather-720, respectively; the running time for univariate and multivariate predictions reduced by an average of 4.6% and 28.7%, respectively.

Key words: long time series prediction, self-attention mechanism, normalization, encoder, decoder, robustness

摘要:

针对长时间序列预测中存在的误差积累、建模困难和计算效率低的问题,提出一种基于混合自注意力和差异归一化的长时间序列预测模型HSADN(Hybrid Self-Attention and Differentiated Normalization)。首先,模型使用堆叠多头自注意力机制捕捉编码器中时间序列的长距离依赖,降低计算复杂度,并使用多层稀疏自注意力机制动态调整解码器中的生成策略;其次,在编码器中通过批量通道归一化(BCN)对特征进行提取、融合和重构,在解码器中通过层归一化(LN)缓解梯度消失和提升训练稳定性,输出预测序列值。实验结果表明,与CALF(Cross-modAl Large Language Model Fine-tuning)模型相比,HSADN的单变量预测的均方误差(MSE)与平均绝对误差(MAE)在ECL-960上分别降低6.2%和6.9%,在ETTh-720上分别降低13.1%和2.9%;多变量预测的MSE和MAE在ETTm-672上分别降低3.5%和2.6%,在Weather-720上分别降低1.8%和0.9%;在单变量和多变量预测时的运行时间分别平均降低4.6%和28.7%。

关键词: 长时间序列预测, 自注意力机制, 归一化, 编码器, 解码器, 鲁棒性

CLC Number: