Abstract: To address the issue of insufficient temporal representation and inadequate feature aggregation in most existing skeleton-based action recognition algorithms, a skeleton-based action recognition method using dual-branch temporal convolution and multi-dimensional attention was developed. ,  Firstly, progressive cross-scale temporal convolution (PCST-Conv) was employed to extract local temporal dependencies of actions, which reduced model parameters while improving recognition performance. Secondly, a du-al-branch parallel temporal block (DBPT-Block) was utilized to capture both long- and short-term temporal dependencies effectively through a parallel temporal convolution architecture. Meanwhile, a cross-temporal adaptive fusion (CTAF) module was introduced to enhance the representation of key action frames through dynamic weight allocation, addressing the problem that traditional mul-ti-scale fusion tends to ignore temporal variations across different action moments. Finally, hybrid multi-dimensional attention (HMDA) was applied to efficiently aggregate multi-dimensional features and further optimize feature representation. Experimental results showed that the method achieved accuracies of 91.7% on the cross-subject (CS) benchmark and 96.0% on the cross-view (CV) benchmark on the NTU-RGB+D 60 dataset. On the NTU-RGB+D 120 dataset, 87.4% accuracy was achieved on the cross-subject (CS-120) benchmark and 88.3% on the cross-setup (SS-120) benchmark. Moreover, compared with existing mainstream methods, higher recognition accuracy was achieved with fewer parameters, demonstrating significant advantages in both accuracy and effi-ciency. 

Key words: action recognition, skeleton sequence, adaptive fusion, multi-dimensional attention, dual-branch temporal convolution

摘要： 针对目前大多数基于骨骼的人体动作识别算法存在时序表征能力不足与特征聚合不充分的问题，提出了一种使用双分支时间卷积和多维度注意力的骨骼动作识别方法。首先使用渐进跨尺度时间卷积（PCST-Conv）提取动作局部时间依赖，在减少模型参数量的同时也提高了模型识别的性能；其次，利用双分支并联时间块（DBPT-Block）有效捕捉长短程动作的时间依赖；同时引入跨时序自适应融合（CTAF）模块，通过动态权重分配强化关键动作帧的表征能力，解决了传统多尺度融合会忽略动作不同时刻的差异性问题；最后通过混合多维度注意力（HMDA）高效地聚合多维度的特征，进一步优化特征表达。实验结果表明，该方法在NTU-RGB+D 60数据集上取得了跨受试者基准（CS）91.7%和跨视角基准（CV）96.0%，在NTU-RGB+D 120数据集的交叉主题基准（CS-120）和交叉设置基准（SS-120）的识别准确率分别为87.4%和88.3%。此外，与现有主流方法相比，该方法以更少的参数量取得了更高的识别准确率，在精度与效率上均展现出显著优势。

关键词: 动作识别, 骨架序列, 自适应融合, 多维度注意力, 双分支时间卷积