关键词: 视频动作识别, 时空特征, 局部运动, 时序建模, 注意力机制

Abstract: Due to the diversity and complexity of spatiotemporal features in videos, as well as the wide variability of actions across different speeds and scales, problems of insufficient capture of local motion details and inadequate modeling of long-range temporal dependencies are commonly encountered in existing action recognition methods. To address these issues, a video action recognition network based on the complementary modeling of local and long-range temporal information was proposed. The network was composed of the Secondary Fusion Motion Excitation (SFME) and the Temporal Aggregation Channel Excitation (TACE) modules. In the SFME module, the first-order and second-order differences between adjacent feature maps were computed and fused, and the fused weights were used to excite the channels of the original feature maps, so as to enhance the fine-grained extraction capability of multi-level motion features and to model local temporal information. In the TACE module, a hierarchical residual pyramid structure was constructed using a channel grouping strategy, which expanded the temporal receptive field and enhanced the learning ability of multi-scale features. Meanwhile, a Temporal Channel Attention (TCA) mechanism was designed to dynamically adjust the aggregated feature maps and optimize the weight allocation among temporal channels, thereby modeling long-range temporal information. Finally, the above complementary modules were integrated into a 2D residual network to realize end-to-end action recognition. On the Something-Something V1 and V2 benchmarks, using only RGB frames with a random 8-frame sampling strategy, the proposed network achieves Top-1 accuracies of 50.6% and 61.9%, respectively. With a 16-frame sampling strategy, the accuracies further increase to 54.1% and 65.6%, respectively. The experimental results demonstrate that the proposed network effectively captures both local motion details and long-range temporal dependencies, offering a new perspective for action recognition in complex temporal scenarios.

Key words: video action recognition, spatio-temporal features, local motion, temporal modeling, attention mechanism