关键词: 关键词: 视频语义分割, 多尺度特征融合, 时序一致性, 运动补偿, 特征对齐

Abstract: To address the challenges faced by current video semantic segmentation techniques in complex outdoor dynamic scenes, such as the lack of temporal multi-frame consistency and feature misalignment between adjacent frames, we propose a novel video semantic segmentation framework called STDA-Net (Spatiotemporal and Temporal Dynamics Alignment Network). In dynamic video analysis, the absence of temporal feature consistency leads to discontinuous segmentation results across adjacent frames, causing abrupt changes that compromise the semantic stability of dynamic scenes. Meanwhile, feature misalignment across multiple frames is often affected by occlusions, lighting variations, and other disturbances, making precise alignment difficult. This misalignment introduces noise during feature fusion, thereby reducing segmentation accuracy.To mitigate these issues, STDA-Net incorporates two key modules: the Multi-Frame Dynamic Feature Aggregation Module (MFDA) and the Temporal Affine Motion Enhancement Module (TACFM). The MFDA module enhances inter-frame feature consistency by leveraging multi-scale feature extraction, channel self-attention mechanisms, and a multi-frame multi-scale wavelet fusion strategy, effectively reducing abrupt changes in segmentation results and ensuring stability in dynamic scenes. The TACFM module integrates affine motion compensation techniques to precisely align non-keyframe features and strengthens edge structure perception using motion displacement information, improving segmentation accuracy in rapidly changing environments.Extensive experiments on the VSPW dataset and a rail transit power grid surveillance video dataset demonstrate that STDA-Net significantly improves segmentation accuracy and temporal consistency. On VSPW, STDA-Net achieves a VmIoU of 40.5% and an mVC of 86.4%, outperforming the baseline TCBst-ppm by 4.0% and 1.1%, and surpassing CFFM by 1.3% and 1.2%, respectively. On the rail transit dataset, STDA-Net attains a VmIoU of 67.5% and an mVC of 91.7%, with improvements of 4.1% and 6.2% over TCBst-ppm, and 2.3% and 6.4% over CFFM. These results strongly validate the effectiveness of STDA-Net in complex dynamic environments, demonstrating significant improvements in both segmentation accuracy and consistency. The framework exhibits strong adaptability and robustness, making it well-suited for applications in rail transit and other challenging dynamic scenarios.

Key words: video semantic segmentation, multi-scale feature fusion, temporal consistency, motion compensation, feature alignment