关键词: 三维目标检测, 跨模态融合, Mamba, 鸟瞰视图, 自动驾驶

Abstract: To address the issue that existing cross-modal fusion methods based on Bird's-Eye View( BEV）space neglect effective preservation of local BEV feature information in the initial fusion stage, leading to insufficient shallow cross-modal interactions, constrain subsequent deep fusion, and reduced accuracy in 3D object detection, a hierarchical cross-modal fusion method based on Mamba model was proposed. Cross-modal features were mapped into a hidden state space to facilitate interactions, thereby enriching local information, reducing discrepancies among modal features, and enhancing the consistency of fused feature representations. In the shallow fusion stage, a feature-channel exchange mechanism was designed to exchange feature channels from different sensor modalities, improving the preservation of shallow-level details. Subsequently, the visual state space block of the Mamba model was improved to strengthen effective interactions among shallow features and enhance feature representation quality. In the deep fusion stage, an attention mechanism was introduced to compute inter-modal difference features, which, combined with a gating mechanism, identified and fused complementary long-range dependencies among modalities. Finally, a channel-adaptive module was employed to calculate channel attention on normalized original features, adaptively learning intra-modal channel relationships to enhance single-modal BEV feature representations, thus compensating for the Mamba model’s inherent limitation in modeling inter-channel relationships. Experimental results show that the proposed method achieves superior detection performance compared to methods such as TransFusion and LoGoNet on the nuScenes and Waymo datasets. On the nuScenes test set, it attains a mean Average Precision (mAP) of 72.4% and a nuScenes Detection Score (NDS) of 73.9%, surpassing the baseline BEVFusion_mit by 2.2 and 1.0 percentage points, respectively.

Key words: 3D object detection, cross-modal fusion, Mamba, Bird's-Eye View (BEV), autonomous driving