关键词: 三维视觉定位, 单目图像, 多模态技术, 目标检测, 场景理解

Abstract: In view of the problems that existed 3D visual grounding methods that relied on expensive sensor equipment, incurred high system costs, and exhibited poor accuracy and robustness in complex multi-target grounding scenarios, a multi-target 3D visual grounding method based on monocular images was proposed. This method combined natural language descriptions to achieve the recognition of multiple 3D targets from a single RGB image. To this end, a multi-target visual grounding dataset was constructed and a cross-modal matching network, TextVizNet, was designed. TextVizNet generated 3D bounding boxes for targets by means of a pre-trained monocular detector and deeply integrated visual and linguistic information via an information fusion module and an information alignment module, thereby realizing text-guided multi-target 3D detection. Experimental results show that compared with existed advanced methods, TextVizNet improves the F1-score, Precision, and Recall by 8.92%, 8.39%, and 9.57%, respectively on the Mmo3DRefer dataset compared with the second-place method, significantly improving the accuracy of text-based multi-target localization in complex scenarios, providing effective support for practical applications such as autonomous driving and intelligent robotics.

Key words: 3D visual grounding, monocular image, multimodal technology, object detection, scene understanding