关键词: 三维重建, 模型修复, 智能识别, 深度图优化, 弱纹理补全

Abstract:

In vision-based 3D reconstruction technology， camera pose information is calculated and sparse point clouds are extracted through image feature matching， and then multi-view fine-grained matching technology was used to reconstruct the 3D model of the object surface in real environment. However， for special areas such as weak texture areas and areas without texture， it is difficult to extract feature information by the 3D reconstruction technology， which leads to the phenomenon of void in dense reconstruction， poor visual effect and limited practical value of 3D reconstruction results. To address the issues of poor reconstruction effects in weak texture areas， and later-stage complex and labor-intensive manual model repair of traditional photogrammetric 3D reconstruction methods， an AI intelligent recognition-based method for 3D reconstruction model repair method for weak texture area was proposed. Firstly， a visual transformer attention mechanism with bi-level routing attention — BiFormer， a boundary box similarity metric based on minimum point distance — MPDIoU （Minimum Point Distance Intersection over Union）， and a genetic programming automatic discovery neural network optimizer Lion were added to the YOLOv8 segmentation network to optimize and improve the YOLOv8 network， that was to increase the network’s accuracy in weak texture recognition， which was used for training and intelligent reasoning in weak texture scenes. Then， based on the intelligently recognized weak texture area mask， the PatchMatch dense depth map was completed and optimized with weak textures， so as to complete the point cloud of weak texture areas in the entire scene automatically and generate a flattened weak texture area model. Without any later-stage manual intervention， high-integrity and high-quality 3D model results were able to be obtained. Experimental results show that the proposed method improves the efficiency of weak texture recognition by 15%， the recognition precision by 10%， and makes the triangular mesh holes in weak texture areas fully completed. The above verifies the effectiveness of the proposed method in repairing scenes in weak texture areas.

Key words: 3D reconstruction, model repair, intelligent recognition, depth map optimization, weak texture completion