Abstract:

In vision-based 3D reconstruction technology， camera pose information is calculated and sparse point clouds are extracted through image feature matching， and then multi-view fine-grained matching technology was used to reconstruct the 3D model of the object surface in real environment. However， for special areas such as weak texture areas and areas without texture， it is difficult to extract feature information by the 3D reconstruction technology， which leads to the phenomenon of void in dense reconstruction， poor visual effect and limited practical value of 3D reconstruction results. To address the issues of poor reconstruction effects in weak texture areas， and later-stage complex and labor-intensive manual model repair of traditional photogrammetric 3D reconstruction methods， an AI intelligent recognition-based method for 3D reconstruction model repair method for weak texture area was proposed. Firstly， a visual transformer attention mechanism with bi-level routing attention — BiFormer， a boundary box similarity metric based on minimum point distance — MPDIoU （Minimum Point Distance Intersection over Union）， and a genetic programming automatic discovery neural network optimizer Lion were added to the YOLOv8 segmentation network to optimize and improve the YOLOv8 network， that was to increase the network’s accuracy in weak texture recognition， which was used for training and intelligent reasoning in weak texture scenes. Then， based on the intelligently recognized weak texture area mask， the PatchMatch dense depth map was completed and optimized with weak textures， so as to complete the point cloud of weak texture areas in the entire scene automatically and generate a flattened weak texture area model. Without any later-stage manual intervention， high-integrity and high-quality 3D model results were able to be obtained. Experimental results show that the proposed method improves the efficiency of weak texture recognition by 15%， the recognition precision by 10%， and makes the triangular mesh holes in weak texture areas fully completed. The above verifies the effectiveness of the proposed method in repairing scenes in weak texture areas.

Key words: 3D reconstruction, model repair, intelligent recognition, depth map optimization, weak texture completion

摘要：

基于视觉的三维重建技术通过图像特征匹配计算相机的姿态信息以及提取稀疏点云，然后利用多视图精细化匹配技术重建真实环境中物体表面的三维模型，然而对于特殊区域如弱纹理区域和无纹理区域，三维重建技术很难提取特征信息，这导致在稠密重建中出现空洞现象，三维重建结果的可视效果差且实用价值受限。针对传统摄影测量三维重建方法在弱纹理区域的重建效果差、后期人工修模工序复杂且工作量大的问题，提出一种基于AI智能识别的三维重建弱纹理区域模型修复方法。首先，向YOLOv8分割网络中添加具有双水平路由注意的视觉变压器注意力机制BiFormer、基于最小点距离的边界框相似度比较度量MPDIoU（Minimum Point Distance Intersection over Union）和遗传编程自动发现神经网络优化器Lion，以优化改进YOLOv8网络，即提高网络对弱纹理的识别精度，用于弱纹理场景的训练与智能推理；其次，基于智能识别的弱纹理区域掩膜，对PatchMatch稠密化的深度图进行弱纹理填补和优化，从而自动补全场景弱纹理区域的点云，并生成整平的弱纹理区域模型，且无需任何后期的人工干预就能得到高完整度、高质量的三维模型成果。实验结果表明，所提方法的弱纹理识别效率提升了15%，识别精度提升了10%，且使弱纹理区域三角网空洞填补完整，验证了所提方法针对弱纹理区域场景修复的有效性。

关键词: 三维重建, 模型修复, 智能识别, 深度图优化, 弱纹理补全