关键词: 三维形貌重建, 深度学习, 有监督, 时空特征耦合, 深度图

Abstract: The essence of 3D shape reconstruction based on multi-depth-of-field image sequences is how to accurately capture the out-of-focus transfer information between image frames and the change of focus area of a single image frame. In response to the inability of existing models to effectively fuse global spatio-temporal information, Global Spatio-Temporal Feature Coupling (GSTFC) model was proposed to extract local and global spatio-temporal feature information of multi-depth-of-field image sequences, and retained the transition information of focus and defocus through the Depth Focus Volume （DFV）module. First, the 3D-ConvNeXt module and 3D downsampling convolutional layer were interspersed in the shrink path to capture multi-scale local spatio-temporal features. Meanwhile, the 3D-SwinTransformer module was added to the bottleneck module to capture the global correlations of local spatio-temporal features of multi-depth-of-field image sequences. Then, the local spatio-temporal features and global correlations were fused into global spatio-temporal. Finally, the deep focus volume extracted the sequence weight information by depth attention and retained the transition information of focus and defocus to get the final depth map. The effectiveness of the proposed model is experimentally verified, and the results show that the proposed model decreases the Root Mean Square Error (RMSE) index by 12.5% in the FoD500 dataset compared with the state-of-the-art AiFDepthNet(All-in-Focus Depth Net) deep learning model, and retains more depth-of-field transition relationships compared with the traditional RFVR-SFF(Robust Focus Volume Regularization in Shape from Focus) model.

Key words: 3D shape reconstruction, deep learning, supervised, spatio-temporal feature coupling, depth map