3D point cloud classification and segmentation network based on Spider convolution
WANG Benjie1, NONG Liping2,3, ZHANG Wenhui1, LIN Jiming1, WANG Junyi1
1. School of Information and Communication, Guilin University of Electronic Technology, Guilin Guangxi 541004, China
2. School of Telecommunication Engineering, Xidian University, Xi’an Shaanxi 710071, China
3. College of Physical Science and Technology, Guangxi Normal University, Guilin Guangxi 541004, China
The traditional Convolutional Neural Network (CNN) cannot directly process point cloud data, and the point cloud data must be converted into a multi-view or voxelized grid, which leads to a complicated process and low point cloud recognition accuracy. Aiming at the problem, a new point cloud classification and segmentation network called Linked-Spider CNN was proposed. Firstly, the deep features of point cloud were extracted by adding more Spider convolution layers based on Spider CNN. Secondly, by introducing the idea of residual network, short links were added to every Spider convolution layer to form residual blocks. Thirdly, the output features of each layer of residual blocks were spliced and fused to form the point cloud features. Finally, the point cloud features were classified by three-layer fully connected layers or segmented by multiple convolution layers. The proposed network was compared with other networks such as PointNet, PointNet++ and Spider CNN on ModelNet40 and ShapeNet Parts datasets. The experimental results show that the proposed network can improve the classification accuracy and segmentation effect of point clouds, and it has faster convergence speed and stronger robustness.
王本杰, 农丽萍, 张文辉, 林基明, 王俊义. 基于Spider卷积的三维点云分类与分割网络[J]. 计算机应用, 2020, 40(6): 1607-1612.
WANG Benjie, NONG Liping, ZHANG Wenhui, LIN Jiming, WANG Junyi. 3D point cloud classification and segmentation network based on Spider convolution. Journal of Computer Applications, 2020, 40(6): 1607-1612.
1 JIANGC, LIM B, ZHANGS. Three-dimensional shape measurement using a structured light system with dual projectors [J]. Applied Optics, 2018, 57(14): 3983-3990.
2 HIRONAGAN, KIMURAT, MITSUDOT, et al. Proposal for an accurate TMS-MRI co-registration process via 3D laser scanning [J]. Neuroscience Research, 2019, 144: 30-39.
3 QIC R, LIUW, WUC, et al. Frustum PointNets for 3D object detection from RGB-D data [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 918-927.
4 ZHUY, MOTTAGHIR, KOLVEE, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning [C]// Proceedings of the 2017 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2017: 3357-3364.
5 ZHANGK, XIONGC, ZHANGW, et al. Environmental features recognition for lower limb prostheses toward predictive walking [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(3):465-476.
6 XUY, FANT, XUM, et al. SpiderCNN: deep learning on point sets with parameterized convolutional filters [C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11212. Cham: Springer, 2018: 90-105.
7 HEK, ZHANGX, RENS, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
8 牛辰庚,刘玉杰,李宗民,等.基于点云数据的三维目标识别和模型分割方法[J].图学学报,2019,40(2):274-281. NIUC G, LIUY J, LIZ M, et al. 3D object recognition and model segmentation based on point cloud data [J]. Journal of Graphics, 2019, 40(2): 274-281.
9 CHENX, MAH, WANJ, et al. Multi-view 3D object detection network for autonomous driving [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6526-6534.
10 KU J, MOZIFIANM, LEE J, et al. Joint 3D proposal generation and object detection from view aggregation [C]// Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2018: 1-8.
11 QIC R, SUH, NIEßNERM, et al. Volumetric and multi-view CNNs for object classification on 3D data [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 5648-5656.
12 WUZ, SONGS, KHOSLAA, et al. 3D ShapeNets: a deep representation for volumetric shapes [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1912-1920.
13 WANGP, LIUY, GUOY, et al. O-CNN: octree-based convolutional neural networks for 3D shape analysis [J]. ACM Transactions on Graphics, 2017, 36(4): Article No.72.
14 QIC R, SUH, MOK, et al. PointNet: deep learning on point sets for 3D classification and segmentation [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 77-85.
15 QIC R, YIL, SUH, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space [C]// Proceedings of the 31st Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 5099-5108.
16 WANGY, SUNY, LIUZ, et al. Dynamic graph CNN for learning on point clouds [EB/OL]. [2019-03-24]. https://arxiv.org/pdf/1801.07829v1.pdf.
17 LIY, BUR, SUNM, et al. PointCNN: convolution on X-transformed points [C]// Proceedings of the 32nd Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2018: 820-830.
18 XIEZ, CHENJ, PENGB. Point clouds learning with attention-based graph convolution networks [EB/OL]. [2019-05-31].https://arxiv.org/pdf/1905.13445.pdf.