3D point cloud classification and segmentation network based on Spider convolution

doi:10.11772/j.issn.1001-9081.2019101879

Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (6): 1607-1612.DOI: 10.11772/j.issn.1001-9081.2019101879

• Artificial intelligence • Previous Articles Next Articles

3D point cloud classification and segmentation network based on Spider convolution

WANG Benjie¹, NONG Liping^2,3, ZHANG Wenhui¹, LIN Jiming¹, WANG Junyi¹

1. School of Information and Communication, Guilin University of Electronic Technology, Guilin Guangxi 541004, China
2. School of Telecommunication Engineering, Xidian University, Xi’an Shaanxi 710071, China
3. College of Physical Science and Technology, Guangxi Normal University, Guilin Guangxi 541004, China

Received:2019-11-04 Revised:2019-12-22 Online:2020-06-10 Published:2020-06-18
Contact: ZHANG Wenhui, born in 1970, M. S., associate professor. Her research interests include computer graphics, computer animation.
About author:WANG Benjie, born in 1993, M. S. candidate. His research interests include point cloud recognition, deep learning.NONG Liping, born in 1985, Ph. D. candidate, lecturer. Her research interests include graph signal processing, geometric deep learning.ZHANG Wenhui, born in 1970, M. S., associate professor. Her research interests include computer graphics, computer animation.LIN Jiming, born in 1970, Ph. D., professor. His research interests include wireless communication, mobile communication.WANG Junyi, born in 1977, Ph. D., research fellow. His research interests include graph signal processing, deep learning, wireless network resource management.
Supported by:
National Natural Science Foundation of China (61966007), the Development Foundation of Key Laboratory of Cognitive Radio and Information Processing of Ministry of Education (CRKL180201), the Project of Guangxi Cooperative Innovation Center of Cloud Computing and Big Data (1716), the Leader Foundation of Guangxi Key Laboratory of Wireless Wideband Communication and Signal Processing (GXKL06180107, CRKL180106).

基于Spider卷积的三维点云分类与分割网络

王本杰¹, 农丽萍^2,3, 张文辉¹, 林基明¹, 王俊义¹

1.桂林电子科技大学信息与通信学院，广西桂林 541004
2.西安电子科技大学通信工程学院，西安 710071
3.广西师范大学物理科学与技术学院，广西桂林 541004

通讯作者: 张文辉(1970—)
作者简介:王本杰(1993—)，男，河南南阳人，硕士研究生，主要研究方向：点云识别、深度学习。农丽萍(1985—)，女，广西天等人，讲师，博士研究生，主要研究方向：图信号处理、几何深度学习。张文辉(1970—),女,湖南益阳人，副教授，硕士，主要研究方向：计算机图形学、计算机动画。林基明（1970—），男，四川三台人，教授，博士，主要研究方向：无线通信、移动通信。王俊义（1977—），男，河北邢台人，研究员，博士，主要研究方向：图信号处理、深度学习、无线网络资源管控。
基金资助:
国家自然科学基金资助项目（61966007）；认知无线电与信息处理教育部重点实验室开发基金资助项目（CRKL180201）；广西云计算与大数据协同创新中心项目（1716）；广西无线宽带通信与信号处理重点实验室主任基金资助项目（GXKL06180107,CRKL180106)。

Abstract

Abstract:

The traditional Convolutional Neural Network (CNN) cannot directly process point cloud data, and the point cloud data must be converted into a multi-view or voxelized grid, which leads to a complicated process and low point cloud recognition accuracy. Aiming at the problem, a new point cloud classification and segmentation network called Linked-Spider CNN was proposed. Firstly, the deep features of point cloud were extracted by adding more Spider convolution layers based on Spider CNN. Secondly, by introducing the idea of residual network, short links were added to every Spider convolution layer to form residual blocks. Thirdly, the output features of each layer of residual blocks were spliced and fused to form the point cloud features. Finally, the point cloud features were classified by three-layer fully connected layers or segmented by multiple convolution layers. The proposed network was compared with other networks such as PointNet, PointNet++ and Spider CNN on ModelNet40 and ShapeNet Parts datasets. The experimental results show that the proposed network can improve the classification accuracy and segmentation effect of point clouds, and it has faster convergence speed and stronger robustness.

Key words: Convolutional Neural Network (CNN), Spider convolution, point cloud classification and segmentation, residual block, robustness

摘要：

针对传统的卷积神经网络（CNN）不能直接处理点云数据，需先将点云数据转换为多视图或者体素化网格，导致过程复杂且点云识别精度低的问题，提出一种新型的点云分类与分割网络Linked-Spider CNN。首先，在Spider CNN基础上通过增加Spider卷积层数以获取点云深层次特征；其次，引入残差网络的思想在每层Spider卷积增加短连接构成残差块；然后，将每层残差块的输出特征进行拼接融合形成点云特征；最后，使用三层全连接层对点云特征进行分类或者利用多层卷积层对点云特征进行分割。在ModelNet40和ShapeNet Parts数据集上将所提网络与PointNet、PointNet++和Spider CNN等网络进行对比实验，实验结果表明，所提网络可以提高点云的分类精度和分割效果，说明该网络具有更快的收敛速度和更强的鲁棒性。

关键词: 卷积神经网络, Spider卷积, 点云分类与分割, 残差块, 鲁棒性

CLC Number:

TP 391.4

WANG Benjie, NONG Liping, ZHANG Wenhui, LIN Jiming, WANG Junyi. 3D point cloud classification and segmentation network based on Spider convolution[J]. Journal of Computer Applications, 2020, 40(6): 1607-1612.

王本杰, 农丽萍, 张文辉, 林基明, 王俊义. 基于Spider卷积的三维点云分类与分割网络[J]. 计算机应用, 2020, 40(6): 1607-1612.

References

1 JIANGC, LIM B, ZHANGS. Three-dimensional shape measurement using a structured light system with dual projectors [J]. Applied Optics, 2018, 57(14): 3983-3990.
2 HIRONAGAN, KIMURAT, MITSUDOT, et al. Proposal for an accurate TMS-MRI co-registration process via 3D laser scanning [J]. Neuroscience Research, 2019, 144: 30-39.
3 QIC R, LIUW, WUC, et al. Frustum PointNets for 3D object detection from RGB-D data [C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 918-927.
4 ZHUY, MOTTAGHIR, KOLVEE, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning [C]// Proceedings of the 2017 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2017: 3357-3364.
5 ZHANGK, XIONGC, ZHANGW, et al. Environmental features recognition for lower limb prostheses toward predictive walking [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(3):465-476.
6 XUY, FANT, XUM, et al. SpiderCNN: deep learning on point sets with parameterized convolutional filters [C]// Proceedings of the 2018 European Conference on Computer Vision, LNCS 11212. Cham: Springer, 2018: 90-105.
7 HEK, ZHANGX, RENS, et al. Deep residual learning for image recognition [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
8 牛辰庚,刘玉杰,李宗民,等.基于点云数据的三维目标识别和模型分割方法[J].图学学报,2019,40(2):274-281. NIUC G, LIUY J, LIZ M, et al. 3D object recognition and model segmentation based on point cloud data [J]. Journal of Graphics, 2019, 40(2): 274-281.
9 CHENX, MAH, WANJ, et al. Multi-view 3D object detection network for autonomous driving [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 6526-6534.
10 KU J, MOZIFIANM, LEE J, et al. Joint 3D proposal generation and object detection from view aggregation [C]// Proceedings of the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway: IEEE, 2018: 1-8.
11 QIC R, SUH, NIEßNERM, et al. Volumetric and multi-view CNNs for object classification on 3D data [C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 5648-5656.
12 WUZ, SONGS, KHOSLAA, et al. 3D ShapeNets: a deep representation for volumetric shapes [C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2015: 1912-1920.
13 WANGP, LIUY, GUOY, et al. O-CNN: octree-based convolutional neural networks for 3D shape analysis [J]. ACM Transactions on Graphics, 2017, 36(4): Article No.72.
14 QIC R, SUH, MOK, et al. PointNet: deep learning on point sets for 3D classification and segmentation [C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 77-85.
15 QIC R, YIL, SUH, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space [C]// Proceedings of the 31st Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 5099-5108.
16 WANGY, SUNY, LIUZ, et al. Dynamic graph CNN for learning on point clouds [EB/OL]. [2019-03-24]. https://arxiv.org/pdf/1801.07829v1.pdf.
17 LIY, BUR, SUNM, et al. PointCNN: convolution on X-transformed points [C]// Proceedings of the 32nd Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2018: 820-830.
18 XIEZ, CHENJ, PENGB. Point clouds learning with attention-based graph convolution networks [EB/OL]. [2019-05-31].https://arxiv.org/pdf/1905.13445.pdf.

[1]	SONG Zhongshan, LIANG Jiarui, ZHENG Lu, LIU Zhenyu, TIE Jun. Remote sensing scene classification based on bidirectional gated scale feature fusion [J]. Journal of Computer Applications, 2021, 41(9): 2726-2735.
[2]	LI Kangkang, ZHANG Jing. Multi-layer encoding and decoding model for image captioning based on attention mechanism [J]. Journal of Computer Applications, 2021, 41(9): 2504-2509.
[3]	ZHANG Yongbin, CHANG Wenxin, SUN Lianshan, ZHANG Hang. Detection method of domains generated by dictionary-based domain generation algorithm [J]. Journal of Computer Applications, 2021, 41(9): 2609-2614.
[4]	ZHAO Hong, KONG Dongyi. Chinese description of image content based on fusion of image feature attention and adaptive attention [J]. Journal of Computer Applications, 2021, 41(9): 2496-2503.
[5]	GAO Gong, YANG Hongyu, LIU Hong. 3D point cloud face recognition based on deep learning [J]. Journal of Computer Applications, 2021, 41(9): 2736-2740.
[6]	XU Jianglang, LI Linyan, WAN Xinjun, HU Fuyuan. Indoor scene recognition method combined with object detection [J]. Journal of Computer Applications, 2021, 41(9): 2720-2725.
[7]	WANG Hebing, ZHANG Chunmei. Facial landmark detection based on ResNeXt with asymmetric convolution and squeeze excitation [J]. Journal of Computer Applications, 2021, 41(9): 2741-2747.
[8]	CAO Yuhong, XU Hai, LIU Sun'ao, WANG Zixiao, LI Hongliang. Review of deep learning-based medical image segmentation [J]. Journal of Computer Applications, 2021, 41(8): 2273-2287.
[9]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[10]	HUANG Chengcheng, DONG Xiaoxiao, LI Zhao. Deep pipeline 5×5 convolution method based on two-dimensional Winograd algorithm [J]. Journal of Computer Applications, 2021, 41(8): 2258-2264.
[11]	ZENG Xiangyin, ZHENG Bochuan, LIU Dan. Detection of left and right railway tracks based on deep convolutional neural network and clustering [J]. Journal of Computer Applications, 2021, 41(8): 2324-2329.
[12]	GAO Qinquan, HUANG Bingcheng, LIU Wenzhe, TONG Tong. Bamboo strip surface defect detection method based on improved CenterNet [J]. Journal of Computer Applications, 2021, 41(7): 1933-1938.
[13]	TAN Daoqiang, ZENG Cheng, QIAO Jinxia, ZHANG Jun. Shadow detection method based on hybrid attention model [J]. Journal of Computer Applications, 2021, 41(7): 2076-2081.
[14]	WU Guangli, LI Leiting, GUO Zhenzhou, WANG Chengxiang. Video summarization generation model based on improved bi-directional long short-term memory network [J]. Journal of Computer Applications, 2021, 41(7): 1908-1914.
[15]	YANG Su, OUYANG Zhi, DU Nisuo. Unsupervised parallel hash image retrieval based on correlation distance [J]. Journal of Computer Applications, 2021, 41(7): 1902-1907.

3D point cloud classification and segmentation network based on Spider convolution

基于Spider卷积的三维点云分类与分割网络

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics