Point cloud classification and segmentation based on Siamese adaptive graph convolution algorithm

doi:10.11772/j.issn.1001-9081.2022101552

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (11): 3396-3402.DOI: 10.11772/j.issn.1001-9081.2022101552

Special Issue: 人工智能

• Artificial intelligence • Previous Articles Next Articles

Point cloud classification and segmentation based on Siamese adaptive graph convolution algorithm

Weigang LI¹^,², Ting CHEN¹(), Zhiqiang TIAN¹

^1.School of Information Science and Engineering，Wuhan University of Science and Technology，Wuhan Hubei 430081，China
^2.Engineering Research Center for Metallurgical Automation and Measurement Technology of Ministry of Education;（Wuhan University of Science and Technology），Wuhan Hubei 430081，China

Received:2022-10-20 Revised:2023-02-03 Accepted:2023-02-08 Online:2023-04-12 Published:2023-11-10
Contact: Ting CHEN
About author:LI Weigang， born in 1977， Ph. D.， professor. His research interests include industrial process control， artificial intelligence， machine learning.
CHEN Ting， born in 1999， M. S. candidate. Her research interests include deep learning， pattern recognition， point cloud data processing.
TIAN Zhiqiang， born in 1996， Ph. D. candidate. His research interests include computer vision.
Supported by:
Key Research and Development Program of Hubei Province(2020BAB098)

基于孪生自适应图卷积算法的点云分类与分割

李维刚¹^,², 陈婷¹(), 田志强¹

^1.武汉科技大学信息科学与工程学院，武汉 430081
^2.武汉科技大学冶金自动化与检测技术教育部工程研究中心，武汉 430081

通讯作者: 陈婷
作者简介:李维刚（1977—），男，湖北咸宁人，教授，博士，主要研究方向：工业过程控制、人工智能、机器学习
陈婷（1999—），女，湖北孝感人，硕士研究生，主要研究方向：深度学习、模式识别、点云数据处理 chenting_myself@163.com
田志强（1996—），男，湖北武汉人，博士研究生，主要研究方向：计算机视觉。
基金资助:
湖北省重点研发计划项目(2020BAB098)

Abstract

Abstract:

Point cloud data has sparsity， irregularity， and permutation invariance， and lacks topological information， which makes it difficult to extract features of point cloud. Therefore， a Siamese Adaptive Graph Convolution Algorithm （SAGCA） was proposed for point cloud classification and segmentation. Firstly， the topological relationships between irregular and sparse point cloud features were mined by constructing feature relationship graph. Then， the Siamese composition idea of sharing convolution learning weights was introduced to ensure the permutation invariance of point cloud data and make the topological relationship expression more accurate. Finally， SAGCA was combined with various deep learning networks for processing point cloud data by both global and local combination methods， thereby enhancing the feature extraction ability of the network. Comparison results with PointNet++ benchmark network of the classification， object part segmentation and scene semantic segmentation experiments on ScanObjectNN， ShapeNetPart and S3DIS datasets， respectively， show that， based on the same dataset and evaluation criteria， SAGCA has the class mean Accuracy （mAcc） of classification increased by 2.80 percentage points， the overall class average Intersection over Union （IoU） of part segmentation increased by 2.31 percentage points， and the class mean Intersection over Union （mIoU） of scene semantic segmentation increased by 2.40 percentage points， verifying that SAGCA can effectively enhance the feature extraction ability of the network and is suitable for multiple point cloud classification and segmentation tasks.

Key words: point cloud data, topological relationship, Siamese, adaptive graph convolution, classification, segmentation

摘要：

点云数据具有稀疏性、不规则性和置换不变性，缺乏拓扑信息，导致它的特征难以被提取，为此，提出一种孪生自适应图卷积算法（SAGCA）进行点云分类与分割。首先，构建特征关系图挖掘不规则、稀疏点云特征间的拓扑关系；其次，引入共享卷积学习权重的孪生构图思想，保证点云的置换不变性，使拓扑关系表达更准确；最后，采用整体、局部两种结合方式，将SAGCA与各种处理点云数据的深度学习网络相结合，增强网络的特征提取能力。分别在ScanObjectNN、ShapeNetPart和S3DIS数据集上进行分类、对象部件分割和场景语义分割实验的结果表明，相较于PointNet++基准网络，基于同样的数据集和评价标准，SAGCA分类实验的类别平均准确率（mAcc）提高了2.80个百分点，对象部件分割实验的总体类别平均交并比（IoU）提高了2.31个百分点，场景语义分割实验的类别平均交并比（mIoU）提高了2.40个百分点，说明SAGCA能有效增强网络的特征提取能力，适用于多种点云分类分割任务。

关键词: 点云数据, 拓扑关系, 孪生, 自适应图卷积, 分类, 分割

CLC Number:

TP391.4

Weigang LI, Ting CHEN, Zhiqiang TIAN. Point cloud classification and segmentation based on Siamese adaptive graph convolution algorithm[J]. Journal of Computer Applications, 2023, 43(11): 3396-3402.

李维刚, 陈婷, 田志强. 基于孪生自适应图卷积算法的点云分类与分割[J]. 《计算机应用》唯一官方网站, 2023, 43(11): 3396-3402.

Figures/Tables 10

References 31

1	李朝，兰海，魏宪. 基于注意力的毫米波-激光雷达融合目标检测［J］. 计算机应用， 2021， 41（7）： 2137-2144. 10.11772/j.issn.1001-9081.2020081334
	LI C， LAN H， WEI X. Attention-based object detection with millimeter wave radar-lidar fusion［J］. Journal of Computer Applications， 2021， 41（7）： 2137-2144. 10.11772/j.issn.1001-9081.2020081334
2	付豪，徐和根，张志明，等. 动态场景下基于语义和光流约束的视觉同步定位与地图构建［J］. 计算机应用， 2021， 41（11）： 3337-3344. 10.11772/j.issn.1001-9081.2021010003
	FU H， XU H G， ZHANG Z M， et al. Visual simultaneous localization and mapping based on semantic and optical flow constraints in dynamic scenes［J］. Journal of Computer Applications， 2021， 41（11）： 3337-3344. 10.11772/j.issn.1001-9081.2021010003
3	FAN T， ZHANG R. Research on automatic lane line extraction method based on onboard lidar point cloud data［C］// Proceedings of the SPIE 12306， 2nd International Conference on Digital Signal and Computer Communications. Bellingham， WA： SPIE， 2022： No.123060P. 10.1117/12.2641593
4	MIRZAEI K， ARASHPOUR M， ASADI E， et al. 3D point cloud data processing with machine learning for construction and infrastructure applications： a comprehensive review［J］. Advanced Engineering Informatics， 2022， 51： No.101501. 10.1016/j.aei.2021.101501
5	XU Y， FAN T， XU M， et al. SpiderCNN： deep learning on point sets with parameterized convolutional filters［C］// Proceedings of the 2018 European Conference on Computer Vision， LNCS 11212. Cham： Springer， 2018： 90-105.
6	LI Y， BU R， SUN M， et al. PointCNN： convolution on X-transformed points［C］// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2018： 828-838.
7	QIU S， ANWAR S， BARNES N. Dense-resolution network for point cloud classification and segmentation［C］// Proceedings of the 2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2021： 3812-3821. 10.1109/wacv48630.2021.00386
8	史怡，魏东，宋强，等. 基于动态图卷积和离散哈特莱转换差异性池化的点云数据分类分割网络［J］. 计算机应用， 2022， 42（S1）： 292-297.
	SHI Y， WEI D， SONG Q， et al. Point cloud data classification and segmentation network based on dynamic graph convolution and discrete Hartley transform different pooling［J］. Journal of Computer Applications， 2021， 42（S1）： 292-297.
9	SU H， JAMPANI V， SUN D， et al. SPLATNet： sparse lattice networks for point cloud processing［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 2530-2539. 10.1109/cvpr.2018.00268
10	YI L， SU H， GUO X， et al. SyncSpecCNN： synchronized spectral CNN for 3D shape segmentation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 6584-6592. 10.1109/cvpr.2017.697
11	PEYGHAMBARZADEH S M M， AZIZMALAYERI F， KHOTANLOU H， et al. Point-PlaneNet： plane kernel based convolutional neural network for point clouds analysis［J］. Digital Signal Processing， 2020， 98： No.102633. 10.1016/j.dsp.2019.102633
12	TCHAPMI L， CHOY C， ARMENI I， et al. SEGCloud： semantic segmentation of 3D point clouds［C］// Proceedings of the 2017 International Conference on 3D Vision. Piscataway： IEEE， 2017： 537-547. 10.1109/3dv.2017.00067
13	LIN Z H， HUANG S Y， WANG Y C F. Convolution in the cloud： learning deformable kernels in 3D graph convolution networks for point cloud analysis［C］// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2020： 1897-1806. 10.1109/cvpr42600.2020.00187
14	ZHOU Y， TUZEL O. VoxelNet： end-to-end learning for point cloud based 3D object detection［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 4490-4499. 10.1109/cvpr.2018.00472
15	SHESHAPPANAVAR S V， KAMBHAMETTU C. SimpleView++： neighborhood views for point cloud classification［C］// Proceedings of the IEEE 5th International Conference on Multimedia Information Processing and Retrieval. Piscataway： IEEE， 2022： 31-34. 10.1109/mipr54900.2022.00013
16	QI C R， SU H， MO K， et al. PointNet： deep learning on point sets for 3D classification and segmentation［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 77-85. 10.1109/cvpr.2017.16
17	QI C R， YI L， SU H， et al. PointNet++： deep hierarchical feature learning on point sets in a metric space［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 5105-5114.
18	WANG Y， SUN Y， LIU Z， et al. Dynamic graph CNN for learning on point clouds［J］. ACM Transactions on Graphics， 2019， 38（5）： No.146. 10.1145/3326362
19	QIAN G， LI Y， PENG H， et al. PointNeXt： revisiting PointNet++ with improved training and scaling strategies［EB/OL］. （2022-10-12）［2022-12-29］..
20	VASWANI A， SHAZEER N， PARMAR N， et al. Attention is all you need［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 6000-6010.
21	RAN H， LIU J， WANG C. Surface representation for point clouds［C］// Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2022： 18942-18952. 10.1109/cvpr52688.2022.01837
22	SCARSELLI F， GORI M， TSOI A C， et al. The graph neural network model［J］. IEEE Transactions on Neural Networks， 2008， 20（1）： 61-80. 10.1109/tnn.2008.2005605
23	KIPF T N， WELLING M. Semi-supervised classification with graph convolutional networks［EB/OL］. （2017-02-22）［2022-08-24］.. 10.48550/arXiv.1609.02907
24	LI G， MÜLLER M， QIAN G， et al. DeepGCNs： making GCNs go as deep as CNNs［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2023， 45（6）： 6923-6939. 10.1109/tpami.2021.3074057
25	SHI L， ZHANG Y， CHENG J， et al. Non-local graph convolutional networks for skeleton-based action recognition［EB/OL］. ［2022-08-24］.. 10.1109/cvpr.2019.01230
26	UY M A， PHAM Q H， HUA B S， et al. Revisiting point cloud classification： a new benchmark dataset and classification model on real-world data［C］// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway： IEEE， 2019： 1588-1597. 10.1109/iccv.2019.00167
27	CHANG A X， FUNKHOUSER T， GUIBAS L， et al. ShapeNet： an information-rich 3D model repository［EB/OL］. ［2022-08-25］..
28	ARMENI I， SENER O， ZAMIR A R， et al. 3D semantic parsing of large-scale indoor spaces［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 1534-1543. 10.1109/cvpr.2016.170
29	PASZKE A， GROSS S， CHINTALA S， et al. Automatic differentiation in PyTorch［EB/OL］. ［2023-01-01］..
30	PANG Y， WANG W， TAY F E H， et al. Masked autoencoders for point cloud self-supervised learning［C］// Proceedings of the 2022 European Conference on Computer Vision， LNCS 13662. Cham： Springer， 2022： 604-621.
31	MA X， QIN C， YOU H， et al. Rethinking network design and local geometry in point cloud： a simple residual MLP framework［EB/OL］. ［2023-01-01］..

方法	OA/%	mAcc/%	GFLOPs	推理速度/（sample·s^-1）
PointNet^［16］	68.20	63.40	0.45	—
SpiderCNN^［5］	73.70	69.80	—	—
PointNet++^［17］	77.90	75.40	0.87	120.35^*
DGCNN^［18］	78.10	73.60	2.43	—
PointCNN^［6］	78.50	75.10	—	—
DRNet^［7］	80.30	78.00	—	—
MVTN+SimpleView++^［15］	84.80	—	—	—
Point-MAE^［30］	85.20	—	—	—
PointMLP^［31］	85.70	84.40	—	—
RepSurf-U^‡［21］	86.00	83.10	2.45	92.79^*
LSAGCA-PointNet++	79.89	78.20	3.71	70.94
GSAGCA-PointNet++	79.89	77.78	0.91	104.15
LSAGCA-RepSurf-U^‡	86.40	84.93	11.68	56.66
GSAGCA-RepSurf-U^‡	86.50	85.69	2.87	84.78

方法	OA/%	mAcc/%	GFLOPs	推理速度/（sample·s^-1）
PointNet^［16］	68.20	63.40	0.45	—
SpiderCNN^［5］	73.70	69.80	—	—
PointNet++^［17］	77.90	75.40	0.87	120.35^*
DGCNN^［18］	78.10	73.60	2.43	—
PointCNN^［6］	78.50	75.10	—	—
DRNet^［7］	80.30	78.00	—	—
MVTN+SimpleView++^［15］	84.80	—	—	—
Point-MAE^［30］	85.20	—	—	—
PointMLP^［31］	85.70	84.40	—	—
RepSurf-U^‡［21］	86.00	83.10	2.45	92.79^*
LSAGCA-PointNet++	79.89	78.20	3.71	70.94
GSAGCA-PointNet++	79.89	77.78	0.91	104.15
LSAGCA-RepSurf-U^‡	86.40	84.93	11.68	56.66
GSAGCA-RepSurf-U^‡	86.50	85.69	2.87	84.78

方法	IoU/%		GFLOPs	推理速度/（sample·s^-1）
方法	实例平均	类别平均	GFLOPs	推理速度/（sample·s^-1）
PointNet^［16］	83.70	—	4.10	—
SPLATNet 3D^［9］	84.60	82.00	—	—
SSCNN^［10］	84.70	82.00	—	—
3D-GCN^［24］	85.10	82.10	—	—
Point-PlaneNet^［11］	85.10	82.50	—	—
DGCNN^［18］	85.20	—	—	—
SpiderCNN^［5］	85.30	82.40	—	—
PointNet++（msg）^［17］	85.10	81.90	4.92^*	69.98^*
LSAGCA-PointNet++	85.42	84.08	15.06	51.07
GSAGCA-PointNet++	85.43	84.21	5.02	68.66

方法	IoU/%		GFLOPs	推理速度/（sample·s^-1）
方法	实例平均	类别平均	GFLOPs	推理速度/（sample·s^-1）
PointNet^［16］	83.70	—	4.10	—
SPLATNet 3D^［9］	84.60	82.00	—	—
SSCNN^［10］	84.70	82.00	—	—
3D-GCN^［24］	85.10	82.10	—	—
Point-PlaneNet^［11］	85.10	82.50	—	—
DGCNN^［18］	85.20	—	—	—
SpiderCNN^［5］	85.30	82.40	—	—
PointNet++（msg）^［17］	85.10	81.90	4.92^*	69.98^*
LSAGCA-PointNet++	85.42	84.08	15.06	51.07
GSAGCA-PointNet++	85.43	84.21	5.02	68.66

方法	mIoU/%	GFLOPs	推理速度/（sample·s^-1）
PointNet^［16］	41.10	4.10	—
SegCloud^［12］	48.90	—	—
DeepGCN^［25］	52.49	—	—
PointNet++^［17］	52.70^*	0.80^*	43.84^*
3D-GCN^［24］	53.40	—	—
LSAGCA-PointNet++	54.76	3.44	34.76
GSAGCA-PointNet++	55.10	1.14	41.28

Point cloud classification and segmentation based on Siamese adaptive graph convolution algorithm

基于孪生自适应图卷积算法的点云分类与分割

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 10

References 31

Related Articles 15

Recommended Articles

Metrics

方法	是否共享卷积的权重	mIoU
PointNet++基准网络	否	52.70^*
LAGCA	否	52.30
LAGCA	是	54.76
GAGCA	否	53.46
GAGCA	是	55.10

[1]	Yuxin HUANG, Jialong XU, Zhengtao YU, Shukai HOU, Jiaqi ZHOU. Unsupervised text sentiment transfer method based on generation prompt [J]. Journal of Computer Applications, 2024, 44(9): 2667-2673.
[2]	Ying HUANG, Jiayu YANG, Jiahao JIN, Bangrui WAN. Siamese mixed information fusion algorithm for RGBT tracking [J]. Journal of Computer Applications, 2024, 44(9): 2878-2885.
[3]	Chun SUN, Chunlong HU, Shucheng HUANG. Consistency preserving age estimation method by ensemble ranking [J]. Journal of Computer Applications, 2024, 44(8): 2381-2386.
[4]	Qiangkui LENG, Xuezi SUN, Xiangfu MENG. Oversampling method for imbalanced data based on sample potential and noise evolution [J]. Journal of Computer Applications, 2024, 44(8): 2466-2475.
[5]	Quanmei ZHANG, Runping HUANG, Fei TENG, Haibo ZHANG, Nan ZHOU. Automatic international classification of disease coding method incorporating heterogeneous information [J]. Journal of Computer Applications, 2024, 44(8): 2476-2482.
[6]	Chenqian LI, Jun LIU. Ultrasound carotid plaque segmentation method based on semi-supervision and multi-scale cascaded attention [J]. Journal of Computer Applications, 2024, 44(8): 2604-2610.
[7]	Junchi GE, Weihua ZHAO. Distance weighted discriminant analysis based on robust principal component analysis for matrix data [J]. Journal of Computer Applications, 2024, 44(7): 2073-2079.
[8]	Xun SUN, Ruifeng FENG, Yanru CHEN. Monocular 3D object detection method integrating depth and instance segmentation [J]. Journal of Computer Applications, 2024, 44(7): 2208-2215.
[9]	Qianhui LU, Yu ZHANG, Mengling WANG, Tingwei WU, Yuzhong SHAN. Classification model of nuclear power equipment quality text based on improved recurrent pooling network [J]. Journal of Computer Applications, 2024, 44(7): 2034-2040.
[10]	Zhe KONG, Han LI, Shaowei GAN, Mingru KONG, Bingtao HE, Ziyu GUO, Ducheng JIN, Zhaowen QIU. Structure segmentation model for 3D kidney images based on asymmetric multi-decoder and attention module [J]. Journal of Computer Applications, 2024, 44(7): 2216-2224.
[11]	Wu XIONG, Congjun CAO, Xuefang SONG, Yunlong SHAO, Xusheng WANG. Handwriting identification method based on multi-scale mixed domain attention mechanism [J]. Journal of Computer Applications, 2024, 44(7): 2225-2232.
[12]	Dongwei WANG, Baichen LIU, Zhi HAN, Yanmei WANG, Yandong TANG. Deep network compression method based on low-rank decomposition and vector quantization [J]. Journal of Computer Applications, 2024, 44(7): 1987-1994.
[13]	Xun YAO, Zhongzheng QIN, Jie YANG. Generative label adversarial text classification model [J]. Journal of Computer Applications, 2024, 44(6): 1781-1785.
[14]	Yan LI, Dazhi PAN, Siqing ZHENG. Improved adaptive large neighborhood search algorithm for multi-depot vehicle routing problem with time window [J]. Journal of Computer Applications, 2024, 44(6): 1897-1904.
[15]	Zixuan YUAN, Xiaoqing WENG, Ningzhen GE. Early classification model of multivariate time series based on orthogonal locality preserving projection and cost optimization [J]. Journal of Computer Applications, 2024, 44(6): 1832-1841.