Diversity represented deep subspace clustering algorithm

doi:10.11772/j.issn.1001-9081.2021122126

Journal of Computer Applications ›› 2023, Vol. 43 ›› Issue (2): 407-412.DOI: 10.11772/j.issn.1001-9081.2021122126

Special Issue: 数据科学与技术

• Data science and technology • Previous Articles Next Articles

Diversity represented deep subspace clustering algorithm

Zhifeng MA¹^,², Junyang YU¹^,², Longge WANG¹^,²()

^1.College of Software，Henan University，Kaifeng Henan 475004，China
^2.Henan Province Intelligent Data Processing Engineering Research Center （Henan University），Kaifeng Henan 475004，China

Received:2021-12-21 Revised:2022-07-04 Accepted:2022-07-15 Online:2022-09-23 Published:2023-02-10
Contact: Longge WANG
About author:MA Zhifeng， born in 1998， M. S. candidate. His research interests include image processing， deep learning.
YU Junyang， born in 1982， Ph. D.， associate professor. His research interests include distributed computing， artificial intelligence.
Supported by:
Key Scientific and Technological Project of Henan Province(212102210078)

多样性表示的深度子空间聚类算法

马志峰¹^,², 于俊洋¹^,², 王龙葛¹^,²()

^1.河南大学软件学院，河南开封 475004
^2.河南省智能数据处理工程研究中心（河南大学），河南开封 475004

通讯作者: 王龙葛
作者简介:马志峰（1998—），男，河南濮阳人，硕士研究生，主要研究方向：图像处理、深度学习
于俊洋（1982—），男，河南淮阳人，副教授，博士，CCF会员，主要研究方向：分布式计算、人工智能；
基金资助:
河南省科技攻关项目(212102210078)

Abstract

Abstract:

Focusing on the challenge task for mining complementary information in different levels of features in the deep subspace clustering problem， based on the deep autoencoder， by exploring complementary information between the low-level and high-level features obtained by the encoder， a Diversity Represented Deep Subspace Clustering （DRDSC） algorithm was proposed. Firstly， based on Hilbert-Schmidt Independence Criterion （HSIC）， a diversity representation measurement model was established for different levels of features. Secondly， a feature diversity representation module was introduced into the deep autoencoder network structure， which explored image features beneficial to enhance the clustering effect. Furthermore， the form of loss function was updated to effectively fuse the underlying subspaces of multi-level representation. Finally， several experiments were conducted on commonly used clustering datasets. Experimental results show that on the datasets Extended Yale B， ORL， COIL20 and Umist， the clustering error rates of DRDSC reach 1.23%， 10.50%， 1.74% and 17.71%， respectively， which are reduced by 10.41， 16.75， 13.12 and 12.92 percentage points， respectively compared with those of Efficient Dense Subspace Clustering （EDSC）， and are reduced by 1.44， 3.50， 3.68 and 9.17 percentage points， respectively compared with Deep Subspace Clustering （DSC）， which indicates that the proposed DRDSC algorithm has better clustering effect.

Key words: Hilbert-Schmidt Independence Criterion (HSIC), autoencoder, similarity matrix, spectral clustering, subspace clustering

摘要：

针对深度子空间聚类问题中不同层次特征中互补信息挖掘困难的问题，在深度自编码器的基础上，提出了一种在编码器获取的低层和高层特征之间探索互补信息的多样性表示的深度子空间聚类（DRDSC）算法。首先，基于希尔伯特-施密特独立性准则（HSIC）建立了不同层次特征衡量多样性表示模型；其次，在深度自编码器网络结构中引入特征多样性表示模块，从而挖掘有利于提升聚类效果的图像特征；此外，更新了损失函数的形式，有效融合了多层次表示的底层子空间；最后，在常用的聚类数据集上进行了多次实验。实验结果表明，DRDSC在数据集Extended Yale B、ORL、COIL20和Umist上的聚类错误率分别达到1.23%、10.50%、1.74%和17.71%，与高效稠密子空间聚类（EDSC）相比，分别降低了10.41、16.75、13.12和12.92个百分点；与深度子空间聚类（DSC）相比，分别降低了1.44、3.50、3.68和9.17个百分点，说明所提出的DRDSC算法有更好的聚类效果。

关键词: 希尔伯特-施密特独立性准则, 自编码器, 相似度矩阵, 谱聚类, 子空间聚类

CLC Number:

TP311.13

Zhifeng MA, Junyang YU, Longge WANG. Diversity represented deep subspace clustering algorithm[J]. Journal of Computer Applications, 2023, 43(2): 407-412.

马志峰, 于俊洋, 王龙葛. 多样性表示的深度子空间聚类算法[J]. 《计算机应用》唯一官方网站, 2023, 43(2): 407-412.

Figures/Tables 8

References 27

1	ZHANG Y X， BAI X Z， FAN R R， et al. Deviation-sparse fuzzy C-means with neighbor information constraint［J］. IEEE Transactions on Fuzzy Systems， 2019， 27（1）： 185-199. 10.1109/tfuzz.2018.2883033
2	TANG Y M， REN F J， PEDRYCZ W. Fuzzy C-Means clustering through SSIM and patch for image segmentation［J］. Applied Soft Computing， 2020， 87： No.105928. 10.1016/j.asoc.2019.105928
3	BEZDEK J C， EHRLICH R， FULL W. FCM： The fuzzy c-means clustering algorithm［J］. Computers and Geosciences， 1984， 10（2/3）： 191-203. 10.1016/0098-3004(84)90020-7
4	RAO S， TRON R， VIDAL R， et al. Motion segmentation in the presence of outlying， incomplete， or corrupted trajectories［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2010， 32（10）： 1832-1845. 10.1109/tpami.2009.191
5	MA Y， DERKSEN H， HONG W， et al. Segmentation of multivariate mixed data via lossy data coding and compression［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2007， 29（9）： 1546-1562. 10.1109/tpami.2007.1085
6	ZHANG C Q， FU H Z， LIU S， et al. Low-rank tensor constrained multiview subspace clustering［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1582-1590. 10.1109/iccv.2015.185
7	YANG X， DENG C， ZHENG F， et al. Deep spectral clustering using dual autoencoder network［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 4061-4070. 10.1109/cvpr.2019.00419
8	NTOUTSI E， STEFANIDIS K， RAUSCH K， et al. "Strength lies in differences"： diversifying friends for recommendations through subspace clustering［C］// Proceedings of the 23rd ACM International Conference on Information and Knowledge Management. New York： ACM， 2014： 729-738. 10.1145/2661829.2662026
9	ZHANG A， FAWAZ N， IOANNIDIS S， et al. Guess who rated this movie： identifying users through subspace clustering［C］// Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence. Arlington， VA： AUAI Press， 2012： 944-953.
10	SHI J B， MALIK J. Normalized cuts and image segmentation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2000， 22（8）： 888-905. 10.1109/34.868688
11	CHEN H Z， WANG W W， FENG X C. Structured sparse subspace clustering with within-cluster grouping［J］. Pattern Recognition， 2018， 83： 107-118. 10.1016/j.patcog.2018.05.020
12	ELHAMIFAR E， VIDAL R. Sparse subspace clustering： algorithm， theory， and applications［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013， 35（11）： 2765-2781. 10.1109/tpami.2013.57
13	CHEN H Z， WANG W W， FENG X C， et al. Discriminative and coherent subspace clustering［J］. Neurocomputing， 2018， 284： 177-186. 10.1016/j.neucom.2018.01.006
14	LIU G C， LIN Z C， YAN S C， et al. Robust recovery of subspace structures by low-rank representation［J］. IEEE Transactions on Pattern Analysis and Machine Intelligence， 2013， 35（1）： 171-184. 10.1109/tpami.2012.88
15	JI P， ZHANG T， LI H D， et al. Deep subspace clustering networks［C］// Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook， NY： Curran Associates Inc.， 2017： 23-32.
16	ZHOU P， HOU Y Q， FENG J S. Deep adversarial subspace clustering［C］// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2018： 1596-1604. 10.1109/cvpr.2018.00172
17	ZHANG T， JI P， HARANDI M， et al. Neural collaborative subspace clustering［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 7384-7393.
18	ZHANG J J， LI C G， YOU C， et al. Self-supervised convolutional subspace clustering network［C］// Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2019： 5468-5477. 10.1109/cvpr.2019.00562
19	KHEIRANDISHFARD M， ZOHRIZADEH F， KAMANGAR F. Multi-level representation learning for deep subspace clustering［C］// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2020： 2028-2037. 10.1109/wacv45572.2020.9093277
20	CAO X C， ZHANG C Q， FU H Z， et al. Diversity-induced multi-view subspace clustering［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 586-594. 10.1109/cvpr.2015.7298657
21	WANG Q Q， CHENG J F， GAO Q X， et al. Deep multi-view subspace clustering with unified and discriminative learning［J］. IEEE Transactions on Multimedia， 2021， 23： 3483-3493. 10.1109/tmm.2020.3025666
22	GRETTON A， BELITSKI A， MURAYAMA Y， et al. The effect of artifacts on dependence measurement in fMRI［J］. Magnetic Resonance Imaging， 2006， 24（4）： 401-409. 10.1016/j.mri.2005.12.036
23	LAUER F， SCHNÖRR C. Spectral clustering of linear subspaces for motion segmentation［C］// Proceedings of the IEEE 12th International Conference on Computer Vision. Piscataway： IEEE， 2009： 678-685. 10.1109/iccv.2009.5459173
24	ZHANG C Q， FU H Z， LIU S， et al. Low-rank tensor constrained multiview subspace clustering［C］// Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway： IEEE， 2015： 1582-1590. 10.1109/iccv.2015.185
25	LUO S R， ZHANG C Q， ZHANG W， et al. Consistent and specific multi-view subspace clustering［C］// Proceedings of the 32nd AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2018： 3730-3737. 10.1609/aaai.v32i1.11617
26	PATEL V M， VIDAL R. Kernel sparse subspace clustering［C］// Proceedings of the 2014 IEEE International Conference on Image Processing. Piscataway： IEEE， 2014： 2849-2853. 10.1109/icip.2014.7025576
27	JI P， SALZMANN M， LI H D. Efficient dense subspace clustering［C］// Proceedings of the 2014 IEEE Winter Conference on Applications of Computer Vision. Piscataway： IEEE， 2014： 461-468. 10.1109/wacv.2014.6836065

自编码层	Extended Yale B		ORL		COIL20		Umist
自编码层	卷积核	通道数	卷积核	通道数	卷积核	通道数	卷积核	通道数
编码层1	5×5	10	3×3	3	3×3	15	3×3	20
编码层2	3×3	20	3×3	3	3×3	10	3×3	10
编码层3	3×3	30	3×3	5	—	—	3×3	5
解码层3	3×3	30	3×3	5	—	—	3×3	5
解码层2	3×3	20	3×3	3	3×3	10	3×3	10
解码层1	5×5	10	3×3	3	3×3	15	3×3	20

自编码层	Extended Yale B		ORL		COIL20		Umist
自编码层	卷积核	通道数	卷积核	通道数	卷积核	通道数	卷积核	通道数
编码层1	5×5	10	3×3	3	3×3	15	3×3	20
编码层2	3×3	20	3×3	3	3×3	10	3×3	10
编码层3	3×3	30	3×3	5	—	—	3×3	5
解码层3	3×3	30	3×3	5	—	—	3×3	5
解码层2	3×3	20	3×3	3	3×3	10	3×3	10
解码层1	5×5	10	3×3	3	3×3	15	3×3	20

聚类数目	衡量维度	算法
聚类数目	衡量维度	LRR	SSC	KSSC	EDSC	AE+EDSC	DSC	S²ConvSCN	MLRDSC	DRDSC
10	平均数	22.22	10.22	14.49	5.64	5.46	1.59	1.18	1.10	1.11
10	中位数	23.49	11.09	15.78	5.47	6.09	1.25	1.09	0.94	1.10
15	平均数	23.22	13.13	16.22	7.63	6.70	1.67	1.12	0.91	0.98
15	中位数	23.49	13.40	17.34	6.41	5.52	1.72	1.14	0.99	0.94
20	平均数	30.23	19.75	16.55	9.30	7.67	1.73	1.30	0.99	0.98
20	中位数	29.30	21.17	17.34	10.31	6.56	1.80	1.25	1.02	0.94
25	平均数	27.92	26.22	18.56	10.67	10.27	1.75	1.29	1.13	1.00
25	中位数	28.13	26.66	18.03	10.84	10.22	1.81	1.28	1.12	1.03
30	平均数	37.98	28.76	20.49	11.24	11.56	2.07	1.67	1.78	1.13
30	中位数	36.82	28.59	20.94	11.09	10.36	2.19	1.72	1.41	1.09
35	平均数	41.85	28.55	26.07	13.10	13.28	2.65	1.62	1.44	1.42
35	中位数	41.81	29.04	25.92	13.10	13.21	2.64	1.60	1.47	1.43
38	平均数	34.87	27.51	27.75	11.64	12.66	2.67	1.52	1.36	1.23
38	中位数	34.87	27.51	27.75	11.64	12.66	2.67	1.52	1.36	1.23

聚类数目	衡量维度	算法
聚类数目	衡量维度	LRR	SSC	KSSC	EDSC	AE+EDSC	DSC	S²ConvSCN	MLRDSC	DRDSC
10	平均数	22.22	10.22	14.49	5.64	5.46	1.59	1.18	1.10	1.11
10	中位数	23.49	11.09	15.78	5.47	6.09	1.25	1.09	0.94	1.10
15	平均数	23.22	13.13	16.22	7.63	6.70	1.67	1.12	0.91	0.98
15	中位数	23.49	13.40	17.34	6.41	5.52	1.72	1.14	0.99	0.94
20	平均数	30.23	19.75	16.55	9.30	7.67	1.73	1.30	0.99	0.98
20	中位数	29.30	21.17	17.34	10.31	6.56	1.80	1.25	1.02	0.94
25	平均数	27.92	26.22	18.56	10.67	10.27	1.75	1.29	1.13	1.00
25	中位数	28.13	26.66	18.03	10.84	10.22	1.81	1.28	1.12	1.03
30	平均数	37.98	28.76	20.49	11.24	11.56	2.07	1.67	1.78	1.13
30	中位数	36.82	28.59	20.94	11.09	10.36	2.19	1.72	1.41	1.09
35	平均数	41.85	28.55	26.07	13.10	13.28	2.65	1.62	1.44	1.42
35	中位数	41.81	29.04	25.92	13.10	13.21	2.64	1.60	1.47	1.43
38	平均数	34.87	27.51	27.75	11.64	12.66	2.67	1.52	1.36	1.23
38	中位数	34.87	27.51	27.75	11.64	12.66	2.67	1.52	1.36	1.23

数据集	算法
数据集	LRR	SSC	KSSC	EDSC	DSC	S²ConvSCN	MLRDSC	DRDSC
ORL	33.50	29.50	34.25	27.25	14.00	10.50	11.25	10.50
COIL20	30.21	14.83	24.65	14.86	5.42	2.14	2.08	1.74
Umist	30.21	30.96	34.69	30.63	26.88	—	—	17.71

Diversity represented deep subspace clustering algorithm

多样性表示的深度子空间聚类算法

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 8

References 27

Related Articles 15

Recommended Articles

Metrics

[1]	Kaili DENG, Weibo WEI, Zhenkuan PAN. Industrial defect detection method with improved masked autoencoder [J]. Journal of Computer Applications, 2024, 44(8): 2595-2603.
[2]	Qing WANG, Jieyu ZHAO, Xulun YE, Nongxiao WANG. Enhanced deep subspace clustering method with unified framework [J]. Journal of Computer Applications, 2024, 44(7): 1995-2003.
[3]	Zongyu LI, Siwei QIANG, Xiaobo GUO, Zhenfeng ZHU. Re-weighted adversarial variational autoencoder and its application in industrial causal effect estimation [J]. Journal of Computer Applications, 2024, 44(4): 1099-1106.
[4]	Yu DING, Hanlin ZHANG, Rong LUO, Hua MENG. Fuzzy clustering algorithm based on belief subcluster cutting [J]. Journal of Computer Applications, 2024, 44(4): 1128-1138.
[5]	Zhuo ZHANG, Huazhu CHEN. Deep subspace clustering based on multiscale self-representation learning with consistency and diversity [J]. Journal of Computer Applications, 2024, 44(2): 353-359.
[6]	Jingxin LIU, Wenjing HUANG, Liangsheng XU, Chong HUANG, Jiansheng WU. Unsupervised feature selection model with dictionary learning and sample correlation preservation [J]. Journal of Computer Applications, 2024, 44(12): 3766-3775.
[7]	Hui JIANG, Qiuyan YAN, Zhujun JIANG. Symmetric positive definite autoencoder method for multivariate time series anomaly detection [J]. Journal of Computer Applications, 2024, 44(10): 3294-3299.
[8]	Tian HE, Zongxin SHEN, Qianqian HUANG, Yanyong HUANG. Adaptive learning-based multi-view unsupervised feature selection method [J]. Journal of Computer Applications, 2023, 43(9): 2657-2664.
[9]	Jianwen GAN, Yan CHEN, Peng ZHOU, Liang DU. Clustering ensemble algorithm with high-order consistency learning [J]. Journal of Computer Applications, 2023, 43(9): 2665-2672.
[10]	Chunyong YIN, Liwen ZHOU. Unsupervised time series anomaly detection model based on re-encoding [J]. Journal of Computer Applications, 2023, 43(3): 804-811.
[11]	Shaokang XU, Zhancheng ZHANG, Haonan YAO, Zhiwei ZOU, Baocheng ZHANG. 2D/3D spine medical image real-time registration method based on pose encoder [J]. Journal of Computer Applications, 2023, 43(2): 589-594.
[12]	Qing JIA, Laihua WANG, Weisheng WANG. Anomaly detection in video via independently recurrent neural network and variational autoencoder network [J]. Journal of Computer Applications, 2023, 43(2): 507-513.
[13]	Qize REN, Hongjie JIA, Dongyu CHEN. Large-scale subspace clustering algorithm with Local structure learning [J]. Journal of Computer Applications, 2023, 43(12): 3747-3754.
[14]	Anqin ZHANG, Xiaohui WANG. Power battery safety warning based on time series anomaly detection [J]. Journal of Computer Applications, 2023, 43(12): 3799-3805.
[15]	Wenbo LI, Bo LIU, Lingling TAO, Fen LUO, Hang ZHANG. Deep spectral clustering algorithm with L1 regularization [J]. Journal of Computer Applications, 2023, 43(12): 3662-3667.