Multi-view clustering network with deep fusion

doi:10.11772/j.issn.1001-9081.2022091394

Abstract

Abstract:

Current deep multi-view clustering methods have the following shortcomings： 1） When feature extraction is carried out for a single view， only attribute information or structural information of the samples is considered， and these two types of information are not integrated. Thus， the extracted features cannot fully represent latent structure of the original data. 2） Feature extraction and clustering were divided into two separated processes， without establishing the relationship between them， so that the feature extraction process cannot be optimized by the clustering process. To solve these problems， a Deep Fusion based Multi-view Clustering Network （DFMCN） was proposed. Firstly， the embedding space of each view was obtained by combining autoencoder and graph convolution autoencoder to fuse attribute information and structure information of samples. Then， the embedding space of the fusion view was obtained through weighted fusion， and clustering was carried out in this space. And in the process of clustering， the feature extraction process was optimized by a two-layer self-supervision mechanism. Experimental results on FM （Fashion-MNIST）， HW （HandWritten numerals）， and YTF （YouTube Face） datasets show that the accuracy of DFMCN is higher than those of all comparison methods； and DFMCN has the accuracy increased by 1.80 percentage points compared with the suboptimal CMSC-DCCA （Cross-Modal Subspace Clustering via Deep Canonical Correlation Analysis） method on FM dataset， the Normalized Mutual Information （NMI） of DFMCN is increased by 1.26 to 14.84 percentage points compared to all methods except for CMSC-DCCA and DMSC （Deep Multimodal Subspace Clustering networks）. Experimental results verify the effectiveness of the proposed method.

Key words: deep learning, multi-view clustering, deep clustering, self-representation, self-supervision

摘要：

现有的深度多视图聚类方法存在以下缺点：1）在对单一视图进行特征提取时，只考虑了样本的属性信息或结构信息，而没有将二者进行融合，导致提取到的特征不能充分表示原始数据的潜在结构；2）将特征提取与聚类划分为两个独立的过程，没有建立两者间的联系，因此无法利用聚类过程优化特征提取过程。针对以上问题，提出一种深度融合多视图聚类网络（DFMCN）。首先，结合自编码器和图卷积自编码器融合样本的属性信息和结构信息，获取每个视图的嵌入空间；然后，通过加权融合获取融合视图嵌入空间并在此空间中进行聚类，并且在聚类过程中采用双层自监督机制优化特征提取过程。在FM（Fashion-MNIST）、HW（HandWritten numerals）、YTF（YouTube Face）数据集上的实验结果表明：DFMCN的准确率高于所有对比方法；在FM数据集上，DFMCN的准确率比次优的CMSC-DCCA（Cross-Modal Subspace Clustering via Deep Canonical Correlation Analysis）方法提高了1.80个百分点，标准化互信息（NMI）高于除CMSC-DCCA和DMSC（Deep Multimodal Subspace Clustering networks）的所有方法1.26~14.84个百分点。实验结果验证了所提方法的有效性。

关键词: 深度学习, 多视图聚类, 深度聚类, 自表示, 自监督

CLC Number:

TP391.1

Ziyi HE, Yan YANG, Yiling ZHANG. Multi-view clustering network with deep fusion[J]. Journal of Computer Applications, 2023, 43(9): 2651-2656.

何子仪, 杨燕, 张熠玲. 深度融合多视图聚类网络[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2651-2656.

Figures/Tables 6

References 28

1	XU D K， TIAN Y J. A comprehensive survey of clustering algorithms［J］. Annals of Data Science， 2015， 2（2）：165-193. 10.1007/s40745-015-0040-1
2	YANG Y， WANG H. Multi-view clustering： a survey［J］. Big Data Mining and Analytics， 2018， 1（2）： 83-107. 10.26599/bdma.2018.9020003
3	LIU J L， WANG C， GAO J， et al. Multi-view clustering via joint nonnegative matrix factorization［C］// Proceedings of the 2013 SIAM International Conference on Data Mining. Philadelphia， PA： SIAM， 2013： 252-260. 10.1137/1.9781611972832.28
4	ANDREW G， ARORA R， BILMES J， et al. Deep canonical correlation analysis［C］// Proceedings of the 30th International Conference on Machine Learning. New York： JMLR.org， 2013： 1247-1255.
5	WANG W R， ARORA R， LIVESCU K， et al. On deep multi-view representation learning［C］// Proceedings of the 32nd International Conference on Machine Learning. New York： JMLR.org， 2015： 1083-1092. 10.1109/icassp.2015.7178840
6	ABAVISANI M， PATEL V M. Deep multimodal subspace clustering networks［J］. IEEE Journal of Selected Topics in Signal Processing， 2018， 12（6）： 1601-1614. 10.1109/jstsp.2018.2875385
7	CAO X C， ZHANG C Q， FU H Z， et al. Diversity-induced multi-view subspace clustering［C］// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2015： 586-594. 10.1109/cvpr.2015.7298657
8	ZHANG C Q， HU Q H， FU H Z， et al. Latent multi-view subspace clustering［C］// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2017： 4333-4341. 10.1109/cvpr.2017.461
9	LIU S Y， WANG S W， ZHANG P， et al. Efficient on-pass multi-view subspace clustering with consensus anchors［C］// Proceedings of the 36th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2022： 7576-7584. 10.1609/aaai.v36i7.20723
10	LUO P， PENG J Y， GUAN Z Y， et al. Dual regularized multi-view non-negative matrix factorization for clustering［J］. Neurocomputing， 2018， 294： 1-11. 10.1016/j.neucom.2017.10.023
11	HUANG S D， KANG Z， XU Z L. Auto-weighted multi-view clustering via deep matrix decomposition［J］. Pattern Recognition， 2020， 97： No.107015. 10.1016/j.patcog.2019.107015
12	NIE F P， LI J， LI X L. Self-weighted multiview clustering with multiple graphs［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 2564-2570. 10.24963/ijcai.2017/357
13	PENG X， HUANG Z Y， LV J C， et al. COMIC： multi-view clustering without parameter selection［C］// Proceedings of the 36th International Conference on Machine Learning. New York： JMLR.org， 2019： 5092-5101.
14	WANG Q Q， CHENG J F， GAO Q X， et al. Deep multi-view subspace clustering with unified and discriminative learning［J］. IEEE Transactions on Multimedia， 2021， 23： 3483-3493. 10.1109/tmm.2020.3025666
15	SUN X K， CHENG M M， MIN C， et al. Self-supervised deep multi-view subspace clustering［C］// Proceedings of the 11th Asian Conference on Machine Learning. New York： JMLR.org， 2019： 1001-1016.
16	GAO Q X， LIAN H H， WANG Q Q， et al. Cross-modal subspace clustering via deep canonical correlation analysis［C］// Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2020： 3938-3945. 10.1609/aaai.v34i04.5808
17	XIE J Y， GIRSHICK R， FARHADI A. Unsupervised deep embedding for clustering analysis［C］// Proceedings of the 33rd International Conference on Machine Learning. New York： JMLR.org， 2016： 478-487.
18	GUO X F， GAO L， LIU X W， et al. Improved deep embedded clustering with local structure preservation［C］// Proceedings of the 26th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2017： 1753-1759. 10.24963/ijcai.2017/243
19	BO D Y， WANG X， SHI C， et al. Structural deep clustering network［C］// Proceedings of the Web Conference 2020. Republic and Canton of Geneva： International World Wide Web Conferences Steering Committee， 2020： 1400-1410. 10.1145/3366423.3380214
20	TU W X， ZHOU S H， LIU X W， et al. Deep fusion clustering network［C］// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto， CA： AAAI Press， 2021： 9978-9987. 10.1609/aaai.v35i11.17198
21	BRBIĆ M， KOPRIVA I. Multi-view low-rank sparse subspace clustering［J］. Pattern Recognition， 2018， 73： 247-258. 10.1016/j.patcog.2017.08.024
22	LI Z Y， WANG Q Q， TAO Z Q， et al. Deep adversarial multi-view clustering network［C］// Proceedings of the 28th International Joint Conference on Artificial Intelligence. California： ijcai.org， 2019： 2952-2958. 10.24963/ijcai.2019/409
23	XU J， REN Y Z， LI G F， et al. Deep embedded multi-view clustering with collaborative training［J］. Information Sciences， 2021， 573： 279-290. 10.1016/j.ins.2020.12.073
24	PAN E L， KANG Z. Multi-view contrastive graph clustering［C/OL］// Proceedings of the 35th Conference on Neural Information Processing Systems ［2022-08-12］..
25	XIAO H， RASUL K， VOLLGRAF R. Fashion-MNIST： a novel image dataset for benchmarking machine learning algorithms［EB/OL］. （2017-09-15）［2022-03-25］..
26	DUA D， GRAFF C. UCI machine learning repository［DB/OL］. ［2021-11-08］..
27	YANG J W， PARIKH D， BATRA D. Joint unsupervised learning of deep representations and image clusters［C］// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway： IEEE， 2016： 5147-5156. 10.1109/cvpr.2016.556
28	LIN B Q， XIE Y， QU Y Y， et al. Deep multi-view clustering via multiple embedding ［EB/OL］. ［2022-02-14］. .

数据集	样本数	视图维度	类数
FM	70 000	784/784	10
HW	2 000	76/216	10
YTF	10 000	9 075/3 025/3 025	41

数据集	样本数	视图维度	类数
FM	70 000	784/784	10
HW	2 000	76/216	10
YTF	10 000	9 075/3 025/3 025	41

方法	FM		HW		YTF
方法	ACC	NMI	ACC	NMI	ACC	NMI
K-Means	51.27	49.99	54.60	49.50	56.01	75.23
DEC	51.80	54.60	70.80	69.13	59.40	75.68
DFCN	61.80	63.57	97.15	93.52	62.53	80.42
DMJC	61.41	63.41	—	—	61.15	77.40
DCCA	52.74	53.82	81.43	78.14	45.19	60.35
DCCAE	51.87	53.01	81.92	78.62	45.57	60.15
DMSC	59.55	65.07	91.64	85.50	62.80	80.16
CMSC-DCCA	62.95	68.33	—	—	66.15	82.67
DFMCN	64.75	64.83	97.75	94.69	66.25	82.75

方法	FM		HW		YTF
方法	ACC	NMI	ACC	NMI	ACC	NMI
K-Means	51.27	49.99	54.60	49.50	56.01	75.23
DEC	51.80	54.60	70.80	69.13	59.40	75.68
DFCN	61.80	63.57	97.15	93.52	62.53	80.42
DMJC	61.41	63.41	—	—	61.15	77.40
DCCA	52.74	53.82	81.43	78.14	45.19	60.35
DCCAE	51.87	53.01	81.92	78.62	45.57	60.15
DMSC	59.55	65.07	91.64	85.50	62.80	80.16
CMSC-DCCA	62.95	68.33	—	—	66.15	82.67
DFMCN	64.75	64.83	97.75	94.69	66.25	82.75

数据集	模型	ACC	NMI
FM	无融合特征提取模块	56.93	62.30
	无双层自监督模块	64.20	63.95
	完整模型	64.75	64.83
HW	无融合特征提取模块	76.35	72.06
	无双层自监督模块	97.35	93.91
	完整模型	97.75	94.69
YTF	无融合特征提取模块	60.40	77.42
	无双层自监督模块	64.50	81.93
	完整模型	66.25	82.75