Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (9): 2696-2703.DOI: 10.11772/j.issn.1001-9081.2023091253

• Data science and technology • Previous Articles     Next Articles

Incomplete multi-view clustering algorithm based on self-attention fusion

Shunyong LI1, Shiyi LI1,2, Rui XU1, Xingwang ZHAO2()   

  1. 1.School of Mathematical Sciences,Shanxi University,Taiyuan Shanxi 030006,China
    2.School of Computer and Information Technology (School of Big Data),Shanxi University,Taiyuan Shanxi 030006,China
  • Received:2023-09-12 Revised:2023-10-31 Accepted:2023-11-02 Online:2023-11-23 Published:2024-09-10
  • Contact: Xingwang ZHAO
  • About author:LI Shunyong, born in 1975, Ph. D., professor. His research interests include statistical machine learning, data mining.
    LI Shiyi, born in 1999, Ph. D. candidate. His research interests include machine learning, data mining.
    XU Rui, born in 1996, M. S. candidate. Her research interests include statistical machine learning, data mining.
  • Supported by:
    National Natural Science Foundation of China(61976128);Fundamental Research Program of Shanxi Province(202303021221054);Shanxi Province Graduate Education and Teaching Reform Project(2022YJJG010)

基于自注意力融合的不完整多视图聚类算法

李顺勇1, 李师毅1,2, 胥瑞1, 赵兴旺2()   

  1. 1.山西大学 数学科学学院,太原 030006
    2.山西大学 计算机与信息技术学院(大数据学院),太原 030006
  • 通讯作者: 赵兴旺
  • 作者简介:李顺勇(1975—),男,山西大同人,教授,博士,主要研究方向:统计机器学习、数据挖掘
    李师毅(1999—),男,山西长治人,博士研究生,主要研究方向:机器学习、数据挖掘
    胥瑞(1996—),女,山西运城人,硕士研究生,主要研究方向:统计机器学习、数据挖掘
    赵兴旺(1984—),男,山西晋中人,副教授,博士,CCF会员,主要研究方向:机器学习、数据挖掘。
  • 基金资助:
    国家自然科学基金资助项目(61976128);山西省基础研究计划项目(202303021221054);山西省研究生教育教学改革课题(2022YJJG010)

Abstract:

Multi-view clustering task based on incomplete data has become one of the research hotspots in the field of unsupervised learning. However, most multi-view clustering algorithms based on “shallow” models often find it difficult to extract and characterize potential feature structures within views when dealing with large-scale high-dimensional data. At the same time, the stacking or averaging methods of multi-view information fusion ignore the differences between views and does not fully consider the different contributions of each view to building a common consensus representation. To address the above issues, an Incomplete Multi-View Clustering algorithm based on Self-Attention Fusion (IMVCSAF) was proposed. Firstly, the potential features of each view were extracted on the basis of a deep autoencoder, and the consistency information among views was maximized by using contrastive learning. Secondly, a self-attention mechanism was adopted to recode and fuse the potential representations of each view, and the inherent causality as well as feature complementarity between different views was considered and mined comprehensively. Thirdly, based on the common consensus representation, the potential representation of missing instance was predicted and recovered, thereby fully implementing the process of multi-view clustering. Experimental results on Scene-15, LandUse-21, Caltech101-20 and Noisy-MNIST datasets show that, the accuracy of IMVCSAF is higher than those of other comparison algorithms while meeting the convergence requirements. On Noisy-MNIST dataset with 50% miss rate, the accuracy of IMVCSAF is 6.58 percentage points higher than that of the second best algorithm — COMPETER (inCOMPlete muLti-view clustEring via conTrastivE pRediction).

Key words: multi-view clustering, self-attention, mutual information, representation learning, deep learning

摘要:

基于不完整数据的多视图聚类任务已经成为无监督学习领域的研究热点之一。然而大多数基于“浅层”模型的多视图聚类算法通常在面对大规模高维数据时难以提取和刻画视图内的潜在特征结构;同时,堆叠或求平均的多视图信息融合方式忽视了视图之间的差异性,没有充分考虑各视图对构建公共一致表示的不同贡献。针对以上问题,提出一种基于自注意力融合的不完整多视图聚类算法(IMVCSAF)。首先,基于深度自编码器提取各视图的潜在特征,并采用对比学习的方式最大化各视图间的一致性信息;其次,采用自注意力机制对各视图的潜在表示进行重新编码和融合,并全面考虑和挖掘不同视图之间的内在因果性和特征互补性;再次,基于公共一致表示对缺失实例样本的潜在表示进行预测和恢复,从而完整地实现多视图聚类的过程。在Scene-15、LandUse-21、Caltech101-20和Noisy-MNIST数据集上的实验结果表明,IMVCSAF在满足收敛性要求的前提下得到的准确率均高于其他对比算法,而在50%缺失率的Noisy-MNIST数据集上,IMVCSAF的准确率比次优的COMPLETER(inCOMPlete muLti-view clustEring via conTrastivE pRediction)算法提高了6.58个百分点。

关键词: 多视图聚类, 自注意力, 互信息, 表示学习, 深度学习

CLC Number: