Journal of Computer Applications ›› 2024, Vol. 44 ›› Issue (12): 3784-3789.DOI: 10.11772/j.issn.1001-9081.2023121866

• Artificial intelligence • Previous Articles     Next Articles

Incomplete multi-view clustering algorithm based on attention mechanism

Chenghao YANG, Jie HU(), Hongjun WANG, Bo PENG   

  1. School of Computing and Artificial Intelligence,Southwest Jiaotong University,Chengdu Sichuan 611756,China
  • Received:2024-01-08 Revised:2024-03-14 Accepted:2024-03-15 Online:2024-03-22 Published:2024-12-10
  • Contact: Jie HU
  • About author:YANG Chenghao, born in 1999, M. S. candidate. His research interests include deep learning, multi-view clustering.
    WANG Hongjun, born in 1977, Ph. D., associate research fellow. His research interests include machine learning, data mining.
    PENG Bo, born in 1980, Ph. D., professor. Her research interests include image segmentation, pattern recognition.
  • Supported by:
    National Natural Science Foundation of China(62276216);Sichuan Science and Technology Program(2023YFG0354);International Student Education Management Research Project of Southwest Jiaotong University(23LXSGL01)

基于注意力机制的不完备多视图聚类算法

杨成昊, 胡节(), 王红军, 彭博   

  1. 西南交通大学 计算机与人工智能学院,成都 611756
  • 通讯作者: 胡节
  • 作者简介:杨成昊(1999—),男,四川成都人,硕士研究生,CCF会员,主要研究方向:深度学习、多视图聚类
    王红军(1977—),男,四川成都人,副研究员,博士,CCF会员,主要研究方向:机器学习、数据挖掘
    彭博(1980—),女,四川成都人,教授,博士,CCF会员,主要研究方向:图像分割、模式识别。
  • 基金资助:
    国家自然科学基金资助项目(62276216);四川省重点研发计划项目(2023YFG0354);2023年西南交通大学国际学生教育管理研究项目(23LXSGL01)

Abstract:

In order to solve the problems of uncertainty in completing missing view data, lack of robustness of embedding learning and low model generalization in traditional deep incomplete multi-view clustering algorithms, an Incomplete Multi-View Clustering algorithm based on Attention Mechanism (IMVCAM) was proposed. Firstly, K-Nearest Neighbors (KNN) algorithm was used to complete the missing data in the view, making the training data complementary. Then, after passing the linear encoding layer, the obtained embedding was passed through the attention layer to improve the quality of the embedding. Finally, the embedding obtained from the training of each view was clustered using k-means clustering algorithm, and the weights of the views were determined by the Pearson correlation coefficient. Experimental results on five classic datasets show that, the optimal result was achieved by IMVCAM on Fashion dataset, compared with the sub-optimal Deep Safe Incomplete Multi-View Clustering (DSIMVC) algorithm, IMVCAM improves the clustering accuracy by 2.85 and 4.35 percentage points respectively when the data missing rate is 0.1 and 0.3. Besides, on Caltech101-20 dataset, the clustering accuracy of IMVCAM is increased by 7.68 and 3.48 percentage points respectively compared to that of the sub-optimal algorithm IMVCSAF (Incomplete Multi-View Clustering algorithm based on Self-Attention Fusion) when the missing rate is 0.1 and 0.3. The proposed algorithm can effectively deal with the incompleteness of multi-view data and the problem of model generalization.

Key words: incomplete multi-view clustering, K-Nearest Neighbors (KNN) algorithm, attention mechanism, k-means clustering algorithm, Pearson correlation coefficient

摘要:

针对传统深度不完备多视图聚类算法中补全缺失视图数据的不确定性、嵌入学习鲁棒性的缺乏和模型泛化性低的问题,提出基于注意力机制的不完备多视图聚类算法(IMVCAM)。首先,通过K最近邻(KNN)算法补全了视图中缺失的数据,使训练数据具有互补性;其次,经过线性编码层后,将获得的嵌入通过注意力层,以提高嵌入的质量;最后,对每个视图训练得到的嵌入使用k均值聚类算法进行聚类,而视图的权重通过皮尔逊相关系数确定。在5个经典的数据集上的实验结果表明,在Fashion数据集上,IMVCAM取得最优的结果,相较于次优的深度安全不完整多视图聚类(DSIMVC)算法,在数据缺失率为0.1、0.3的情况下,IMVCAM的聚类准确率分别提升了2.85、4.35个百分点;此外,在Caltech101-20数据集上,IMVCAM相较于次优的基于自注意力融合的不完整多视图聚类算法(IMVCSAF),在数据缺失率为0.1、0.3的情况下的聚类准确率分别提升了7.68、3.48个百分点。所提算法能够有效应对多视图数据的不完备性和模型泛化性问题。

关键词: 不完备多视图聚类, K最近邻算法, 注意力机制, k均值聚类算法, 皮尔逊相关系数

CLC Number: