关键词: 谱聚类, 文本聚类, 大数据集

Abstract: To alleviate the computational bottleneck of spectral clustering, in this paper a general ensemble algorithm, called indirect spectral clustering, was developed. The algorithm first grouped a given large dataset into many overclusters and then regarded each obtained overcluster as a basic object. And then the standard spectral clustering ran at this object level. By convention, when applying this new idea to large text datasets, the cosine distance would be the appropriate manner in measuring the similarities between overclusters. The empirical studies on 20-Newgroups dataset show that the proposed algorithm has a 14.72% higher accuracy on average than the K-Means algorithm and has a 0.88% lower accuracy than the normalizedcut spectral clustering. However, the proposed algorithm saves 16.8 times computation time compared to the normalizedcut spectral clustering. In conclusion, with the increase of data size, the computation time of the normalizedcut spectral clustering might become unacceptable; however, the proposed algorithm might efficiently give the nearoptimal solutions.

Key words: spectral clustering, text clustering, large datasets