Journal of Computer Applications ›› 2016, Vol. 36 ›› Issue (2): 311-315.DOI: 10.11772/j.issn.1001-9081.2016.02.0311

Previous Articles     Next Articles

Parallel optimization sampling clustering K-means algorithm for big data processing

ZHOU Runwu, LI Zhiyong, CHEN Shaomiao, CHEN Jing, LI Renfa   

  1. College of Computer Science and Electronic Engineering, Hunan University, Changsha Hunan 410082, China
  • Received:2015-08-29 Revised:2015-09-14 Online:2016-02-10 Published:2016-02-03

面向大数据处理的并行优化抽样聚类K-means算法

周润物, 李智勇, 陈少淼, 陈京, 李仁发   

  1. 湖南大学 信息科学与工程学院, 长沙 410082
  • 通讯作者: 李智勇(1971-),男,湖南长沙人,教授,博士,主要研究方向:云计算、大数据分析、模式识别、机器视觉。
  • 作者简介:周润物(1992-),男,湖北咸宁人,博士研究生,主要研究方向:人工智能、数据挖掘;陈少淼(1989-),男,湖南邵阳人,博士研究生,主要研究方向:多目标优化、智能计算;陈京(1992-),男,湖南怀化人,硕士研究生,主要研究方向:云计算、大数据学习;李仁发(1957-),男,湖南郴州人,教授,博士生导师,博士,主要研究方向:高性能嵌入式计算。
  • 基金资助:
    国家自然科学基金资助项目(61173107);国家863计划项目(2012AA01A301-01)。

Abstract: Focusing on the low accuracy and slow convergence of K-means clustering algorithm, an improved K-means algorithm based on optimization sample clustering named OSCK (Optimization Sampling Clustering K-means Algorithm) was proposed. Firstly, multiple samples were obtained from mass data by probability sampling. Secondly, based on Euclidean distance similarity principle of optimal clustering center, the results of sample clustering were modeled and evaluated, and the sub-optimal solution of sample clustering results was removed. Finally, the final k clustering centers were got by weighted integration evaluation of clustering results, and the final k clustering centers were used as cluster centers of big data set. Theoretical analysis and experimental results show that the proposed method for mass data analysis with respect to the comparison algorithm has better clustering accuracy, and has strong robustness and scalability.

Key words: big data, K-means, probability sampling, Euclidean distance, clustering accuracy

摘要: 针对大数据环境下K-means聚类算法聚类精度不足和收敛速度慢的问题,提出一种基于优化抽样聚类的K-means算法(OSCK)。首先,该算法从海量数据中概率抽样多个样本;其次,基于最佳聚类中心的欧氏距离相似性原理,建模评估样本聚类结果并去除抽样聚类结果的次优解;最后,加权整合评估得到的聚类结果得到最终k个聚类中心,并将这k个聚类中心作为大数据集聚类中心。理论分析和实验结果表明,OSCK面向海量数据分析相对于对比算法具有更好的聚类精度,并且具有很强的稳健性和可扩展性。

关键词: 大数据, K-均值, 概率抽样, 欧氏距离, 聚类精度

CLC Number: