计算机应用 ›› 2014, Vol. 34 ›› Issue (1): 31-35.DOI: 10.11772/j.issn.1001-9081.2014.01.0031

• 先进计算 • 上一篇    下一篇

基于COG-OS框架利用SMART预测云计算平台的硬盘故障

宋云华1,2,柏文阳1,2,周琦3   

  1. 1. 计算机软件新技术国家重点实验室(南京大学),南京 210023
    2. 南京大学 计算机科学与技术系,南京 210023
    3. 阿里云计算有限公司 飞天结构化数据服务,杭州 311121
  • 收稿日期:2013-08-14 修回日期:2013-11-12 出版日期:2014-01-01 发布日期:2014-02-14
  • 通讯作者: 宋云华
  • 作者简介:宋云华(1984-),女,江苏盐城人,硕士研究生,主要研究方向:云计算、数据挖掘;柏文阳(1967-),男,江苏扬州人,副教授, CCF高级会员,主要研究方向:数据库、信息安全;周琦(1984-),男,浙江绍兴人,硕士,主要研究方向:分布式系统监控、诊断与日志分析。
  • 基金资助:

    国家863计划项目

Prediction on hard disk failure of cloud computing framework by using SMART on COG-OS framework

SONG Yunhua1,2,BO Wenyang1,2,ZHOU Qi3   

  1. 1. Department of Computer Science and Technology, Nanjing University, Nanjing Jiangsu 210023, China;
    2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing Jiangsu 210023, China;
    3. Flying-Structured Data Services, Alibaba Cloud Computing Limited, Hangzhou Zhejiang 311121, China
  • Received:2013-08-14 Revised:2013-11-12 Online:2014-01-01 Published:2014-02-14
  • Contact: SONG Yunhua

摘要: 针对云计算平台的硬盘不可靠问题,提出基于带过采样的COG(COG-OS)框架,利用硬盘自我监测分析和报告技术(SMART)日志预测故障硬盘。首先采用DBScan或K-means聚类算法将无故障硬盘样本划分成多个不相交子集;再与故障硬盘样本结合,采用少量样本合成过采样技术(SMOTE)使整体样本集趋于平衡;最后采用LIBSVM分类算法预测故障硬盘。调整参数,将COG-OS与SMOTE+支持向量机(SVM)的预测性能相比较,实验结果表明该方法具有可行性。当采用K-means方法划分无故障盘样本,并采用径向基函数(RBF)内核的LIBSVM方法预测故障盘时,COG-OS改善了SMOTE+SVM对故障硬盘的预测查全率和整体性能。

关键词: COG-OS框架, 自我监测分析和报告技术, K-均值, 少量样本合成过采样技术, LIBSVM, 支持向量机

Abstract: The hard disk of cloud computing platform is not reliable. This paper proposed to use Self-Monitoring Analysis and Reporting Technology (SMART) log to predict hard disk failure based on Classification using lOcal clusterinG with Over-Sampling (COG-OS) framework. First, faultless hard disks were divided into multiple disjoint sample subsets by using DBScan or K-means clustering algorithm. And then these subsets and another sample set of faulty hard disks were mixed, and Synthetic Minority Over-sampling TEchnique (SMOTE) was used to make the overall sample set tend to balance. At last, faulty hard disks was predicted by using LIBSVM classification algorithm. The experimental results show that the method is feasible. COG-OS improves SMOTE+Support Vector Machine (SVM) on faulty hard disks' recall and overall performance, when using K-means method to divide samples of faultless hard disks and using LIBSVM method with Radial Basis Function (RBF) kernel to predict faulty hard disks.

Key words: Classification using lOcal clusterinG with Over-Sampling (COG-OS) framework, Self-Monitoring Analysis and Reporting Technology (SMART), K-means, Synthetic Minority Over-sampling TEchnique (SMOTE), LIBSVM, Support Vector Machine (SVM)

中图分类号: