计算机应用 ›› 2015, Vol. 35 ›› Issue (7): 1959-1964.DOI: 10.11772/j.issn.1001-9081.2015.07.1959

• 人工智能 • 上一篇    下一篇

面向不平衡微博数据集的转发行为预测方法

赵煜, 邵必林, 边根庆, 宋丹   

  1. 西安建筑科技大学 管理学院, 西安 710055
  • 收稿日期:2015-01-21 修回日期:2015-03-18 出版日期:2015-07-10 发布日期:2015-07-17
  • 通讯作者: 赵煜(1981-),男,陕西西安人,博士研究生,CCF会员,主要研究方向:数据挖掘、大数据处理,zhaoyu_xauat_mail@163.com
  • 作者简介:邵必林(1965-),男,云南腾冲人,教授,博士生导师,主要研究方向:云计算、大数据处理; 边根庆(1968-),男,浙江浦江人,副教授,硕士,主要研究方向:云计算、云存储;宋丹(1991-),女,陕西汉中人,硕士研究生,主要研究方向:数据挖掘、大数据处理。
  • 基金资助:

    国家自然科学基金资助项目(61272458)。

Prediction of retweeting behavior for imbalanced dataset in microblogs

ZHAO Yu, SHAO Bilin, BIAN Genqing, SONG Dan   

  1. School of Management, Xi'an University of Architecture and Technology, Xi'an Shaanxi 710055, China
  • Received:2015-01-21 Revised:2015-03-18 Online:2015-07-10 Published:2015-07-17

摘要:

针对微博转发预测方法研究中的数据集不平衡问题,提出了一种融合过采样技术和随机森林(RF)算法的微博转发行为预测方法。首先,定义了个体信息、社交关系和微博主题3类与微博转发行为相关的特征,并基于信息增益算法实现了关键特征选取;其次,综合微博特征数据的特点来改进少数类样本合成过采样技术(SMOTE),对原始数据集进行非参数概率分布估计,并根据近似概率分布对数据集进行过采样处理,从而使正反例数据量达到平衡;最后,利用随机森林算法,依据微博转发关键特征进行分类器训练,并利用袋外(OOB)数据误差估计来分析和设置随机森林算法的相关参数。通过与基于决策树(DT)、支持向量机(SVM)、朴素贝叶斯(NB)和随机森林等算法的微博转发预测方法进行对比,所提方法整体性能优于基准方法中性能最优的SVM方法,召回率提高了8%,F值提高了5%。实验结果表明,所提方法在实际应用中能够有效提高微博转发行为预测的准确率。

关键词: 微博, 转发预测, 不均匀数据集, 过采样, 随机森林

Abstract:

Focusing on the issue that imbalanced dataset influencing the effect of prediction for retweeting behavior in microblogs, a novel predicting algorithm based on oversampling techniques and Random Forest (RF) algorithm was proposed. Firstly, the retweeting-related features, including individual information, social relationships and topic information, were defined. The key feature selection method was implemented based on information gain algorithm. Secondly, by considering the characteristics of the microblogs feature data, an improved algorithm for oversampling based on Synthetic Minority Over-sampling Technique (SMOTE) was proposed. In the course of this algorithm, the probability distribution of the original dataset was estimated based on nonparametric distribution estimation. In order to ensure a balanced number of positive examples and negative examples, an oversampling method was executed based on the improved SMOTE method, according to approximate probability distribution of the original dataset. Finally, a classifier based on random forest algorithm was trained, according to retweeting-related key features. The algorithm parameters of random forest were selected by analyzing the error estimation of Out Of Bag (OOB) data. By comparison with Decision Tree (DT), Support Vector Machine (SVM), Naive Bayesian (NB) and RF algorithms, which were used in the analysis for microblog retweeting behavior, the overall performance of the proposed method is superior to the method based on SVM, which obtains optimal results in all the baseline methods. The recall rate and F-measure of the proposed method are improved by 8%, 5% respectively. The experimental results show that the proposed method can effectively improve the prediction accuracy of microblog retweeting behavior analysis in practical application.

Key words: microblog, retweet prediction, imbalanced dataset, oversampling, Random Forest (RF)

中图分类号: