关键词: 不平衡分类, Xgboost, 梯度分布, 损失贡献, 损失贡献密度

Abstract: In order to solve the problem of low detection rate of minority class by ensemble learning model eXtreme gradient boosting （Xgboost） in the binary classification problem, an improved Xgboost algorithm based on gradient distribution harmonized strategy called Loss Contribution Gradient Harmonized Algorithm (LCGHA)-Xgboost was proposed. Firstly, Loss Contribution (LC) was defined to simulate the losses of the samples in Xgboost algorithm. Secondly, by defining Loss Contribution Density (LCD), the difficulty of samples being correctly classified in Xgboost algorithm was measured. Finally, a gradient distribution harmonized algorithm called LCGHA was proposed to dynamically adjust the one order gradient distribution of samples according to the LCD. In the algorithm, the losses of hard samples (mainly in minority class) were indirectly increased, and the losses of easy samples (mainly in majority class) were indirectly reduced, making Xgboost algorithm tend to learn the hard samples. The experimental results show that compared with three ensemble learning algorithms Xgboost, GBDT (Gradient Boosting Decision Tree) and Random_Forest, LCGHA-Xgboost has the recall increased by 5.4%-16.7%, and Area Under the Curve (AUC) improved by 0.94%-7.41% on multiple UCI datasets, and the Recall increased by 44.4%-383.3%, and AUC improved by 5.8%-35.6% on WebSpam-UK2007 and DC2010 datasets. LCGHA-Xgboost can effectively improve the classification and detection ability for minority class, and reduce the classification error rate of minority class.

Key words: imbalanced classification, Xgboost, gradient distribution, loss contribution, loss contribution density