Journal of Computer Applications ›› 2020, Vol. 40 ›› Issue (11): 3139-3145.DOI: 10.11772/j.issn.1001-9081.2020030381

• Artificial intelligence • Previous Articles     Next Articles

Multiple birth support vector machine based on Rescaled Hinge loss function

LI Hui, YANG Zhixia   

  1. College of Mathematics and System Sciences, Xinjiang University, Urumqi Xinjiang 830046, China
  • Received:2020-03-30 Revised:2020-05-25 Online:2020-11-10 Published:2020-06-22
  • Supported by:
    This work is partially supported by the Key Natural Science Program of Education Department of Xinjiang Autonomous Region (XJEDU2018I002).

基于Rescaled Hinge损失函数的多子支持向量机

李卉, 杨志霞   

  1. 新疆大学 数学与系统科学学院, 乌鲁木齐 830046
  • 通讯作者: 杨志霞(1977-),女,新疆奎屯人,教授,博士,主要研究方向:最优化方法、机器学习;yangzhx@xju.edu.cn
  • 作者简介:李卉(1994-),女,新疆库尔勒人,硕士研究生,主要研究方向:机器学习
  • 基金资助:
    新疆自治区教育厅自然科学重点项目(XJEDU2018I002)。

Abstract: As the performance of multi-classification learning model is effected by outliers, a Multiple Birth Support Vector Machine based on Rescaled Hinge loss function (RHMBSVM) was proposed. First, the corresponding optimization problem was constructed by introducing a bounded non-convex Rescaled Hinge loss function. Then, the conjugate function theory was used to make equivalent transformation of the optimization problem. Finally, the variable alternation strategy was used to form an iterative algorithm to solve the non-convex optimization problem. The penalty weight of each sample point was automatically adjusted during the solution process, so that the effect of outliers on K hyperplanes was eliminated, and the robustness was enhanced. The method of 5-fold cross-validation was used to complete the numerical experiment. Results show that, in the case of no outliers in the datasets, the accuracy of the proposed method is 1.11 percentage point higher than that of Multiple Birth Support Vector Machine (MBSVM) and 0.74 percentage point higher than that of Robust Support Vector Machine based on Rescaled Hinge loss function (RSVM-RHHQ); in the case of having outliers in the datasets, the accuracy of the proposed method is 2.10 percentage point higher than that of MBSVM and 1.47 percentage point higher than that of RSVM-RHHQ. Experimental results verify the robustness of the proposed method in solving multi-classification problems with outliers.

Key words: machine learning, optimization method, Support Vector Machine (SVM), Rescaled Hinge loss function, Multiple Birth Support Vector Machine (MBSVM)

摘要: 针对多分类学习模型性能会受异常值影响的问题,提出基于Rescaled Hinge损失函数的多子支持向量机(RHMBSVM)。首先,该方法通过引入有界、非凸的Rescaled Hinge损失函数来构建相应的优化问题;然后,利用共轭函数理论将优化问题作等价变换;最后,使用变量交替策略形成一个迭代算法来求解非凸优化问题,该方法在求解的过程中可自动调节每个样本点的惩罚权重,从而削弱了异常值对K个超平面的影响,增强了鲁棒性。使用5折交叉验证的方法进行数值实验,实验结果表明,在数据集无异常值的情况下,该方法的正确率比多子支持向量机(MBSVM)提升了1.11个百分点,比基于Rescaled Hinge损失函数的鲁棒支持向量机(RSVM-RHHQ)提升了0.74个百分点;在数据集有异常值的情况下,该方法的正确率比MBSVM提升了2.10个百分点,比RSVM-RHHQ提升了1.47个百分点。实验结果证明了所提方法在解决有异常值的多分类问题上的鲁棒性。

关键词: 机器学习, 最优化方法, 支持向量机, Rescaled Hinge损失函数, 多子支持向量机

CLC Number: