Multiple birth support vector machine based on Rescaled Hinge loss function

doi:10.11772/j.issn.1001-9081.2020030381

Abstract

Abstract: As the performance of multi-classification learning model is effected by outliers, a Multiple Birth Support Vector Machine based on Rescaled Hinge loss function (RHMBSVM) was proposed. First, the corresponding optimization problem was constructed by introducing a bounded non-convex Rescaled Hinge loss function. Then, the conjugate function theory was used to make equivalent transformation of the optimization problem. Finally, the variable alternation strategy was used to form an iterative algorithm to solve the non-convex optimization problem. The penalty weight of each sample point was automatically adjusted during the solution process, so that the effect of outliers on K hyperplanes was eliminated, and the robustness was enhanced. The method of 5-fold cross-validation was used to complete the numerical experiment. Results show that, in the case of no outliers in the datasets, the accuracy of the proposed method is 1.11 percentage point higher than that of Multiple Birth Support Vector Machine (MBSVM) and 0.74 percentage point higher than that of Robust Support Vector Machine based on Rescaled Hinge loss function (RSVM-RHHQ); in the case of having outliers in the datasets, the accuracy of the proposed method is 2.10 percentage point higher than that of MBSVM and 1.47 percentage point higher than that of RSVM-RHHQ. Experimental results verify the robustness of the proposed method in solving multi-classification problems with outliers.

Key words: machine learning, optimization method, Support Vector Machine (SVM), Rescaled Hinge loss function, Multiple Birth Support Vector Machine (MBSVM)

摘要： 针对多分类学习模型性能会受异常值影响的问题，提出基于Rescaled Hinge损失函数的多子支持向量机（RHMBSVM）。首先，该方法通过引入有界、非凸的Rescaled Hinge损失函数来构建相应的优化问题；然后，利用共轭函数理论将优化问题作等价变换；最后，使用变量交替策略形成一个迭代算法来求解非凸优化问题，该方法在求解的过程中可自动调节每个样本点的惩罚权重，从而削弱了异常值对K个超平面的影响，增强了鲁棒性。使用5折交叉验证的方法进行数值实验，实验结果表明，在数据集无异常值的情况下，该方法的正确率比多子支持向量机（MBSVM）提升了1.11个百分点，比基于Rescaled Hinge损失函数的鲁棒支持向量机（RSVM-RHHQ）提升了0.74个百分点；在数据集有异常值的情况下，该方法的正确率比MBSVM提升了2.10个百分点，比RSVM-RHHQ提升了1.47个百分点。实验结果证明了所提方法在解决有异常值的多分类问题上的鲁棒性。

关键词: 机器学习, 最优化方法, 支持向量机, Rescaled Hinge损失函数, 多子支持向量机

CLC Number:

TP181

LI Hui, YANG Zhixia. Multiple birth support vector machine based on Rescaled Hinge loss function[J]. Journal of Computer Applications, 2020, 40(11): 3139-3145.

李卉, 杨志霞. 基于Rescaled Hinge损失函数的多子支持向量机[J]. 计算机应用, 2020, 40(11): 3139-3145.

References

[1] CORTES C,VAPNIK V. Support-vector networks[J]. Machine Learning,1995,20(3):273-297.
[2] HASSOUN M H,INTRATOR N,MCKAY S,et al. Fundamentals of artificial neural networks[J]. Computers in Physics,1998,10(2):137-137.
[3] OZCIFT A. SVM feature selection based rotation forest ensemble classifiers to improvecomputer-aided diagnosis of Parkinson disease[J]. Journal of Medical Systems,2012,36(4):2141-2147.
[4] 吴辰文, 李长生, 王伟, 等. 一种改进的SVM算法在乳腺癌诊断方面的应用[J]. 计算机工程与科学,2017,39(3):562-566. (WU C W,LI C S,WANG W,et al. Application of an improved support vector machine algorithm in the diagnosis of breast cancer[J]. Computer Engineering and Science,2017,39(3):562-566.)
[5] 张秋余, 竭洋, 李凯. 基于模糊支持向量机与决策树的文本分类器[J]. 计算机应用,2008,28(12):3227-3230.(ZHANG Q Y, JIE Y,LI K. Text classifier based on fuzzy support vector machine and decision tree[J]. Journal of Computer Applications,2008, 28(12):3227-3230.)
[6] LEE Y C. Application of support vector machines to corporate credit rating prediction[J]. Expert Systems with Applications, 2007,33(1):67-74.
[7] JAYADEVA,KHEMCHANDANI R,CHANDRA S. Twin support vector machines for pattern classification[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(5):905-910.
[8] MANGASARIAN O L,WILD E W. Multisurface proximal support vector machine classification via generalized eigenvalues[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2006, 28(1):69-74.
[9] TANVEER M. Robust and sparse linear programming twin support vector machines[J]. Cognitive Computation, 2015, 7(1):137-149.
[10] WANG C,YE Q,LUO P,et al. Robust capped L₁-norm twin support vector machine[J]. Neural Networks,2019,114:47-59.
[11] 刘雨桐, 李志清, 杨晓玲. 改进卷积神经网络在遥感图像分类中的应用[J]. 计算机应用,2018,38(4):949-954.(LIU Y T, LI Z Q,YANG X L. Application of improved convolution neural network in remote sensing image classification[J]. Journal of Computer Applications,2018,38(4):949-954.)
[12] LIAO R J,ZHENG H B,GRZYBOWSKI S,et al. A multiclass SVM-based classifier for transformer fault diagnosis using a particle swarm optimizer with time-varying acceleration coefficients[J]. International Transactions on Electrical Energy Systems, 2013,23(2):181-190.
[13] 施建宇, 潘泉, 张绍武, 等. 基于支持向量机融合网络的蛋白质折叠子识别研究[J]. 生物化学与生物物理进展,2006,33(2):155-162.(SHI J Y,PAN Q,ZHANG S W,et al. Protein fold recognition with support vector machines fusion network[J]. Progress in Biochemistry and Biophysics, 2006, 33(2):155-162.)
[14] YANG Z,SHAO Y,ZHANG X. Multiple birth support vector machine for multi-class classification[J]. Neural Computing and Applications,2013,22(S1):153-161.
[15] XU G,CAO Z,HU B,et al. Robust support vector machines based on the rescaled hinge loss function[J]. Pattern Recognition, 2017,63:139-148.
[16] SINGH A,POKHAREL R,PRINCIPE J. The C-loss function for pattern classification[J]. Pattern Recognition,2014,47(1):441-453.
[17] WU Y, LIU Y. Robust truncated hinge loss support vector machines[J]. Journal of the American Statistical Association, 2007,102(479):974-983.
[18] VAN GESTEL T, SUYKENS J A K, BAESENS B, et al. Benchmarking least squares support vector machine classifiers[J]. Machine Learning,2004,54(1):5-32.
[19] BOYD S, VANDENBERGHE L, FAYBUSOVICH L. Convex optimization[J]. IEEE Transactions on Automatic Control,2006, 51(11):1859-1859.
[20] BRAGA I,DO CARMO L P,BENATTI C C,et al. A note on parameter selection for support vector machines[C]//Proceedings of the 12th Mexican International Conference on Artificial Intelligence,LNCS 8266. Berlin:Springer,2013:233-244.