拉格朗日支持向量回归的有限牛顿算法

doi:10.3724/SP.J.1087.2012.02504

计算机应用 ›› 2012, Vol. 32 ›› Issue (09): 2504-2507.DOI: 10.3724/SP.J.1087.2012.02504

拉格朗日支持向量回归的有限牛顿算法

郑逢德^*,张鸿宾

北京工业大学计算机学院,北京 100124

收稿日期:2012-03-19 修回日期:2012-05-22 发布日期:2012-09-01 出版日期:2012-09-01
通讯作者: 郑逢德
作者简介:郑逢德(1980-),男,河南新乡人,博士研究生,主要研究方向:核方法、机器学习; 张鸿宾(1944-),男,教授,博士生导师,主要研究方向:模式识别、神经网络、数字水印。
基金资助:
国家自然科学基金资助项目(60775011)

Finite Newton algorithm for Lagrangian support vector regression

ZHENG Feng-de^*,ZHANG Hong-bin

College of Computer Science,Beijing University of Technology,Beijing 100124,China

Received:2012-03-19 Revised:2012-05-22 Online:2012-09-01 Published:2012-09-01
Contact: Feng-De ZHENG

摘要/Abstract

摘要： 拉格朗日支持向量回归是一种有效的快速回归算法,求解时需要对维数等于样本数加一的矩阵求逆,求解需要较多的迭代次数才能收敛。采用一种Armijo步长有限牛顿迭代算法求解拉格朗日支持向量回归的优化问题,只需有限次求解一组线性等式而不需要求解二次规划问题,该方法具有全局收敛和有限步终止的性质。在多个标准数据集上的实验验证了所提算法的有效性和快速性。

关键词: 支持向量回归, 拉格朗日支持向量机, 有限牛顿算法, 迭代算法

Abstract: Lagrangian Support Vector Regression (SVR) is an effective algorithm and its solution is obtained by taking the inverse of a matrix of order equaling the number of samples plus one, but needs many times to terminate from a starting point. This paper proposed a finite Armijo-Newton algorithm solving the Lagrangian SVR's optimization problem. A solution was obtained by solving a system of linear equations at a finite number of times rather than solving a quadratic programming problem. The proposed method has the advantage that the result optimization problem is solved with global convergence and finite-step termination. The experimental results on several benchmark datasets indicate that the proposed algorithm is fast, and shows good generalization performance.

Key words: Support Vector Regression (SVR), Lagrangian support vector machine, finite Newton algorithm, iterative algorithm

中图分类号:

TP391.4

郑逢德张鸿宾. 拉格朗日支持向量回归的有限牛顿算法[J]. 计算机应用, 2012, 32(09): 2504-2507.

ZHENG Feng-de ZHANG Hong-bin. Finite Newton algorithm for Lagrangian support vector regression[J]. Journal of Computer Applications, 2012, 32(09): 2504-2507.

参考文献

[1]CRISTIANINI N,SHAWE-TAYLOR J.Kernel methods for pattern analysis[M].Cambridge:Cambridge University Press,2004.
[2]BURGES C J C.A tutorial on support vector machines for pattern recognition[J].Data Mining Knowledge Discovery,1998,2(2):121-167.
[3]SCHOLKOPF B,SMOLA A J.Learning with kernels[M].Cambridge:MIT Press,2002.
[4]VAPNIK V.The nature of statistical learning theory[M].Berlin:Springer-Verlag,1995.
[5]VAPNIK V.An overview of statistical learning theory[J].IEEE Transactions on Neural Networks,1999,10(5):988-999.
[6]CORTES C,VAPNIK V.Support vector networks[J].Maching Learning,1995,20(3):273-297.
[7]CRISTIANINI N,SHAWE-TAYLOR J.An Introduction to support vector machines and other kernel-based learning methods[M].北京:机械工业出版社,2005.
[8]MULLER L R,MIKA S,RATSCH G,et al.An introduction to kernel-based learning[J].IEEE Transactions on Neural Networks,2001,12(2):181-201.
[9]JOHAN A K S,TONY V G,JOS D B,et al.Least squares support vector machines[M].[S.l.]:World Scientific Press,2002.
[10]PENG X J.TSVR:An efficient twin support vector machine for regression[J].Neural Networks,2010,23(3):365-342.
[11]BALASUNDARAM S,KAPIL.On Lagrangian support vector re-gression[J].Expert Systems with Applications,2010,37(12):8784-8792.
[12]MANGASARIAN O L,MUSICANT D R.Lagrangian support vector machine[J].Journal of Machine Learning Research,2001,1:161-177.
[13]MANGASARIAN O L,MUSICANT D R.Finite Newton method for Lagrangian support vector machine[J].Neurocomputing,2003 55(1/2):39-55.
[14]GUNN S R.SVM Matlab toolbox[EB/OL].[2011-10-10].http:// www.isis.ecs.soton.ac.uk/resources/svminfo.
[15]WANG.M,HUA X S,SONG Y,et al.Semisupervised kernel regression[C]// ICDM'06:Proceedings of the 6th International Conference on Data Mining.Washington,DC:IEEE Computer Society,2006:1130-1135.
[16]ZHOU Z H,LI M.Semi-supervised regression with cotraining-style algorithms[J].IEEE Transactions on Knowledge and Data Engineering,2007,19(11):1479-1493.
[17]Department of Statistics at Carnegie Mellon University.Statlib datasets[EB/OL].[2009-03-06].http://lib.stat.cmu.edu/datasets/.
[18]Center for Spoken Language Understanding.M-G Datasets[EB/OL].[2009-03-06].http://www.cse.ogi.edu/~ericwan/datahtml.
[19]Department of Information and Computer Sciences at University of California.UCI repository for machine learning databases[EB/OL].[2009-03-06].http://www.ics.uci.edu/_mlearn/MLRepository.html.

拉格朗日支持向量回归的有限牛顿算法

Finite Newton algorithm for Lagrangian support vector regression

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	杨雅倩, 唐绍婷. 基于扩展的低阶多元广义线性模型的脑节点识别方法[J]. 计算机应用, 2018, 38(10): 3048-3052.
[2]	黄凤, 王晓明. 增强的单幅图像自学习超分辨方法[J]. 计算机应用, 2017, 37(9): 2636-2642.
[3]	覃雄派, 陈跃国, 王邦国. 大数据分析的应用案例——投资模型的稳健性[J]. 计算机应用, 2017, 37(3): 660-667.
[4]	管非凡, 郁梅, 宋洋, 邵华, 蒋刚毅. 高动态范围图像客观质量评价方法[J]. 计算机应用, 2017, 37(3): 695-698.
[5]	王定成, 赵友志, 陈北京, 陆一祎. 多输出数据依赖核支持向量回归快速学习算法[J]. 计算机应用, 2017, 37(3): 746-749.
[6]	王春荣, 夏尔冬, 吴龙, 刘建军, 熊昌炯. 基于改进支持向量回归算法的移动机器人定位[J]. 计算机应用, 2016, 36(9): 2545-2549.
[7]	王坤, 员晓阳, 王力. 基于改进模糊支持向量回归模型的机场能源需求预测[J]. 计算机应用, 2016, 36(5): 1458-1463.
[8]	马允, 王晓东, 章联军. 基于空域自然场景统计的无参考立体图像质量评价模型[J]. 计算机应用, 2016, 36(3): 783-788.
[9]	李俊山, 仝奇, 叶霞, 许元. 基于相空间重构的自适应残差修正支持向量回归预测算法[J]. 计算机应用, 2016, 36(11): 3229-3233.
[10]	张汝波, 孟雷, 史长亭. 基于微重启和部分客观马尔可夫决策模型的智能水下机器人软件自修复方法[J]. 计算机应用, 2015, 35(8): 2375-2379.
[11]	游子毅, 陈世国, 王义. 基于ε-支持向量回归理论的区域交通信号智能控制[J]. 计算机应用, 2015, 35(5): 1361-1366.
[12]	陈长远韩军伟胡新韬程塨郭雷. 基于视觉显著熵与Object Bank特征的图像记忆性模型[J]. 计算机应用, 2013, 33(11): 3176-3178.
[13]	史荧中王士同蒋亦樟刘培林. 迁移学习支持向量回归机[J]. 计算机应用, 2013, 33(11): 3084-3089.
[14]	孙德厂史海波刘昶. 支持向量回归在不确定面向订单装配环境下交货期预测中的应用[J]. 计算机应用, 2013, 33(08): 2362-2365.
[15]	李松顾巧论高铁杠. 信息嵌入位置自适应选择的鲁棒水印算法[J]. 计算机应用, 2013, 33(02): 438-446.