计算机应用 ›› 2013, Vol. 33 ›› Issue (07): 1912-1916.DOI: 10.11772/j.issn.1001-9081.2013.07.1912

• 先进计算 • 上一篇    下一篇

基于交替方向乘子法的非光滑损失坐标优化算法

高乾坤,王玉军,王惊晓   

  1. 陆军军官学院 十一系,合肥 230031
  • 收稿日期:2013-01-25 修回日期:2013-02-24 出版日期:2013-07-01 发布日期:2013-07-06
  • 通讯作者: 高乾坤
  • 作者简介:高乾坤(1989-),男,安徽合肥人,硕士研究生,主要研究方向:模式识别、人工智能;王玉军(1984-),男,江苏盐城人,硕士研究生,主要研究方向:模式识别、人工智能;王惊晓(1988-),女,安徽合肥人,硕士研究生,主要研究方向:模式识别、人工智能。
  • 基金资助:

    国家自然科学基金资助项目(61273296, 60975040)

New coordinate optimization method for non-smooth losses based on alternating direction method of multipliers

GAO Qiankun,WANG Yujun,WANG Jingxiao   

  1. The 11th Department, Chinese People's Liberation Army Officer Academy, Hefei Anhui 230031, China
  • Received:2013-01-25 Revised:2013-02-24 Online:2013-07-06 Published:2013-07-01
  • Contact: GAO Qiankun

摘要: 交替方向乘子法(ADMM)在机器学习问题中已有一些实际应用。针对大规模数据的处理和非光滑损失凸优化问题,将镜面下降方法引入原ADMM批处理算法,得到了一种新的改进算法,并在此基础上提出了一种求解非光滑损失凸优化问题的坐标优化算法。该算法具有操作简单、计算高效的特点。通过详尽的理论分析,证明了新算法的收敛性,在一般凸条件下其具有目前最优的收敛速度。最后与相关算法进行了对比,实验结果表明该算法在保证解稀疏性的同时拥有更快的收敛速度。

关键词: 机器学习, 交替方向乘子法, 坐标优化, 大规模, 非光滑损失

Abstract: Alternating Direction Method of Multipliers (ADMM) already has some practical applications in machine learning field. In order to adapt to the large-scale data processing and non-smooth loss convex optimization problem, the original batch ADMM algorithm was improved by using mirror descent method, and a new coordinate optimization algorithm was proposed for solving non-smooth loss convex optimization. This new algorithm has a simple operation and efficient computation. Through detailed theoretical analysis, the convergence of the new algorithm is verified and it also has the optimal convergence rate in general convex condition. Finally, the experimental results compared with the state-of-art algorithms demonstrate it gets better convergence rate under the sparsity of solution.

Key words: machine learning, Alternating Direction Method of Multipliers (ADMM), coordinate optimization, large-scale, non-smooth loss

中图分类号: