计算机应用 ›› 2015, Vol. 35 ›› Issue (10): 2757-2760.DOI: 10.11772/j.issn.1001-9081.2015.10.2757

• 第十五届中国机器学习会议(CCML2015)论文 • 上一篇    下一篇

基于共轭梯度的极速学习机

张沛洲1,2, 王熙照1, 顾迪3, 赵士欣4,5   

  1. 1. 河北大学 数学与信息科学学院, 河北 保定 071002;
    2. 河北省机器学习重点实验室(河北大学), 河北 保定 071000;
    3. 河北大学 计算机科学与技术学院, 河北 保定 071002;
    4. 河北大学 管理学院, 河北 保定 071002;
    5. 石家庄铁道大学 数理系, 石家庄 050043
  • 收稿日期:2015-06-15 修回日期:2015-07-07 出版日期:2015-10-10 发布日期:2015-10-14
  • 通讯作者: 张沛洲(1988-),男,河北邢台人,硕士研究生,主要研究方向:极速学习机、神经网络的并行处理,737202205@qq.com
  • 作者简介:王熙照(1963-),男,河北曲阳人,教授,博士生导师,主要研究方向:机器学习、不确定性信息处理;顾迪(1989-),男,江苏淮安人,硕士研究生,主要研究方向:极速学习机、神经网络的并行处理;赵士欣(1978-),女,河北石家庄人,讲师,博士研究生,主要研究方向:极速学习机、粗糙集、神经网络稳定性。
  • 基金资助:
    国家自然科学基金资助项目(61170040,71371063,71201111)。

Extreme learning machine based on conjugate gradient

ZHANG Peizhou1,2, WANG Xizhao1, GU Di3, ZHAO Shixin4,5   

  1. 1. College of Mathematics and Information Science, Hebei University, Baoding Hebei 071002, China;
    2. Hebei Province Key Laboratory of Machine Learning (Hebei University), Baoding Hebei 071000, China;
    3. School of Computer Science and Technology, Hebei University, Baoding Hebei 071002, China;
    4. College of Management, Hebei University, Baoding Hebei 071002, China;
    5. Department of Mathematics and Physics, Shijiazhuang Tiedao University, Shijiazhuang Hebei 050043, China
  • Received:2015-06-15 Revised:2015-07-07 Online:2015-10-10 Published:2015-10-14

摘要: 极速学习机(ELM)由于具有较快的训练速度和较好的泛化能力而被广泛的应用到很多的领域,然而在计算数据样例个数较大的情况下,它的训练速度就会下降,甚至会出现程序报错,因此提出在ELM模型中用改进的共轭梯度算法代替广义逆的计算方法。实验结果表明,与求逆矩阵的ELM算法相比,在同等泛化精度的条件下,共轭梯度ELM有着更快的训练速度。通过研究发现:基于共轭梯度的极速学习机算法不需要计算一个大型矩阵的广义逆,而大部分广义逆的计算依赖于矩阵的奇异值分解(SVD),但这种奇异值分解对于阶数很高的矩阵具有很低的效率;因为已经证明共轭梯度算法可通过有限步迭代找到其解,所以基于共轭剃度的极速学习机有着较高的训练速度,而且也比较适用于处理大数据。

关键词: 极速学习机, 广义逆, 共轭梯度法, 奇异值分解

Abstract: Extreme Learning Machine (ELM) has been widely used in many applications due to its fast convergence and good generalization performance. However, the training speed will slow down or ELM will make error when the number of the training samples reaches a certain scale. Conjugate gradient algorithm was introduced into the ELM model instead of the generalized inverse. The experimental results show that, under the condition of the same generalization accuracy, conjugate gradient-based ELM has faster training speed than that of ELM with matrix inversion. Because conjugate gradient-based ELM do not need to calculate the generalized inverse of a hidden layer output matrix, while most of the generalized inverse calculations depend on the matrix Singular Value Decomposition (SVD), which has low efficiency for a high-order matrix. It has been proved that the conjugate gradient algorithm can find the solution through iteration with finite steps, so the conjugate gradient-based ELM algorithm has faster training speed and is also suitable for processing big data.

Key words: Extreme Learning Machine (ELM), generalized inverse, conjugate gradient method, Singular Value Decomposition (SVD)

中图分类号: