可自适应分组的超球多任务学习算法

doi:10.11772/j.issn.1001-9081.2014.07.2061

计算机应用 ›› 2014, Vol. 34 ›› Issue (7): 2061-2065.DOI: 10.11772/j.issn.1001-9081.2014.07.2061

可自适应分组的超球多任务学习算法

毛文涛¹,王海成²,刘尚旺¹

1. 河南师范大学计算机与信息工程学院，河南新乡 453007;
2. 清华大学工程物理系，北京 100084

收稿日期:2014-01-14 修回日期:2014-03-10 出版日期:2014-07-01 发布日期:2014-08-01
通讯作者: 毛文涛
作者简介:毛文涛(1980-),男,河南新乡人,副教授,博士,主要研究方向:机器学习、模式识别；王海成(1979-),男, 河南辉县人，硕士研究生, 主要研究方向:智能控制、模式识别；刘尚旺(1973-),男, 河南新乡人, 讲师,博士, 主要研究方向:模式识别、神经网络。
基金资助:
国家自然科学基金资助项目;河南省基础与前沿技术研究计划项目

Hyper-spherical multi-task learning algorithm with adaptive grouping

MAO Wentao¹,WANG Haicheng²,LIU Shangwang¹

1. College of Computer and Information Engineering, Henan Normal University, Xinxiang Henan 453007, China;
2. Department of Engineering Physics, Tsinghua University, Beijing 100084, China

Received:2014-01-14 Revised:2014-03-10 Online:2014-07-01 Published:2014-08-01
Contact: MAO Wentao

摘要/Abstract

摘要：

针对现有回归多任务学习中各任务独立评估风险、缺乏统一约束条件的缺点，提出了一种具有自适应分组能力的超球多任务学习算法。该算法以极限学习机(ELM)为基础形式，首先引入超球损失函数对所有任务的风险进行统一评估，并采用迭代再权最小二乘法求解；其次，考虑到任务之间关联度存在差异，基于相关性强的任务其权重向量也较相似的假设，构建带分组结构的正则项，使得同组内的任务独立进行训练，最终将优化目标转为混合0-1规划问题，并采用多目标优化方法自动确定模型参数和最优分组结构。基于仿真数据和圆柱壳振动信号数据的测试结果表明，该算法可有效识别出任务中的分组结构，同时与现有算法相比，可明显提高回归模型的泛化能力。

Abstract:

To solve the problem in most of conventional multi-task learning algorithms which evaluate risk independently for single task and lack uniform constraint across all tasks, a new hyper-spherical multi-task learning algorithm with adaptive grouping was proposed in this paper. Based on Extreme Learning Machine (ELM) as basic framework, this algorithm introduced hyper-spherical loss function to evaluate the risks of all tasks uniformly, and got decision model via iterative reweighted least squares solution. Furthermore, considering the existence of relatedness between tasks, this paper also constructed regularizer with grouping structure based on the assumption that related tasks had more similar weight vector, which would make the tasks in same group be trained independently. Finally, the optimization object was transformed into a mixed 0-1 programming problem, and a multi-objective method was utilized to identify optimal grouping structure and get model parameters. The simulation results on toy data and cylindrical vibration signal data show that the proposed algorithm outperforms state-of-the-art methods in terms of generalization performance and the ability of identifying inner structure in tasks.

中图分类号:

TP181

毛文涛王海成刘尚旺. 可自适应分组的超球多任务学习算法[J]. 计算机应用, 2014, 34(7): 2061-2065.

MAO Wentao WANG Haicheng LIU Shangwang. Hyper-spherical multi-task learning algorithm with adaptive grouping[J]. Journal of Computer Applications, 2014, 34(7): 2061-2065.

参考文献

［1］ARGYRIOU A, EVGENIOU T, PONTIL M. Convex multi-task feature learning［J］. Machine Learning, 2008, 73(3):243-272.
［2］GONG P, YE J, ZHANG C. Multi-stage multi-task feature learning［J］. Journal of Machine Learning Research, 2013, 14(1): 2979-3010.
［3］EVGENIOU T, MICCHELLI C A, PONTIL M. Learning multiple tasks with kernel methods［J］. Journal of Machine Learning Research, 2005, 6: 615-637.
［4］MAO W T, XU J, ZHAO S, et al. Research of multi-task learning based on extreme learning machine［J］. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 2013, 21(S2):75-85.
［5］DAUM H Ⅲ. Bayesian multitask learning with latent hierarchies［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington: AUAI Press, 2009:135-142.
［6］HUANG G-B, ZHU Q, SIEW C, et al. Extreme learning machine: Theory and applications［J］. Neurocomputing, 2006, 70(1/2/3): 489-501.
［7］HUANG G-B, ZHOU H, DING X, et al. Extreme learning machine for regression and multicalss classification［J］. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 2012, 42(2): 513-529.
［8］SNCHEZ-FERNNDEZ M,DE-PRADO-CUMPLIDO M,ARENAS-GARCA J, et al. SVM multiregression for nonlinear channel estimation in multiple-input multiple-output systems［J］. IEEE Transactions on Signal Processing, 2004, 52(8):2298-2307.
［9］PU J, JIANG Y G, WANG J, et al. Multiple task learning using iteratively reweighted least square［C］// Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2013: 1607-1613.
［10］MAO W, FENG Y, YAN G. New support vector machine with multi-input and multi-output based on adaptive grouping［J］. Journal of Xi'an Jiaotong University, 2013, 47(6): 50-54.(毛文涛, 冯云芝, 闫桂荣. 基于自适应分组的多输入多输出支持向量机算法研究［J］. 西安交通大学学报, 2013, 47(6): 50-54.)
［11］HUANG V L, SUGANTHAN P N, LIANG J J. Comprehensive learning particle swarm optimizer for solving multiobjective optimization problems［J］. International Journal of Intelligence System, 2006, 21(2): 209-226.
［12］MAO W. Research on model selection for support vector regression and its application in prediction of combined dynamics environment［D］. Xi'an: Xi'an Jiaotong University, 2011.(毛文涛. 支持向量回归机模型选择研究及在综合力学环境预示中的应用［D］. 西安: 西安交通大学, 2011.)

[1]	陈露张晓霞于洪. 基于先验知识的非负矩阵半可解释三因子分解算法[J]. 计算机应用, 0, (): 0-0.
[2]	李晓杰崔超然宋广乐苏雅茜吴天泽张春云. 基于时序超图卷积神经网络的股票趋势预测方法[J]. 计算机应用, 0, (): 0-0.
[3]	李宗正周恺卿丁雷欧云. 基于基因交换的自适应人工鱼群算法[J]. 计算机应用, 0, (): 0-0.
[4]	刘清华廖士中. 基于随机素描方法的在线核回归[J]. 计算机应用, 0, (): 0-0.
[5]	刘忠慧王梓宥闵帆. 近似概念的遗传生成算法及其推荐应用[J]. 计算机应用, 0, (): 0-0.
[6]	汪敏冯婷婷闵帆唐洪明闫建平廖纪佳. 页岩气储层预测的多标签主动学习算法[J]. 计算机应用, 0, (): 0-0.
[7]	王雅辉, 钱宇华, 刘郭庆. 基于模糊优势互补互信息的有序决策树算法[J]. 计算机应用, 2021, 41(10): 2785-2792.
[8]	任柯舟, 彭甫镕, 郭鑫, 王喆, 张晓静. 动态融合社交信息的社会化推荐[J]. 计算机应用, 2021, 41(10): 2806-2812.
[9]	张志浩, 林耀进, 卢舜, 郭晨, 王晨曦. 缺失标记下基于类属属性的多标记特征选择[J]. 计算机应用, 2021, 41(10): 2849-2857.
[10]	顾军华樊帅李宁宁张素琪. 基于知识图偏好注意力网络的长短期推荐模型及其更新方法[J]. 计算机应用, 0, (): 0-0.
[11]	王海起王志海李留珂孔浩然王琼徐建波. 基于网格划分的城市短时交通流量时空预测模型[J]. 计算机应用, 0, (): 0-0.
[12]	陈恒王思懿李正光李冠宇刘鑫. 基于关系记忆的胶囊网络知识图谱嵌入模型[J]. 计算机应用, 0, (): 0-0.
[13]	张成, 万源, 强浩鹏. 基于知识蒸馏的深度无监督离散跨模态哈希[J]. 计算机应用, 2021, 41(9): 2523-2531.
[14]	孙浩艺, 王传美, 丁义明. 基于隐藏层输出矩阵的极限学习机算法优化[J]. 计算机应用, 2021, 41(9): 2481-2488.
[15]	卞凌志, 王直杰. 基于增强多维多粒度级联森林的信用评分模型[J]. 计算机应用, 2021, 41(9): 2539-2544.

可自适应分组的超球多任务学习算法

Hyper-spherical multi-task learning algorithm with adaptive grouping

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics