可自适应分组的超球多任务学习算法

doi:10.11772/j.issn.1001-9081.2014.07.2061

计算机应用 ›› 2014, Vol. 34 ›› Issue (7): 2061-2065.DOI: 10.11772/j.issn.1001-9081.2014.07.2061

可自适应分组的超球多任务学习算法

毛文涛¹,王海成²,刘尚旺¹

1. 河南师范大学计算机与信息工程学院，河南新乡 453007;
2. 清华大学工程物理系，北京 100084

收稿日期:2014-01-14 修回日期:2014-03-10 发布日期:2014-08-01 出版日期:2014-07-01
通讯作者: 毛文涛
作者简介:毛文涛(1980-),男,河南新乡人,副教授,博士,主要研究方向:机器学习、模式识别；王海成(1979-),男, 河南辉县人，硕士研究生, 主要研究方向:智能控制、模式识别；刘尚旺(1973-),男, 河南新乡人, 讲师,博士, 主要研究方向:模式识别、神经网络。
基金资助:
国家自然科学基金资助项目;河南省基础与前沿技术研究计划项目

Hyper-spherical multi-task learning algorithm with adaptive grouping

MAO Wentao¹,WANG Haicheng²,LIU Shangwang¹

1. College of Computer and Information Engineering, Henan Normal University, Xinxiang Henan 453007, China;
2. Department of Engineering Physics, Tsinghua University, Beijing 100084, China

Received:2014-01-14 Revised:2014-03-10 Online:2014-08-01 Published:2014-07-01
Contact: MAO Wentao

摘要/Abstract

摘要：

针对现有回归多任务学习中各任务独立评估风险、缺乏统一约束条件的缺点，提出了一种具有自适应分组能力的超球多任务学习算法。该算法以极限学习机(ELM)为基础形式，首先引入超球损失函数对所有任务的风险进行统一评估，并采用迭代再权最小二乘法求解；其次，考虑到任务之间关联度存在差异，基于相关性强的任务其权重向量也较相似的假设，构建带分组结构的正则项，使得同组内的任务独立进行训练，最终将优化目标转为混合0-1规划问题，并采用多目标优化方法自动确定模型参数和最优分组结构。基于仿真数据和圆柱壳振动信号数据的测试结果表明，该算法可有效识别出任务中的分组结构，同时与现有算法相比，可明显提高回归模型的泛化能力。

Abstract:

To solve the problem in most of conventional multi-task learning algorithms which evaluate risk independently for single task and lack uniform constraint across all tasks, a new hyper-spherical multi-task learning algorithm with adaptive grouping was proposed in this paper. Based on Extreme Learning Machine (ELM) as basic framework, this algorithm introduced hyper-spherical loss function to evaluate the risks of all tasks uniformly, and got decision model via iterative reweighted least squares solution. Furthermore, considering the existence of relatedness between tasks, this paper also constructed regularizer with grouping structure based on the assumption that related tasks had more similar weight vector, which would make the tasks in same group be trained independently. Finally, the optimization object was transformed into a mixed 0-1 programming problem, and a multi-objective method was utilized to identify optimal grouping structure and get model parameters. The simulation results on toy data and cylindrical vibration signal data show that the proposed algorithm outperforms state-of-the-art methods in terms of generalization performance and the ability of identifying inner structure in tasks.

中图分类号:

TP181

毛文涛王海成刘尚旺. 可自适应分组的超球多任务学习算法[J]. 计算机应用, 2014, 34(7): 2061-2065.

MAO Wentao WANG Haicheng LIU Shangwang. Hyper-spherical multi-task learning algorithm with adaptive grouping[J]. Journal of Computer Applications, 2014, 34(7): 2061-2065.

参考文献

［1］ARGYRIOU A, EVGENIOU T, PONTIL M. Convex multi-task feature learning［J］. Machine Learning, 2008, 73(3):243-272.
［2］GONG P, YE J, ZHANG C. Multi-stage multi-task feature learning［J］. Journal of Machine Learning Research, 2013, 14(1): 2979-3010.
［3］EVGENIOU T, MICCHELLI C A, PONTIL M. Learning multiple tasks with kernel methods［J］. Journal of Machine Learning Research, 2005, 6: 615-637.
［4］MAO W T, XU J, ZHAO S, et al. Research of multi-task learning based on extreme learning machine［J］. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 2013, 21(S2):75-85.
［5］DAUM H Ⅲ. Bayesian multitask learning with latent hierarchies［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington: AUAI Press, 2009:135-142.
［6］HUANG G-B, ZHU Q, SIEW C, et al. Extreme learning machine: Theory and applications［J］. Neurocomputing, 2006, 70(1/2/3): 489-501.
［7］HUANG G-B, ZHOU H, DING X, et al. Extreme learning machine for regression and multicalss classification［J］. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 2012, 42(2): 513-529.
［8］SNCHEZ-FERNNDEZ M,DE-PRADO-CUMPLIDO M,ARENAS-GARCA J, et al. SVM multiregression for nonlinear channel estimation in multiple-input multiple-output systems［J］. IEEE Transactions on Signal Processing, 2004, 52(8):2298-2307.
［9］PU J, JIANG Y G, WANG J, et al. Multiple task learning using iteratively reweighted least square［C］// Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2013: 1607-1613.
［10］MAO W, FENG Y, YAN G. New support vector machine with multi-input and multi-output based on adaptive grouping［J］. Journal of Xi'an Jiaotong University, 2013, 47(6): 50-54.(毛文涛, 冯云芝, 闫桂荣. 基于自适应分组的多输入多输出支持向量机算法研究［J］. 西安交通大学学报, 2013, 47(6): 50-54.)
［11］HUANG V L, SUGANTHAN P N, LIANG J J. Comprehensive learning particle swarm optimizer for solving multiobjective optimization problems［J］. International Journal of Intelligence System, 2006, 21(2): 209-226.
［12］MAO W. Research on model selection for support vector regression and its application in prediction of combined dynamics environment［D］. Xi'an: Xi'an Jiaotong University, 2011.(毛文涛. 支持向量回归机模型选择研究及在综合力学环境预示中的应用［D］. 西安: 西安交通大学, 2011.)

[1]	宫智宇王士同. 面向重尾噪声图像分类的残差网络学习方法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[2]	王虎王晓峰李可马云洁. 融合多头自注意力的标签语义嵌入联邦类增量学习方法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[3]	刘晶鑫, 黄雯静, 徐亮胜, 黄冲, 吴建生. 字典学习与样本关联保持结合的无监督特征选择模型[J]. 《计算机应用》唯一官方网站, 2024, 44(12): 3766-3775.
[4]	宋逸飞, 柳毅. 基于数据增强和标签噪声的快速对抗训练方法[J]. 《计算机应用》唯一官方网站, 2024, 44(12): 3798-3807.
[5]	赵小阳许新征李仲年. 物联网应用中的可解释人工智能研究综述[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[6]	余家宸, 杨晔. 基于裁剪近端策略优化算法的软机械臂不规则物体抓取[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3629-3638.
[7]	李志杰, 廖旭红, 李元香, 李青蓝. 基于基因关联分析的贝叶斯网络疾病样本分类算法[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3449-3458.
[8]	黄雨鑫, 黄贻望, 黄辉. 基于浅层网络预测的元标签校正方法[J]. 《计算机应用》唯一官方网站, 2024, 44(11): 3364-3370.
[9]	胡婕郑启扬孙军张龑. 基于多关系标签图和局部动态重构学习的多标签分类模型[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[10]	柴汶泽, 范菁, 孙书魁, 梁一鸣, 刘竟锋. 深度度量学习综述[J]. 《计算机应用》唯一官方网站, 2024, 44(10): 2995-3010.
[11]	尹春勇, 周永成. 双端聚类的自动调整聚类联邦学习[J]. 《计算机应用》唯一官方网站, 2024, 44(10): 3011-3020.
[12]	曹锋, 杨小玲, 易见兵, 李俊. 矛盾体分离超演绎方法及应用[J]. 《计算机应用》唯一官方网站, 2024, 44(10): 3074-3080.
[13]	许鹏程何磊李川钱炜祺赵暾. 基于Transformer的深度符号回归方法[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[14]	颜文婧王瑞东左敏张青川. 基于风味嵌入异构图层次学习的食谱推荐模型[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.
[15]	郭书剑余节约尹学松. 图正则化弹性网子空间聚类[J]. 《计算机应用》唯一官方网站, 0, (): 0-0.

可自适应分组的超球多任务学习算法

Hyper-spherical multi-task learning algorithm with adaptive grouping

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics