Hyper-spherical multi-task learning algorithm with adaptive grouping

doi:10.11772/j.issn.1001-9081.2014.07.2061

Journal of Computer Applications ›› 2014, Vol. 34 ›› Issue (7): 2061-2065.DOI: 10.11772/j.issn.1001-9081.2014.07.2061

• Artificial intelligence • Previous Articles Next Articles

Hyper-spherical multi-task learning algorithm with adaptive grouping

MAO Wentao¹,WANG Haicheng²,LIU Shangwang¹

1. College of Computer and Information Engineering, Henan Normal University, Xinxiang Henan 453007, China;
2. Department of Engineering Physics, Tsinghua University, Beijing 100084, China

Received:2014-01-14 Revised:2014-03-10 Online:2014-08-01 Published:2014-07-01
Contact: MAO Wentao

可自适应分组的超球多任务学习算法

毛文涛¹,王海成²,刘尚旺¹

1. 河南师范大学计算机与信息工程学院，河南新乡 453007;
2. 清华大学工程物理系，北京 100084

通讯作者: 毛文涛
作者简介:毛文涛(1980-),男,河南新乡人,副教授,博士,主要研究方向:机器学习、模式识别；王海成(1979-),男, 河南辉县人，硕士研究生, 主要研究方向:智能控制、模式识别；刘尚旺(1973-),男, 河南新乡人, 讲师,博士, 主要研究方向:模式识别、神经网络。
基金资助:
国家自然科学基金资助项目;河南省基础与前沿技术研究计划项目

Abstract

Abstract:

To solve the problem in most of conventional multi-task learning algorithms which evaluate risk independently for single task and lack uniform constraint across all tasks, a new hyper-spherical multi-task learning algorithm with adaptive grouping was proposed in this paper. Based on Extreme Learning Machine (ELM) as basic framework, this algorithm introduced hyper-spherical loss function to evaluate the risks of all tasks uniformly, and got decision model via iterative reweighted least squares solution. Furthermore, considering the existence of relatedness between tasks, this paper also constructed regularizer with grouping structure based on the assumption that related tasks had more similar weight vector, which would make the tasks in same group be trained independently. Finally, the optimization object was transformed into a mixed 0-1 programming problem, and a multi-objective method was utilized to identify optimal grouping structure and get model parameters. The simulation results on toy data and cylindrical vibration signal data show that the proposed algorithm outperforms state-of-the-art methods in terms of generalization performance and the ability of identifying inner structure in tasks.

摘要：

针对现有回归多任务学习中各任务独立评估风险、缺乏统一约束条件的缺点，提出了一种具有自适应分组能力的超球多任务学习算法。该算法以极限学习机(ELM)为基础形式，首先引入超球损失函数对所有任务的风险进行统一评估，并采用迭代再权最小二乘法求解；其次，考虑到任务之间关联度存在差异，基于相关性强的任务其权重向量也较相似的假设，构建带分组结构的正则项，使得同组内的任务独立进行训练，最终将优化目标转为混合0-1规划问题，并采用多目标优化方法自动确定模型参数和最优分组结构。基于仿真数据和圆柱壳振动信号数据的测试结果表明，该算法可有效识别出任务中的分组结构，同时与现有算法相比，可明显提高回归模型的泛化能力。

CLC Number:

TP181

MAO Wentao WANG Haicheng LIU Shangwang. Hyper-spherical multi-task learning algorithm with adaptive grouping[J]. Journal of Computer Applications, 2014, 34(7): 2061-2065.

毛文涛王海成刘尚旺. 可自适应分组的超球多任务学习算法[J]. 计算机应用, 2014, 34(7): 2061-2065.

References

［1］ARGYRIOU A, EVGENIOU T, PONTIL M. Convex multi-task feature learning［J］. Machine Learning, 2008, 73(3):243-272.
［2］GONG P, YE J, ZHANG C. Multi-stage multi-task feature learning［J］. Journal of Machine Learning Research, 2013, 14(1): 2979-3010.
［3］EVGENIOU T, MICCHELLI C A, PONTIL M. Learning multiple tasks with kernel methods［J］. Journal of Machine Learning Research, 2005, 6: 615-637.
［4］MAO W T, XU J, ZHAO S, et al. Research of multi-task learning based on extreme learning machine［J］. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 2013, 21(S2):75-85.
［5］DAUM H Ⅲ. Bayesian multitask learning with latent hierarchies［C］// Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington: AUAI Press, 2009:135-142.
［6］HUANG G-B, ZHU Q, SIEW C, et al. Extreme learning machine: Theory and applications［J］. Neurocomputing, 2006, 70(1/2/3): 489-501.
［7］HUANG G-B, ZHOU H, DING X, et al. Extreme learning machine for regression and multicalss classification［J］. IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, 2012, 42(2): 513-529.
［8］SNCHEZ-FERNNDEZ M,DE-PRADO-CUMPLIDO M,ARENAS-GARCA J, et al. SVM multiregression for nonlinear channel estimation in multiple-input multiple-output systems［J］. IEEE Transactions on Signal Processing, 2004, 52(8):2298-2307.
［9］PU J, JIANG Y G, WANG J, et al. Multiple task learning using iteratively reweighted least square［C］// Proceedings of the 23rd International Joint Conference on Artificial Intelligence. Menlo Park: AAAI Press, 2013: 1607-1613.
［10］MAO W, FENG Y, YAN G. New support vector machine with multi-input and multi-output based on adaptive grouping［J］. Journal of Xi'an Jiaotong University, 2013, 47(6): 50-54.(毛文涛, 冯云芝, 闫桂荣. 基于自适应分组的多输入多输出支持向量机算法研究［J］. 西安交通大学学报, 2013, 47(6): 50-54.)
［11］HUANG V L, SUGANTHAN P N, LIANG J J. Comprehensive learning particle swarm optimizer for solving multiobjective optimization problems［J］. International Journal of Intelligence System, 2006, 21(2): 209-226.
［12］MAO W. Research on model selection for support vector regression and its application in prediction of combined dynamics environment［D］. Xi'an: Xi'an Jiaotong University, 2011.(毛文涛. 支持向量回归机模型选择研究及在综合力学环境预示中的应用［D］. 西安: 西安交通大学, 2011.)

[1]	. Privacy preserving localization of surveillance images based on large vision models [J]. Journal of Computer Applications, 0, (): 0-0.
[2]	. Federated class-incremental learning method with multi-head self-attention for label semantic embedding [J]. Journal of Computer Applications, 0, (): 0-0.
[3]	Jingxin LIU, Wenjing HUANG, Liangsheng XU, Chong HUANG, Jiansheng WU. Unsupervised feature selection model with dictionary learning and sample correlation preservation [J]. Journal of Computer Applications, 2024, 44(12): 3766-3775.
[4]	Yifei SONG, Yi LIU. Fast adversarial training method based on data augmentation and label noise [J]. Journal of Computer Applications, 2024, 44(12): 3798-3807.
[5]	. Research review on explainable artificial intelligence in internet of things applications [J]. Journal of Computer Applications, 0, (): 0-0.
[6]	Jiachen YU, Ye YANG. Irregular object grasping by soft robotic arm based on clipped proximal policy optimization algorithm [J]. Journal of Computer Applications, 2024, 44(11): 3629-3638.
[7]	Zhijie LI, Xuhong LIAO, Yuanxiang LI, Qinglan LI. Disease sample classification algorithm by Bayesian network with gene association analysis [J]. Journal of Computer Applications, 2024, 44(11): 3449-3458.
[8]	Yuxin HUANG, Yiwang HUANG, Hui HUANG. Meta label correction method based on shallow network predictions [J]. Journal of Computer Applications, 2024, 44(11): 3364-3370.
[9]	HU Jie, ZHENG Qiyang, SUN Jun, ZHANG Yan. Multi-label classification model based on multi-relational label graph and local dynamic reconstruction learning [J]. Journal of Computer Applications, 0, (): 0-0.
[10]	Wenze CHAI, Jing FAN, Shukui SUN, Yiming LIANG, Jingfeng LIU. Overview of deep metric learning [J]. Journal of Computer Applications, 2024, 44(10): 2995-3010.
[11]	Chunyong YIN, Yongcheng ZHOU. Automatically adjusted clustered federated learning for double-ended clustering [J]. Journal of Computer Applications, 2024, 44(10): 3011-3020.
[12]	Feng CAO, Xiaoling YANG, Jianbing YI, Jun LI. Contradiction separation super-deduction method and application [J]. Journal of Computer Applications, 2024, 44(10): 3074-3080.
[13]	. Deep symbol regression method based on Transformer [J]. Journal of Computer Applications, 0, (): 0-0.
[14]	. RecipeFlavor: Recipe Recommendation Model Based on Flavor Embedding and Heterogeneous Graph Hierarchical Learning [J]. Journal of Computer Applications, 0, (): 0-0.
[15]	. Graph regularized Elastic Net Subspace Clustering [J]. Journal of Computer Applications, 0, (): 0-0.

Hyper-spherical multi-task learning algorithm with adaptive grouping

可自适应分组的超球多任务学习算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics