Journal of Computer Applications ›› 2022, Vol. 42 ›› Issue (1): 16-25.DOI: 10.11772/j.issn.1001-9081.2021010171
Special Issue: 人工智能
• Artificial intelligence • Previous Articles Next Articles
Received:
2021-01-29
Revised:
2021-04-24
Accepted:
2021-05-10
Online:
2021-06-04
Published:
2022-01-10
Contact:
Yue YANG
About author:
YANG Yue, born in 1995, M. S. candidate. Her research interests include machine learning, neural network.Supported by:
通讯作者:
杨悦
作者简介:
杨悦(1995—),女,云南玉溪人,硕士研究生,主要研究方向:机器学习、神经网络基金资助:
CLC Number:
Yue YANG, Shitong WANG. Four-layer multiple kernel learning method based on random feature mapping[J]. Journal of Computer Applications, 2022, 42(1): 16-25.
杨悦, 王士同. 基于随机特征映射的四层多核学习方法[J]. 《计算机应用》唯一官方网站, 2022, 42(1): 16-25.
Add to citation manager EndNote|Ris|BibTeX
URL: https://www.joca.cn/EN/10.11772/j.issn.1001-9081.2021010171
数据集 | 样本数 | 特征数 | 类别数 |
---|---|---|---|
Letter | 20 000 | 16 | 26 |
Robot | 5 456 | 24 | 4 |
Ecoli | 336 | 7 | 8 |
ACT | 46 728 | 8 | 2 |
Adult | 48 841 | 14 | 2 |
Australian | 690 | 14 | 2 |
Eye state | 14 980 | 15 | 2 |
Magic | 19 020 | 11 | 2 |
Car | 1 594 | 6 | 2 |
Tab. 1 Details of datasets
数据集 | 样本数 | 特征数 | 类别数 |
---|---|---|---|
Letter | 20 000 | 16 | 26 |
Robot | 5 456 | 24 | 4 |
Ecoli | 336 | 7 | 8 |
ACT | 46 728 | 8 | 2 |
Adult | 48 841 | 14 | 2 |
Australian | 690 | 14 | 2 |
Eye state | 14 980 | 15 | 2 |
Magic | 19 020 | 11 | 2 |
Car | 1 594 | 6 | 2 |
C值 | Ecoli | Australian | Car | |||
---|---|---|---|---|---|---|
Train_acc | Test_acc | Train_acc | Test_acc | Train_acc | Test_acc | |
2-24 | 87.36±1.37 | 82.18±2.38 | 88.34±0.71 | 85.17±0.81 | 92.11±0.24 | 90.79±0.30 |
2-22 | 86.72±1.70 | 80.30±2.27 | 87.87±0.48 | 84.78±0.82 | 91.78±0.18 | 90.79±0.40 |
2-20 | 87.40±1.66 | 83.37±1.46 | 87.99±0.39 | 84.69±0.92 | 91.63±0.21 | 90.42±0.48 |
2-18 | 86.77±1.78 | 81.49±2.47 | 87.50±0.50 | 84.44±0.89 | 91.39±0.33 | 90.77±0.58 |
2-16 | 85.06±2.32 | 79.21±3.07 | 87.68±0.65 | 84.93±0.60 | 90.98±0.33 | 90.29±0.34 |
2-14 | 85.32±1.21 | 80.89±1.66 | 87.41±0.50 | 84.59±0.76 | 90.75±0.36 | 90.27±0.46 |
2-12 | 84.55±2.64 | 79.21±3.04 | 87.62±0.50 | 84.69±0.75 | 90.54±0.31 | 89.96±0.36 |
2-10 | 82.47±1.54 | 78.81±2.43 | 87.37±0.46 | 84.35±0.44 | 90.25±0.36 | 89.60±0.43 |
Table 2 Experimental results with different regularization parameters
C值 | Ecoli | Australian | Car | |||
---|---|---|---|---|---|---|
Train_acc | Test_acc | Train_acc | Test_acc | Train_acc | Test_acc | |
2-24 | 87.36±1.37 | 82.18±2.38 | 88.34±0.71 | 85.17±0.81 | 92.11±0.24 | 90.79±0.30 |
2-22 | 86.72±1.70 | 80.30±2.27 | 87.87±0.48 | 84.78±0.82 | 91.78±0.18 | 90.79±0.40 |
2-20 | 87.40±1.66 | 83.37±1.46 | 87.99±0.39 | 84.69±0.92 | 91.63±0.21 | 90.42±0.48 |
2-18 | 86.77±1.78 | 81.49±2.47 | 87.50±0.50 | 84.44±0.89 | 91.39±0.33 | 90.77±0.58 |
2-16 | 85.06±2.32 | 79.21±3.07 | 87.68±0.65 | 84.93±0.60 | 90.98±0.33 | 90.29±0.34 |
2-14 | 85.32±1.21 | 80.89±1.66 | 87.41±0.50 | 84.59±0.76 | 90.75±0.36 | 90.27±0.46 |
2-12 | 84.55±2.64 | 79.21±3.04 | 87.62±0.50 | 84.69±0.75 | 90.54±0.31 | 89.96±0.36 |
2-10 | 82.47±1.54 | 78.81±2.43 | 87.37±0.46 | 84.35±0.44 | 90.25±0.36 | 89.60±0.43 |
数据集 | BLS | FRMFNN | MK-FRMFNN |
---|---|---|---|
[Nf ×Nm,Ne,C] | [Nf ×Nm,Ne,C] | [Nf ×Nm,Ne, | |
Letter | [11×9, 2 500, 2-24] | [11×11, 2 940, 2-24] | [11×10, 960, 10-2, 1, 2-24] |
Robot | [5×6, 2 930, 2-24] | [3×9, 2 690, 2-24] | [3×7, 940, 10-2, 10, 2-24] |
Ecoli | [2×2, 245, 2-24] | [2×2, 235, 2-24] | [2×2, 41, 10-2, 10-1, 2-24] |
ACT | [3×8, 1 040, 2-24] | [4×5, 990, 2-24] | [4×5, 300, 10-1, 10, 2-24] |
Adult | [4×7, 1 890, 2-24] | [8×5, 1 020, 2-24] | [7×4, 330, 102, 1, 2-24] |
Australian | [7×6, 84, 2-24] | [7×4, 80, 2-24] | [6×3, 28, 10, 102, 2-24] |
Eye state | [11×10, 2 880, 2-24] | [11×11, 2 000, 2-24] | [10×10, 675, 10-1, 10-1, 2-24] |
Magic | [10×9, 1 900, 2-24] | [11×9, 810, 2-24] | [10×8, 310, 1, 1, 2-24] |
Car | [4×2, 1 520, 2-24] | [5×3, 1 130, 2-24] | [5×2, 390, 1, 102, 2-24] |
Tab.3 Parameter settings of BLS, FRMFNN and MK-FRMFNN models
数据集 | BLS | FRMFNN | MK-FRMFNN |
---|---|---|---|
[Nf ×Nm,Ne,C] | [Nf ×Nm,Ne,C] | [Nf ×Nm,Ne, | |
Letter | [11×9, 2 500, 2-24] | [11×11, 2 940, 2-24] | [11×10, 960, 10-2, 1, 2-24] |
Robot | [5×6, 2 930, 2-24] | [3×9, 2 690, 2-24] | [3×7, 940, 10-2, 10, 2-24] |
Ecoli | [2×2, 245, 2-24] | [2×2, 235, 2-24] | [2×2, 41, 10-2, 10-1, 2-24] |
ACT | [3×8, 1 040, 2-24] | [4×5, 990, 2-24] | [4×5, 300, 10-1, 10, 2-24] |
Adult | [4×7, 1 890, 2-24] | [8×5, 1 020, 2-24] | [7×4, 330, 102, 1, 2-24] |
Australian | [7×6, 84, 2-24] | [7×4, 80, 2-24] | [6×3, 28, 10, 102, 2-24] |
Eye state | [11×10, 2 880, 2-24] | [11×11, 2 000, 2-24] | [10×10, 675, 10-1, 10-1, 2-24] |
Magic | [10×9, 1 900, 2-24] | [11×9, 810, 2-24] | [10×8, 310, 1, 1, 2-24] |
Car | [4×2, 1 520, 2-24] | [5×3, 1 130, 2-24] | [5×2, 390, 1, 102, 2-24] |
数据集 | BLS | FRMFNN | MK-FRMFNN | |||
---|---|---|---|---|---|---|
Train_acc | Test_acc | Train_acc | Test_acc | Train_acc | Test_acc | |
Letter | 96.58±0.57 | 93.42±0.53 | 96.59±0.006 0 | 93.41±0.005 3 | 96.58±0.004 7 | 93.43±0.004 3 |
Robot | 96.78±0.27 | 90.19±0.32 | 96.35±0.002 3 | 90.00±0.004 8 | 94.36±0.005 8 | 89.54±0.006 3 |
Ecoli | 89.11±1.05 | 82.08±1.50 | 88.00±1.42 | 82.57±1.73 | 87.36±1.37 | 82.18±2.38 |
ACT | 99.21±0.01 | 99.32±0.01 | 99.21±0.01 | 99.31±0.01 | 99.19±0.03 | 99.29±0.04 |
Adult | 79.54±0.08 | 79.72±0.16 | 79.68±0.05 | 79.78±0.07 | 79.60±0.08 | 79.71±0.11 |
Australian | 90.48±0.61 | 83.53±1.25 | 89.83±0.41 | 84.06±0.86 | 88.34±0.71 | 85.17±0.81 |
Eye state | 86.19±0.36 | 85.12±0.28 | 86.07±0.60 | 85.07±0.57 | 86.10±0.31 | 85.07±0.25 |
Magic | 83.40±0.21 | 82.85±0.26 | 83.43±0.28 | 82.82±0.24 | 82.58±0.17 | 82.11±0.17 |
Car | 92.05±0.27 | 90.94±0.42 | 92.07±0.35 | 90.67±0.38 | 92.11±0.24 | 90.79±0.30 |
Table 4 Accuracy comparison of BLS, FRMFNN and MK-FRMFNN models on datasets for classification
数据集 | BLS | FRMFNN | MK-FRMFNN | |||
---|---|---|---|---|---|---|
Train_acc | Test_acc | Train_acc | Test_acc | Train_acc | Test_acc | |
Letter | 96.58±0.57 | 93.42±0.53 | 96.59±0.006 0 | 93.41±0.005 3 | 96.58±0.004 7 | 93.43±0.004 3 |
Robot | 96.78±0.27 | 90.19±0.32 | 96.35±0.002 3 | 90.00±0.004 8 | 94.36±0.005 8 | 89.54±0.006 3 |
Ecoli | 89.11±1.05 | 82.08±1.50 | 88.00±1.42 | 82.57±1.73 | 87.36±1.37 | 82.18±2.38 |
ACT | 99.21±0.01 | 99.32±0.01 | 99.21±0.01 | 99.31±0.01 | 99.19±0.03 | 99.29±0.04 |
Adult | 79.54±0.08 | 79.72±0.16 | 79.68±0.05 | 79.78±0.07 | 79.60±0.08 | 79.71±0.11 |
Australian | 90.48±0.61 | 83.53±1.25 | 89.83±0.41 | 84.06±0.86 | 88.34±0.71 | 85.17±0.81 |
Eye state | 86.19±0.36 | 85.12±0.28 | 86.07±0.60 | 85.07±0.57 | 86.10±0.31 | 85.07±0.25 |
Magic | 83.40±0.21 | 82.85±0.26 | 83.43±0.28 | 82.82±0.24 | 82.58±0.17 | 82.11±0.17 |
Car | 92.05±0.27 | 90.94±0.42 | 92.07±0.35 | 90.67±0.38 | 92.11±0.24 | 90.79±0.30 |
数据集 | 算法 | 参数 | 精度/% |
---|---|---|---|
Ecoli | simpleMKL | C=23 | 73.30 |
easyMKL | λ=0.3 | 75.26 | |
GLMKL | C=24 | 73.29 | |
NLMKL | C=24 | 75.26 | |
MK-FRMFNN | C=2-24 | 82.18 | |
Car | simpleMKL | C=23 | 81.36 |
easyMKL | λ=0.1 | 83.33 | |
GLMKL | C=2-1 | 80.48 | |
NLMKL | C=25 | 83.15 | |
MK-FRMFNN | C=2-24 | 90.79 | |
Australian | simpleMKL | C=24 | 78.18 |
easyMKL | λ=0.1 | 80.28 | |
GLMKL | C=20 | 79.43 | |
NLMKL | C=25 | 80.12 | |
MK-FRMFNN | C=2-24 | 85.17 |
Tab. 5 Experimental parameters and results on three datasets
数据集 | 算法 | 参数 | 精度/% |
---|---|---|---|
Ecoli | simpleMKL | C=23 | 73.30 |
easyMKL | λ=0.3 | 75.26 | |
GLMKL | C=24 | 73.29 | |
NLMKL | C=24 | 75.26 | |
MK-FRMFNN | C=2-24 | 82.18 | |
Car | simpleMKL | C=23 | 81.36 |
easyMKL | λ=0.1 | 83.33 | |
GLMKL | C=2-1 | 80.48 | |
NLMKL | C=25 | 83.15 | |
MK-FRMFNN | C=2-24 | 90.79 | |
Australian | simpleMKL | C=24 | 78.18 |
easyMKL | λ=0.1 | 80.28 | |
GLMKL | C=20 | 79.43 | |
NLMKL | C=25 | 80.12 | |
MK-FRMFNN | C=2-24 | 85.17 |
1 | SCHÖLKOPF B, SMOLA A J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond[M]. 1st ed. Cambridge: MIT Press, 2001:10-41. |
2 | WANG Z, CHEN S C, SUN T K. MultiK-MHKS: a novel multiple kernel learning algorithm[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2):348-353. 10.1109/tpami.2007.70786 |
3 | ZHENG H, YE Q L, JIN Z. A novel multiple kernel sparse representation based classification for face recognition[J]. KSII Transactions on Internet and Information Systems, 2014, 8(4): 1463-1480. 10.3837/tiis.2014.04.017 |
4 | SAUNDERS C, STITSON M O, WESTON J, et al. Support vector machine[J]. Computer Science, 2002, 1(4):1-28. |
5 | SUYKENS J A K, VANDEWALLE J. Least squares support vector machine classifiers[J]. Neural Processing Letters, 1999, 9(3):293-300. 10.1023/a:1018628609742 |
6 | 陶剑文,王士同. 领域适应核支持向量机[J]. 自动化学报, 2012, 38(5):797-811. 10.3724/SP.J.1004.2012.00797 |
TAO J W, WANG S T. Kernel support vector machine for domain adaptation[J]. Acta Automatica Sinica, 2012, 38(5):797-811. 10.3724/SP.J.1004.2012.00797 | |
7 | HUANG G B, ZHOU H M, DING X J, et al. Extreme learning machine for regression and multiclass classification[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(2):513-529. 10.1109/tsmcb.2011.2168604 |
8 | CHEN C L P, LIU Z L. Broad learning system: an effective and efficient incremental learning system without the need for deep architecture[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(1):10-24. 10.1109/tnnls.2017.2716952 |
9 | 杨悦,王士同. 随机特征映射的四层神经网络及其增量学习[J]. 计算机科学与探索, 2021, 15(7):1265-1278. 10.3389/fpsyg.2021.760060 |
YANG Y, WANG S T. A novel four-layer neural network and its incremental learning based on randomly mapped features[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(7):1265-1278. 10.3389/fpsyg.2021.760060 | |
10 | HOERL A E, KENNARD R W. Ridge regression: biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1):55-67. 10.1080/00401706.1970.10488634 |
11 | BEN-HUR A, NOBLE W S. Kernel methods for predicting protein-protein interactions[J]. Bioinformatics, 2005, 21(S1):i38-i46. 10.1093/bioinformatics/bti1016 |
12 | 钟志鹏,张立保. 基于多核学习特征融合的人脸表情识别[J]. 计算机应用, 2015, 35(S2):245-249. 10.1007/s10489-021-02575-0 |
ZHONG Z P, ZHANG L B. Facial expression recognition based on multi-feature fusion by multi-kernel SVM[J]. Journal of Computer Applications, 2015, 35(S2):245-249. 10.1007/s10489-021-02575-0 | |
13 | KINGSBURY N, TAY D B H, PALANISWAMI M. Multi-scale kernel methods for classification[C]// Proceedings of the 2005 IEEE Workshop on Machine Learning for Signal Processing. Piscataway: IEEE, 2005:43-48. 10.1109/mlsp.2005.1532872 |
14 | DAMOULAS T, GIROLAMI M A, NOTES A. Probabilistic multi-class multi-kernel learning: on protein fold recognition and remote homology detection[J]. Bioinformatics, 2008, 24(10):1264-1270. 10.1093/bioinformatics/btn112 |
15 | 汪洪桥,孙富春,蔡艳宁,等. 多核学习方法[J]. 自动化学报, 2010, 36(8):1037-1050. 10.3724/SP.J.1004.2010.01037 |
WANG H Q, SUN F C, CAI Y N, et al. On multiple kernel learning methods[J]. Acta Automatica Sinica, 2010, 36(8):1037-1050. 10.3724/SP.J.1004.2010.01037 | |
16 | 周志华. 机器学习[M]. 北京:清华大学出版社, 2016:63-66, 137-140. 10.21436/inbom.12382432 |
ZHOU Z H. Machine Learning[M]. Beijing: Tsinghua University Press, 2016:63-66, 137-140. 10.21436/inbom.12382432 | |
17 | AIOLLI F, DONINI M. EasyMKL: a scalable multiple kernel learning algorithm[J]. Neurocomputing, 2015, 169:215-224. 10.1016/j.neucom.2014.11.078 |
18 | RAKOTOMAMONJY A, BACH F R, CANU S, et al. Simple MKL[J]. Journal of Machine Learning Research, 2008, 9: 2491-2521. |
19 | VARMA M, BABU B R. More generality in efficient multiple kernel learning[C]// Proceedings of the 26th International Conference on Machine Learning. New York: ACM, 2009: 1065-1072. 10.1145/1553374.1553510 |
20 | CORTES C, MOHRI M, ROSTAMIZADEH A. Learning non-linear combinations of kernels[C]// Proceedings of the 22nd International Conference on Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc., 2009:396-404. 10.1109/icassp.2003.1198859 |
21 | XU Z L, JIN R, YANG H Q, et al. Simple and efficient multiple kernel learning by group lasso[C]// Proceedings of the 27th International Conference on Machine Learning. Madison, WI: Omnipress, 2010: 1175-1182. |
22 | TIBSHIRANI R. Regression shrinkage and selection via the lasso[J]. Journal of the Royal Statistical Society: Series B (Methodological), 1996, 58(1):267-288. 10.1111/j.2517-6161.1996.tb02080.x |
23 | BOYD S, PARIKH N, CHU E, et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends in Machine Learning, 2010, 3(1):1-122. 10.1561/2200000016 |
24 | GOLDSTEIN T, O’DONOGHUE B, SETZER S, et al. Fast alternating direction optimization methods[J]. SIAM Journal on Imaging Sciences, 2014, 7(3) 1588-1623. 10.1137/120896219 |
25 | CRISTIANINTNELLO. An introduction to support vector machines and other kernel-based learning methods[J]. Kybernetes, 2000, 30(1):103-115. 10.1108/k.2001.30.1.103.6 |
26 | DUA D, GRAFF C. UCI repository of machine learning repository[DS/OL]. [2021-01-12].. |
[1] | Shuaihua ZHANG, Shufen ZHANG, Mingchuan ZHOU, Chao XU, Xuebin CHEN. Malicious traffic detection model based on semi-supervised federated learning [J]. Journal of Computer Applications, 2024, 44(11): 3487-3494. |
[2] | Jie WU, Xuezhong QIAN, Wei SONG. Personalized federated learning based on similarity clustering and regularization [J]. Journal of Computer Applications, 2024, 44(11): 3345-3353. |
[3] | Mengjie LAN, Jianping CAI, Lan SUN. Self-regularization optimization methods for Non-IID data in federated learning [J]. Journal of Computer Applications, 2023, 43(7): 2073-2081. |
[4] | Wenbo LI, Bo LIU, Lingling TAO, Fen LUO, Hang ZHANG. Deep spectral clustering algorithm with L1 regularization [J]. Journal of Computer Applications, 2023, 43(12): 3662-3667. |
[5] | Xiangzhou QI, Hongjie XING. Centered kernel alignment based multiple kernel one-class support vector machine [J]. Journal of Computer Applications, 2022, 42(2): 349-356. |
[6] | Lili FAN, Guifu LU, Ganyi TANG, Dan YANG. Low-rank representation subspace clustering method based on Hessian regularization and non-negative constraint [J]. Journal of Computer Applications, 2022, 42(1): 115-122. |
[7] | LIN Junchao, WAN Yuan. Self-adaptive multi-measure unsupervised feature selection method with structured graph optimization [J]. Journal of Computer Applications, 2021, 41(5): 1282-1289. |
[8] | Qian LIU, Hongyuan WANG, Liang CAO, Boyan SUN, Yu XIAO, Ji ZHANG. Cloth-changing person re-identification based on joint loss capsule network [J]. Journal of Computer Applications, 2021, 41(12): 3596-3601. |
[9] | Xian CHEN, Liying HU, Xiaowei LIN, Lifei CHEN. Directed graph clustering algorithm based on kernel nonnegative matrix factorization [J]. Journal of Computer Applications, 2021, 41(12): 3447-3454. |
[10] | Hua LI, Guifu LU, Qinru YU. Manifold regularized nonnegative matrix factorization based on clean data [J]. Journal of Computer Applications, 2021, 41(12): 3492-3498. |
[11] | SHEN Marui, LI Jincheng, ZHANG Ya, ZOU Jian. Magnetic resonance image reconstruction algorithm via non-convex total variation regularization [J]. Journal of Computer Applications, 2020, 40(8): 2358-2364. |
[12] | WANG Zhiyuan, JIANG Ailian, MUHAMMAD Osman. Unsupervised feature selection method based on regularized mutual representation [J]. Journal of Computer Applications, 2020, 40(7): 1896-1900. |
[13] | XIE Qi, XU Xu, CHENG Gengguo, CHEN Heping. Feature selection algorithm based on new forest optimization algorithm [J]. Journal of Computer Applications, 2020, 40(5): 1266-1271. |
[14] | GENG Yuanqian, WU Chuansheng, LIU Wen. Mixed non-convex and non-smooth regularization constraint based blind image restoration [J]. Journal of Computer Applications, 2020, 40(4): 1171-1176. |
[15] | Meng ZENG, Bin NING, Zhihua CAI, Qiong GU. Hyperspectral band selection based on deep adversarial subspace clustering [J]. Journal of Computer Applications, 2020, 40(2): 381-385. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||