基于新的森林优化算法的特征选择算法

doi:10.11772/j.issn.1001-9081.2019091614

计算机应用 ›› 2020, Vol. 40 ›› Issue (5): 1266-1271.DOI: 10.11772/j.issn.1001-9081.2019091614

基于新的森林优化算法的特征选择算法

谢琪¹, 徐旭², 程耕国¹, 陈和平¹

1.武汉科技大学信息科学与工程学院，武汉 430081
2.格勒诺布尔高等商学院高等商业学院，格勒诺布尔 38000, 法国

收稿日期:2019-09-23 修回日期:2019-10-16 出版日期:2020-05-10 发布日期:2020-05-15
通讯作者: 谢琪（1984—）
作者简介:谢琪（1984—），男，福建福州人，博士研究生，主要研究方向：机器学习；徐旭（1995—），男，湖北武汉人，硕士研究生，主要研究方向：机器学习、量化投资；程耕国（1947—），男，安徽绩溪人，教授，博士生导师，博士，主要研究方向：控制理论及应用；陈和平（1956—），男，湖北武汉人，教授，博士生导师，CCF会员，主要研究方向：人工智能、大数据。
基金资助:
国家自然科学基金资助项目（61702381, 61602351）。

Feature selection algorithm based on new forest optimization algorithm

XIE Qi¹, XU Xu², CHENG Gengguo¹, CHEN Heping¹

1.School of Information Science and Engineering, Wuhan University of Science and Technology, WuhanHubei 430081, China
2.Graduate School of Business, Grenoble Graduate School of Business, Grenoble 38000, France

Received:2019-09-23 Revised:2019-10-16 Online:2020-05-10 Published:2020-05-15
Contact: XIE Qi，born in 1984，Ph. D. candidate. His research interests include machine learning.
About author:XIE Qi，born in 1984，Ph. D. candidate. His research interests include machine learning.XU Xu，born in 1995，M. S. candidate. His research interests include machine learning, quantitative investment.CHENG Gengguo，born in 1947，Ph. D., professor. His research interests include control theory and applications.CHEN Heping，born in 1956，M. S., professor. His research interests include artificial intelligence, big data.
Supported by:
This work is partially supported by the National Natural Science Foundation of China (61702381, 61602351).

摘要/Abstract

摘要：

针对传统的基于森林优化算法的特征选择算法在初始化阶段、候选森林生成阶段和更新阶段存在的问题，提出了一种新的基于森林优化算法的特征选择算法。该算法在初始化阶段采用皮尔森相关系数和L₁正则化方法代替随机初始化策略；在候选森林生成阶段，采用优劣树分开和差额补足的方法解决优劣树不完备问题；在更新阶段，将与最优树精度相同但维度不同的树木添加到森林中。在实验中，所提算法采用与传统的基于森林优化算法的特征选择算法相同的实验数据和实验参数，分别测试了小维度、中维度和大维度数据。实验结果表明，在2个大维度数据和2个中维度数据上，所提算法的分类精度和维度缩减能力均高于传统的基于森林优化算法的特征选择算法。实验结果验证了所提算法在处理特征选择问题的有效性。

关键词: 特征选择, L₁正则化, 候选森林, 更新机制, 森林优化算法

Abstract:

A new feature selection algorithm using forest optimization algorithm was proposed, which aimed at solving the problems of the traditional feature selection using forest optimization algorithm in the stages of initialization, candidate forest generation and updating. In the algorithm, Pearson correlation coefficient and L₁ regularization method were used to replace the random initialization strategy in the initialization stage, the methods of separating good and bad trees and fulfilling the difference were used to solve the problems of incompletion of good and bad trees in the candidate forest generation stage, and trees having the same precision but different dimension with the optimal tree were added to the forest in the updating stage. In the experiments, with the same experimental data and experimental parameters, the proposed algorithm and the traditional feature selection using forest optimization algorithm were used to test the small, medium and large dimension data respectively. The experimental results show that the proposed algorithm is better than the traditional feature selection using forest optimization algorithm in the classification performance and dimension reduction ability on two medium and two large dimension data. The experimental results prove the effectiveness of the proposed algorithm in solving feature selection problems.

Key words: feature selection, L₁ regularization, candidate forest, updating mechanism, forest optimization algorithm

中图分类号:

TP312

谢琪, 徐旭, 程耕国, 陈和平. 基于新的森林优化算法的特征选择算法[J]. 计算机应用, 2020, 40(5): 1266-1271.

XIE Qi, XU Xu, CHENG Gengguo, CHEN Heping. Feature selection algorithm based on new forest optimization algorithm[J]. Journal of Computer Applications, 2020, 40(5): 1266-1271.

参考文献

1 姚旭,王晓丹,张玉玺,等 . 特征选择方法综述[J]. 控制与决策, 2012, 27(2): 161-166, 192. YAO X , WANG X D , ZHANG Y X , et al . Summary of feature selection algorithms[J]. Control and Decision, 2012, 27(2): 161-166, 192.
2 毛勇,周晓波,夏铮,等 . 特征选择算法研究综述[J]. 模式识别与人工智能, 2007, 20(2):211-218. MAO Y , ZHOU X B , XIA Z , et al . A survey for study of feature selection algorithms[J]. Pattern Recognition and Artificial Intelligence, 2007, 20(2):211-218.
3 BISHOP C M . Pattern Recognition and Machine Learning[M]. New York: Springer, 2006:40-65.
4 周志华 .机器学习[M]. 北京:清华大学出版社, 2016:247-261 ZHOU Z H . Machine Learning[M]. Beijing: Tsinghua University Press, 2016:247-261.
5 XUE B , ZHANG M , BROWNE W N , et al . A survey on evolutionary computation approaches to feature selection[J]. IEEE Transactions on Evolutionary Computation, 2016, 20(4): 606-626.
6 GUYON I , ELISSEEFF A . An introduction to variable and feature selection[J]. Journal of Machine Learning Research, 2003, 3: 1157-1182.
7 LI J , CHENG K , WANG S , et al . Feature selection: a data perspective[J]. ACM Computing Surveys, 2018, 50(6): No.94.
8 ÖZŞEN S , GüNEŞ S . Attribute weighting via genetic algorithms for Attribute Weighted Artificial Immune System (AWAIS) and its application to heart disease and liver disorders problems[J]. Expert Systems with Applications, 2009, 36(1): 386-392.
9 TAHIR M A , BOURIDANE A , KURUGOLLU F . Simultaneous feature selection and feature weighting using hybrid tabu search/K-nearest neighbor classifier[J]. Pattern Recognition Letters, 2007, 28(4): 438-446.
10 HALL M A . Correlation-based feature selection of discrete and numeric class machine learning[EB/OL]. [2019-10-14].https://core.ac.uk/download/pdf/29195068.pdf.
11 GHAEMI M , FEIZI-DERAKHSHI M R . Forest optimization algorithm[J]. Expert Systems with Applications, 2014, 41(15): 6676-6687.
12 GHAEMI M , FEIZI-DERAKHSHI M R . Feature selection using forest optimization algorithm[J]. Pattern Recognition, 2016, 60: 121-129.
13 HUANG J , CAI Y , XU X . A hybrid genetic algorithm for feature selection wrapper based on mutual information[J]. Pattern Recognition Letters, 2007, 28(13): 1825-1844.
14 XUE B , ZHANG M , BROWNE W N . Particle swarm optimisation for feature selection in classification: novel initialisation and updating mechanisms[J]. Applied Soft Computing, 2014, 18:261-276.
15 MOUSTAKIDIS S P , THEOCHARIS J . SVM-FuzCoC: a novel SVM-based feature selection method using a fuzzy complementary criterion[J]. Pattern Recognition, 2010, 43(11): 3712-3729.
16 DASH M , LIU H . Feature selection for classification[J]. Intelligent Data Analysis, 1997, 1(1/2/3/4): 131-156.
17 LIU H , MOTODA H . Feature Selection for Knowledge Discovery and Data Mining, SECS 454[M]. Boston: Springer, 1998: 153-172.
18 MOLINA L C , BELANCHE L , à NEBOT . Feature selection algorithms: a survey and experimental evaluation[C]// Proceedings of the 2002 IEEE International Conference on Data Mining. Piscataway: IEEE, 2002: 306-313.
19 HALL M A . Correlation-based feature selection for discrete and numeric class machine learning[C]// Proceedings of the 17th International Conference on Machine Learning. San Francisco, CA: Morgan Kaufmann Publisher Inc., 2000: 359-366.
20 KIRA K , RENDELL L A . The feature selection problem: traditional methods and a new algorithm[C]// Proceedings of the 10th National Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press, 1992:129-134.
21 KONONENKO I . Estimating attributes: analysis and extension of relief[C]// Proceedings of the 1994 European Conference on Machine Learning, LNCS 784. Berlin: Springer, 1994: 171-182.
22 LIU H , SETIONO R . Feature selection and classification — a probabilistic wrapper approach[C]// Proceedings of the 1996 International Conference on Industrial and Engineering Applications of Artificial Intelligence and Expert Systems. Philadelphia, PA: Gordon and Breach Science Publishers, 1996: 419-424.
23 HSU W H . Genetic wrappers for feature selection in decision tree induction and variable ordering in Bayesian network structure learning[J]. Information Sciences, 2004, 163(1/2/3): 103-122.
24 CHIANG L H , PELL R J . Genetic algorithms combined with discriminant analysis for key variable identification[J]. Journal of Process Control, 2004, 14(2): 143-155.
25 GUYON I , WESTON J , BARNHILL S , et al . Gene selection for cancer classification using support vector machines[J]. Machine Learning, 2002, 46(1/2/3): 389-422.
26 TIKHONOV A N , ARSENIN V Y . Solutions of Ill-Posed Problems[M]. Washington, DC： V.H. Winston & Sons, 1977: 64-68.
27 TIBSHIRANI R . Regression shrinkage and selection via the Lasso[J]. Journal of the Royal Statistical Society: Series B (Methodological), 1996, 58(1): 267-288.
28 姚新,刘勇 .进化算法研究进展[J].计算机学报, 1995, 18(9): 694-706. (YAO X, LIU Y. Advances in evolutionary algorithms[J]. Chinese Journal of Computers, 1995, 18(9): 694-706.)
29 COELLO C A C , LAMONT G B , VELDHUIZEN D A VAN . Multi-criteria decision making[M]// Evolutionary Algorithms for Solving Multi-Objective Problems, 2nd ed. Boston: Springer, 2007:515-545.
30 YANG J , HONAVAR V . Feature subset selection using a genetic algorithm[M]// LIU H, MOTODA H. Feature Extraction, Construction and Selection, SECS 453. Cham: Springer, 1998: 117-136.
31 DONG H , LI T , DING R , et al . A novel hybrid genetic algorithm with granular information for feature selection and optimization[J]. Applied Soft Computing, 2018, 65:33-46.
32 DORIGO M , STüTZLE T . Ant colony optimization: overview and recent advances[M]// Handbook of Metaheuristics, ISOR 272. Cham: Springer, 2019: 311-351.
33 DORIGO M , BIRATTARI M . Ant colony optimization[M]// SAMMUT C, WEBB G I. Encyclopedia of Machine Learning. Cham: Springer, 2010: 25-33.
34 KABIR M M , SHAHJAHAN M , MURASE K . A new hybrid ant colony optimization algorithm for feature selection[J]. Expert Systems with Applications, 2012, 39(3):3747-3763.
35 WAN Y , WANG M , YE Z , et al . A feature selection method based on modified binary coded ant colony optimization algorithm[J]. Applied Soft Computing, 2016, 49:248-258.
36 POLI R , KENNEDY J , BLACKWELL T . Particle swarm optimization[J]. Swarm Intelligence, 2007, 1(1):33-57.
37 KENNEDY J . Particle swarm optimization[M]// SAMMUT C, WEBB G I. Encyclopedia of Machine Learning. Boston: Springer, 2011: 760-766.
38 ZHANG Y , GONG D , HU Y , et al . Feature selection algorithm based on bare bones particle swarm optimization[J]. Neurocomputing, 2015, 148:150-157.
39 TRAN B , XUE B , ZHANG M . Overview of particle swarm optimization for feature selection in classification[C]// Proceedings of the 2014 Asia-Pacific Conference on Simulated Evolution and Learning, LNCS 8886. Cham: Springer, 2014:605-617.
40 聂大干 . 森林优化算法的改进及离散化研究[D]. 兰州:兰州大学, 2016:7-10. (NIE D G. Research om improvements and discretization of forest optimization algorithm[D]. Lanzhou: Lanzhou University, 2016:7-10.)
41 ALTMAN N S . An introduction to kernel and nearest-neighbor nonparametric regression[J]. The American Statistician, 1992, 46(3):175-185.
42 CORTES C , VAPNIK V . Support-vector networks[J]. Machine Learning, 1995, 20(3):273-297.
43 SALZBERG S L . Book review: C4.5: Programs for Machine Learning by J. Ross Quinlan. Morgan Kaufmann Publishers, Inc., 1993[J]. Machine Learning, 1994, 16(3): 235-240.
44 Machine Learning Repository UCI . Welcome to the UC Irvine Machine Learning Repository! [DB/OL]. [2019-10-14]. http://www.ics.uci.edu/~mlearn/MLRepository.html.

[1]	湛航, 何朗, 黄樟灿, 李华峰, 张蔷, 谈庆. 改进的基于层次距离的基因表达式编程特征选择分类算法[J]. 计算机应用, 2021, 41(9): 2658-2667.
[2]	祝承, 赵晓琦, 赵丽萍, 焦玉宏, 朱亚飞, 陈建英, 周伟, 谭颖. 基于谱聚类半监督特征选择的功能磁共振成像数据分类[J]. 计算机应用, 2021, 41(8): 2288-2293.
[3]	李蒙蒙, 秦伟, 刘艺, 刁兴春. 结合头脑风暴优化的混合蚁群优化算法[J]. 计算机应用, 2021, 41(8): 2412-2417.
[4]	贾鹤鸣, 姜子超, 李瑶, 孙康健. 基于改进斑点鬣狗优化算法的同步优化特征选择[J]. 计算机应用, 2021, 41(5): 1290-1298.
[5]	林筠超, 万源. 基于图结构优化的自适应多度量非监督特征选择方法[J]. 计算机应用, 2021, 41(5): 1282-1289.
[6]	冉家敏, 倪志伟, 彭鹏, 朱旭辉. 考虑空间众包工作者服务质量的任务分配策略及其萤火虫群优化算法求解[J]. 计算机应用, 2021, 41(3): 794-802.
[7]	张志浩, 林耀进, 卢舜, 郭晨, 王晨曦. 缺失标记下基于类属属性的多标记特征选择[J]. 计算机应用, 2021, 41(10): 2849-2857.
[8]	顾桐, 许国良, 李万林, 李家浩, 王志愿, 雒江涛. 基于集成LightGBM和贝叶斯优化策略的房价智能评估模型[J]. 计算机应用, 2020, 40(9): 2762-2767.
[9]	黄学雨, 徐浩特, 陶剑文. 具有特征选择的多源自适应分类框架[J]. 计算机应用, 2020, 40(9): 2499-2506.
[10]	刘丹, 姚立霜, 王云锋, 裴作飞. 面向类不平衡流量数据的分类模型[J]. 计算机应用, 2020, 40(8): 2327-2333.
[11]	肖跃雷, 张云娇. 基于特征选择和超参数优化的恐怖袭击组织预测方法[J]. 计算机应用, 2020, 40(8): 2262-2267.
[12]	汪志远, 降爱莲, 奥斯曼·穆罕默德. 基于正则互表示的无监督特征选择方法[J]. 计算机应用, 2020, 40(7): 1896-1900.
[13]	曹堉, 王成, 王鑫, 高悦尔. 基于时空节点选择和深度学习的城市道路短时交通流预测[J]. 计算机应用, 2020, 40(5): 1488-1493.
[14]	曾元鹏, 王开军, 林崧. 面向二类区分能力的干扰熵特征选择方法[J]. 计算机应用, 2020, 40(3): 626-630.
[15]	章夏杰, 朱敬华, 陈杨. Spark下的分布式粗糙集属性约简算法[J]. 计算机应用, 2020, 40(2): 518-523.

基于新的森林优化算法的特征选择算法

Feature selection algorithm based on new forest optimization algorithm

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics