Ordinal decision tree algorithm based on fuzzy advantage complementary mutual information

doi:10.11772/j.issn.1001-9081.2020122006

Journal of Computer Applications ›› 2021, Vol. 41 ›› Issue (10): 2785-2792.DOI: 10.11772/j.issn.1001-9081.2020122006

Special Issue: 人工智能

• Artificial intelligence • Next Articles

Ordinal decision tree algorithm based on fuzzy advantage complementary mutual information

WANG Yahui^1,2, QIAN Yuhua^1,2,3, LIU Guoqing^1,2

1. Institute of Big Data Science and Industry, Shanxi University, Taiyuan Shanxi 030006, China;
2. School of Computer and Information Technology, Shanxi University, Taiyuan Shanxi 030006, China;
3. Key Laboratory of Computational Intelligence and Chinese Information Processing of Ministry of Education(Shanxi University), Taiyuan Shanxi 030006, China

Received:2020-12-21 Revised:2021-04-29 Online:2021-10-10 Published:2021-07-14
Supported by:
This work is partially supported by the Surface Program of National Natural Science Foundation of China (61672332), the Top Innovative Talents Support Program of Shanxi Province (02150116072021), the Key Research and Development Program of Shanxi Province (International Science and Technology Cooperation) (201903D421003), the San Jin Scholars Project of Shanxi Province (2016769), the Funded Scientific Research Project for Returned Overseas Students of Shanxi Province (2017-023).

基于模糊优势互补互信息的有序决策树算法

王雅辉^1,2, 钱宇华^1,2,3, 刘郭庆^1,2

1. 山西大学大数据科学与产业研究院, 太原 030006;
2. 山西大学计算机与信息技术学院, 太原 030006;
3. 计算智能与中文信息处理教育部重点实验室(山西大学), 太原 030006

通讯作者: 钱宇华
作者简介:王雅辉(1995-),女,山西临汾人,硕士研究生,CCF会员,主要研究方向:机器学习;钱宇华(1976-),男,山西晋城人,教授,博士,CCF会员,主要研究方向:模式识别、特征选择、粗糙集理论、粒计算、人工智能;刘郭庆(1994-),女,山西临汾人,硕士研究生,主要研究方向:强化学习。
基金资助:
国家自然科学基金面上项目（61672332）；山西省拔尖创新人才支持计划项目（02150116072021）；山西省重点研发计划（国际科技合作）项目（201903D421003）；山西省三晋学者项目（2016769）；山西省回国留学人员科研资助项目（2017-023）。

Abstract

Abstract: When the traditional decision tree algorithm is applied to the ordinal classification task, there are two problems:the traditional decision tree algorithm does not introduce the order relation, so it cannot learn and extract the order structure of the dataset; in real life, there is a lot of fuzzy but not exact knowledge, however the traditional decision tree algorithm cannot deal with the data with fuzzy attribute value. To solve these problems, an ordinal decision tree algorithm based on fuzzy advantage complementary mutual information was proposed. Firstly, the dominant set was used to represent the order relations in the data, and the fuzzy set was introduced to calculate the dominant set for forming a fuzzy dominant set. The fuzzy dominant set was able to not only reflect the order information in the data, but also obtain the inaccurate knowledge automatically. Then, the complementary mutual information was generalized on the basis of fuzzy dominant set, and the fuzzy advantage complementary mutual information was proposed. Finally, the fuzzy advantage complementary mutual information was used as a heuristic method, and an decision tree algorithm based on fuzzy advantage complementary mutual information was designed. Experimental results on 5 synthetic datasets and 9 real datasets show that, the proposed algorithm has less classification errors compared with the classical decision tree algorithm on the ordinal classification tasks.

Key words: machine learning, decision tree algorithm, ordinal classification, fuzzy mathematics, dominant set

摘要： 传统决策树算法应用于有序分类任务时存在两个问题：传统决策树算法没有引入序关系，因此无法学习和抽取数据集中的序结构；现实生活中存在大量模糊而非精确的知识，而传统的决策树算法无法处理存在模糊属性取值的数据。针对上述问题，提出了基于模糊优势互补互信息的有序决策树算法。首先，使用优势集表示数据中的序关系，并引入模糊集来计算优势集以形成模糊优势集。模糊优势集不仅能反映数据中的序信息，而且能自动获取不精确知识。然后，在模糊优势集的基础上将互补互信息进行推广，并提出了模糊优势互补互信息。最后，使用模糊优势互补互信息作为启发式，设计出基于模糊优势互补互信息的有序决策树算法。在5个人工数据集及9个现实数据集上的实验结果表明，所提算法在有序分类任务上较经典决策树算法取得了更低的分类误差。

关键词: 机器学习, 决策树算法, 有序分类, 模糊数学, 优势集

CLC Number:

TP181

WANG Yahui, QIAN Yuhua, LIU Guoqing. Ordinal decision tree algorithm based on fuzzy advantage complementary mutual information[J]. Journal of Computer Applications, 2021, 41(10): 2785-2792.

王雅辉, 钱宇华, 刘郭庆. 基于模糊优势互补互信息的有序决策树算法[J]. 计算机应用, 2021, 41(10): 2785-2792.

References

[1] CANO J R, GUTIÉRREZ P A, KRAWCZYK B, et al. Monotonic classification:an overview on algorithms, performance measures and data sets[J]. Neurocomputing, 2019, 341:168-182.
[2] WANG X Z, ASHFAQ R A R, FU A M. Fuzziness based sample categorization for classifier performance improvement[J]. Journal of Intelligent and Fuzzy Systems, 2015, 29(3):1185-1196.
[3] DOYLE O M, WESTMAN E, MARQUAND A F, et al. Predicting progression of Alzheimer's disease using ordinal regression[J]. PLoS ONE, 2014, 9(8):No. e105542.
[4] PAN W W. Fraudulent firm classification using monotonic classification techniques[C]//Proceedings of the IEEE 9th Joint International Information Technology and Artificial Intelligence Conference. Piscataway:IEEE, 2020:1773-1776.
[5] LIANG J Y, CHIN K S, DANG C Y, et al. A new method for measuring uncertainty and fuzziness in rough set theory[J]. International Journal of General Systems, 2002, 31(4):331-342.
[6] DEMBCZYŃSKI K, KOTŁOWSKI W, SŁOWIŃSKI R. Ensemble of decision rules for ordinal classification with monotonicity constraints[C]//Proceedings of the 2008 International Conference on Rough Sets and Knowledge Technology, LNCS 5009. Berlin:Springer, 2008:260-267.
[7] TEHRANI A F, HÜLLERMEIER E. Ordinal choquistic regression[C]//Proceedings of the 8th Conference of the European Society for Fuzzy Logic and Technology. Dordrecht:Atlantis Press, 2013:842-849.
[8] BARTLEY C, LIU W, REYNOLDS M. Effective monotone knowledge integration in kernel support vector machines[C]//Proceedings of the 2016 International Conference on Advanced Data Mining and Applications, LNCS 10086. Cham:Springer, 2016:3-18.
[9] DANIELS H, VELIKOVA M. Monotone and partially monotone neural networks[J]. IEEE Transactions on Neural Networks, 2010, 21(6):906-917.
[10] XU H, WANG W J, QIAN Y H. Fusing complete monotonic decision trees[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 29(10):2223-2235.
[11] ZHU H, TSANG E C C, WANG X Z, et al. Monotonic classification extreme learning machine[J]. Neurocomputing, 2017, 225:205-213.
[12] CARDOSO J S, PINTO DA COSTA J F. Learning to classify ordinal data:the data replication method[J]. Journal of Machine Learning Research, 2007, 8:1393-1429.
[13] TANG M Z, PÉREZ-FERNÁNDEZ R, DE BAETS B. Fusing absolute and relative information for augmenting the method of nearest neighbors for ordinal classification[J]. Information Fusion, 2020, 56:128-140.
[14] GONZÁLEZ S, HERRERA F, GARCÍA S. Managing monotonicity in classification by a pruned AdaBoost[C]//Proceedings of the 2016 International Conference on Hybrid Artificial Intelligence Systems, LNCS 9648. Cham:Springer, 2016:512-523.
[15] PINTO DA COSTA J, CARDOSO J S. Classification of ordinal data using neural networks[C]//Proceedings of the 2005 European Conference on Machine Learning, LNCS 3720. Berlin:Springer, 2005:690-697.
[16] QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986, 1(1):81-106.
[17] QUINLAN J R. C4.5:Programs for Machine Learning[M]. San Mateo, CA:Morgan Kaufmann Publishers, 1992:8-10.
[18] FEELDERS A, PARDOEL M. Pruning for monotone classification trees[C]//Proceedings of the 2003 International Symposium on Intelligent Data Analysis, LNCS 2810. Berlin:Springer, 2003:1-12.
[19] VERBEKE W, MARTENS D, BAESENS B. RULEM:a novel heuristic rule learning approach for ordinal classification with monotonicity constraints[J]. Applied Soft Computing, 2017, 60:858-873.
[20] XIA F, ZHANG W S, LI F X, et al. Ranking with decision tree[J]. Knowledge and Information Systems, 2008, 17(3):381-395.
[21] HU Q H, CHE X J, ZHANG L, et al. Rank entropy-based decision trees for monotonic classification[J]. IEEE Transactions on Knowledge and Data Engineering, 2012, 24(11):2052-2064.
[22] 车勋建. 基于有序决策树的故障程度诊断研究[D]. 哈尔滨:哈尔滨工业大学, 2011:53-57.(CHE X J. Ordinal decision tree based fault level detection[D]. Harbin:Harbin Institute of Technology, 2011:53-57.)
[23] 许行, 王文剑, 任丽芳. 一种基于决策森林的单调分类方法[J]. 计算机研究与发展, 2017, 54(7):1477-1487.(XU H, WANG W J, REN L F. A method for monotonic classification based on decision forest[J]. Journal of Computer Research and Development, 2017, 54(7):1477-1487.)
[24] XIA S Y, WANG G Y, CHEN Z Z, et al. Complete random forest based class noise filtering learning for improving the generalizability of classifiers[J]. IEEE Transactions on Knowledge and Data Engineering, 2019, 31(11):2063-2078.
[25] FRÉNAY B, HAMMER B. Label-noise-tolerant classification for streaming data[C]//Proceedings of the 2017 International Joint Conference on Neural Networks. Piscataway:IEEE, 2017:1748-1755.
[26] XIA S Y, CHEN B Y, WANG G Y, et al. mCRF and mRD:two classification methods based on a novel multiclass label noise filtering learning framework[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, PP (99):1-15.

Ordinal decision tree algorithm based on fuzzy advantage complementary mutual information

基于模糊优势互补互信息的有序决策树算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics

[1]	GUO Mian, ZHANG Jinyou. Computation offloading policy for machine learning in mobile edge computing environments [J]. Journal of Computer Applications, 2021, 41(9): 2639-2645.
[2]	MAO Mingze, CAO Ruihao, YAN Chungang. Semi-supervised classification algorithm based on weight diversity [J]. Journal of Computer Applications, 2021, 41(9): 2473-2480.
[3]	QIN Binbin, PENG Liangkang, LU Xiangming, QIAN Jiangbo. Research progress on driver distracted driving detection [J]. Journal of Computer Applications, 2021, 41(8): 2330-2337.
[4]	JIANG Qianyu, WANG Fengying, JIA Lipeng. Malware detection method based on perceptual hash algorithm and feature fusion [J]. Journal of Computer Applications, 2021, 41(3): 780-785.
[5]	QIN Jing, ZUO Changqing, WANG Zumin, JI Changqing, WANG Baofeng. Design of abnormal electrocardiograph monitoring model based on stacking classifier [J]. Journal of Computer Applications, 2021, 41(3): 887-890.
[6]	MENG Xiangrui, YANG Wenzhong, WANG Ting. Survey of sentiment analysis based on image and text fusion [J]. Journal of Computer Applications, 2021, 41(2): 307-317.
[7]	JIANG Yangsheng, WANG Shengnan, TU Jiaqi, LI Sha, WANG Hongjun. Comprehensive prediction of thermal comfort and energy consumption for high-speed railway stations [J]. Journal of Computer Applications, 2021, 41(1): 249-257.
[8]	ZHU Lin, YU Haitao, LEI Xinyu, LIU Jing, WANG Ruofan. Brain network feature identification algorithm for Alzheimer's patients based on MRI image [J]. Journal of Computer Applications, 2020, 40(8): 2455-2459.
[9]	LIANG Denggao, ZHOU Anmin, ZHENG Rongfeng, LIU Liang, DING Jianwei. WeChat payment behavior recognition model based on division of large and small burst blocks [J]. Journal of Computer Applications, 2020, 40(7): 1970-1976.
[10]	XU Zhoubo, YANG Jian, LIU Huadong, HUANG Wenwen. Protein complex identification algorithm based on XGboost and topological structural information [J]. Journal of Computer Applications, 2020, 40(5): 1510-1514.
[11]	ZHANG Junsheng, XU Jingjing, YU Wei. No-reference image quality assessment method for facial beautification image [J]. Journal of Computer Applications, 2020, 40(4): 1184-1190.
[12]	WANG Yang, ZHAO Hongdong. Human activity recognition based on improved particle swarm optimization-support vector machine and context-awareness [J]. Journal of Computer Applications, 2020, 40(3): 665-671.
[13]	LYU Yali, MIAO Junzhong, HU Weixin. Semi-supervised learning algorithm of graph based on label metric learning [J]. Journal of Computer Applications, 2020, 40(12): 3430-3436.
[14]	LI Hui, YANG Zhixia. Multiple birth support vector machine based on Rescaled Hinge loss function [J]. Journal of Computer Applications, 2020, 40(11): 3139-3145.
[15]	LIU Ran, LIU Yu, GU Jinguang. Improved AdaNet based on adaptive learning rate optimization [J]. Journal of Computer Applications, 2020, 40(10): 2804-2810.