面向二类区分能力的干扰熵特征选择方法

doi:10.11772/j.issn.1001-9081.2019071200

计算机应用 ›› 2020, Vol. 40 ›› Issue (3): 626-630.DOI: 10.11772/j.issn.1001-9081.2019071200

面向二类区分能力的干扰熵特征选择方法

曾元鹏^1,2, 王开军¹, 林崧²

1. 福建师范大学数学与信息学院, 福州 350007;
2. 福建师范大学数字福建环境监测物联网实验室, 福州 350007

收稿日期:2019-07-10 修回日期:2019-09-23 出版日期:2020-03-10 发布日期:2019-10-25
通讯作者: 王开军
作者简介:曾元鹏(1995-),男,福建漳州人,硕士研究生,主要研究方向:机器学习、数据挖掘;王开军(1965-),男,福建福州人,副教授,博士,主要研究方向:机器学习、数据挖掘;林崧(1979-),男,福建福州人,教授,博士,主要研究方向:量子密码、量子智能计算。
基金资助:
国家自然科学基金资助项目（61672157，61772134）；福建省自然科学基金资助项目（2018J01778）；中国博士后科学基金资助项目（2016M600494）。

Interference entropy feature selection method for two-class distinguishing ability

ZENG Yuanpeng^1,2, WANG Kaijun¹, LIN Song²

1. College of Mathematics and Informatics, Fujian Normal University, Fuzhou Fujian 350007, China;
2. Digit Fujian Internet-of-Things Laboratory of Environmental Monitoring, Fujian Normal University, Fuzhou Fujian 350007, China

Received:2019-07-10 Revised:2019-09-23 Online:2020-03-10 Published:2019-10-25
Supported by:
This work is partially supported by the National Natural Science Foundation of China(61672157, 61772134), the Natural Science Foundation of Fujian Province (2018J01778), the China Postdoctoral Science Foundation (2016M600494).

摘要/Abstract

摘要： 针对现有的特征选择方法对衡量不同类别数据重叠/分离能力的不足，提出了一种用于评价特征的二类区分能力的干扰熵方法（IET-CD）。对于包含两个类别（正类和负类）样本的特征，首先，计算正类数据范围内的负类样本的混合条件概率，以及负类样本归属于正类的概率；然后，由混合条件概率和归属概率计算混淆概率，再利用混淆概率计算正类干扰熵，同理，计算负类干扰熵；最后，将正、负类干扰熵之和作为该特征的二类干扰熵。干扰熵用于评价特征对二类样本的区分能力，该特征的干扰熵值小，表明该特征的二类区分能力强，反之则弱。在3个UCI数据集和1个模拟基因表达数据集上，每个方法挑选出5个最优特征，并对比了这些特征的二类区分能力，由此比较这些方法的性能。实验结果表明：所提方法与NEFS方法相比，二类区分能力相当或更好；与单索引近邻熵特征选择（SNEFS）方法、相关性最大冗余性最小特征选择（MRMR）算法、联合互信息（JMI）方法、Relief方法相比，绝大多数情况都是所提方法获胜。IET-CD方法能有效地选择二类区分能力更好的特征。

关键词: 特征选择, 二类区分能力, 条件概率, 干扰熵

Abstract: Aiming at the existing feature selection methods lacking the ability to measure the overlap/separation of different classes of data, an Interference Entropy of Two-Class Distinguishing (IET-CD) method was proposed to evaluate the two-class distinguishing ability of features. For the feature containing two classes (positive and negative), firstly, the mixed conditional probability of the negative class samples within the range of positive class data and the probability of the negative class samples belonging to the positive class were calculated; then, the confusion probability was calculated by the mixed conditional probability and attribution probability, and the confusion probability was used to calculate the positive interference entropy. In the similar way, the negative interference entropy was calculated. Finally, the sum of positive and negative interference entropies was taken as the two-class interference entropy of the feature. The interference entropy was used to evaluate the distinguishing ability of the feature to the two-class sample. The smaller the interference entropy value of the feature, the stronger the two-class distinguishing ability of the feature. On three UCI datasets and one simulated gene expression dataset, five optimal features were selected for each method, and the two-class distinguishing ability of the features were compared, so as to compare the performance of the methods. The experimental results show that the proposed method is equivalent or better than the NEFS （Neighborhood Entropy Feature Selection） method, and compared with the Single-indexed Neighborhood Entropy Feature Selection (SNEFS), feature selection based on Max-Relevance and Min-Redundancy (MRMR), Joint Mutual Information (JMI) and Relief method, the proposed method is better in most cases. The IET-CD method can effectively select features with better two-class distinguishing ability.

Key words: feature selection, two-class distinguishing ability, conditional probability, interference entropy

中图分类号:

TP391.4

曾元鹏, 王开军, 林崧. 面向二类区分能力的干扰熵特征选择方法[J]. 计算机应用, 2020, 40(3): 626-630.

ZENG Yuanpeng, WANG Kaijun, LIN Song. Interference entropy feature selection method for two-class distinguishing ability[J]. Journal of Computer Applications, 2020, 40(3): 626-630.

参考文献

[1] 翟俊海, 刘博, 张素芳. 基于粗糙集相对分类信息熵和粒子群优化的特征选择方法[J]. 智能系统学报,2017,12(3):397-404. (ZHAI J H,LIU B,ZHANG S F. A feature selection approach based on rough set relative classification information entropy and particle swarm optimization[J]. CAAI Transactions on Intelligent Systems,2017,12(3):397-404.)
[2] MARIELLO A,BATTITI R. Feature selection based on the neighborhood entropy[J]. IEEE Transactions on Neural Networks and Learning Systems,2018,29(12):6313-6322.
[3] 毛莺池, 曹海, 平萍, 等. 基于最大联合条件互信息的特征选择[J]. 计算机应用,2019,39(3):734-741.(MAO Y C,CAO H, PING P,et al. Feature selection based on maximum conditional and joint mutual information[J]. Journal of Computer Applications, 2019,39(3):734-741.)
[4] PENG H,LONG F,DING C. Feature selection based on mutual information criteria of max-dependency,max-relevance,and min-redundancy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2005,27(8):1226-1238.
[5] 董红斌, 滕旭阳, 杨雪. 一种基于关联信息熵度量的特征选择方法[J]. 计算机研究与发展,2016,53(8):1684-1695. (DONG H B,TENG X Y,YANG X. Feature selection based on the measurement of correlation information entropy[J]. Journal of Computer Research and Development,2016,53(8):1684-1695.)
[6] 马忱, 姜高霞, 王文剑. 面向函数型数据的动态互信息特征选择方法[J]. 计算机科学与探索,2019,13(1):158-168. (MA C, JIANG G X,WANG W J. Dynamic mutual information feature selection for functional data[J]. Journal of Frontiers of Computer Science and Technology,2019,13(1):158-168.)
[7] YANG H H,MOODY J. Data visualization and feature selection:new algorithms for nonGaussian data[C]//Proceedings of the 12th International Conference on Neural Information Processing Systems. Cambridge:MIT Press,1999:687-693.
[8] ZHENG K,WANG X. Feature selection method with joint maximal information entropy between features and class[J]. Pattern Recognition,2018,77:20-29.
[9] KIRA K,RENDELL L A. A practical approach to feature selection[C]//Proceedings of the 9th International Workshop on Machine Learning. San Francisco:Morgan Kaufmann Publishers Inc., 1992:249-256.
[10] 翟俊海, 刘博, 张素芳. 基于相对分类信息熵的进化特征选择算法[J]. 模式识别与人工智能, 2016,29(8):682-690. (ZHAI J H,LIU B,ZHANG S F. Feature selection via evolutionary computation based on relative classification information entropy[J]. Pattern Recognition and Artificial Intelligence,2016,29(8):682-690.)
[11] 张振海, 李士宁, 李志刚, 等. 一类基于信息熵的多标签特征选择算法[J]. 计算机研究与发展,2013,50(6):1177-1184. (ZHANG Z H,LI S N,LI Z G,et al. Multi-label feature selection algorithm based on information entropy[J]. Journal of Computer Research and Development,2013,50(6):1177-1184.)
[12] 周红标, 乔俊飞. 基于高维k-近邻互信息的特征选择方法[J]. 智能系统学报,2017,12(5):595-600. (ZHOU H B,QIAO J F. Feature selection method based on high dimensional k-nearest neighbors mutual information[J]. CAAI Transactions on Intelligent Systems,2017,12(5):595-600.)
[13] 董泽民, 石强. 基于归一化模糊联合互信息最大的特征选择[J]. 计算机工程与应用,2017,53(22):105-110.(DONG Z M, SHI Q. Feature selection using normalized fuzzy joint mutual information maximum[J]. Computer Engineering and Applications, 2017,53(22):105-110.)
[14] BLAKE C L,MERZ C J. UCI repository of machine learning databases[EB/OL].[2019-05-13]. http://mlearn.ics.uci.edu/MLRepository.html.
[15] WANG K,ZHENG J,ZHANG J,et al. Estimating the number of clusters via system evolution for cluster analysis of gene expression data[J]. IEEE Transactions on Information Technology in Biomedicine,2009,13(5):848-853.

面向二类区分能力的干扰熵特征选择方法

Interference entropy feature selection method for two-class distinguishing ability

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	湛航, 何朗, 黄樟灿, 李华峰, 张蔷, 谈庆. 改进的基于层次距离的基因表达式编程特征选择分类算法[J]. 计算机应用, 2021, 41(9): 2658-2667.
[2]	祝承, 赵晓琦, 赵丽萍, 焦玉宏, 朱亚飞, 陈建英, 周伟, 谭颖. 基于谱聚类半监督特征选择的功能磁共振成像数据分类[J]. 计算机应用, 2021, 41(8): 2288-2293.
[3]	李蒙蒙, 秦伟, 刘艺, 刁兴春. 结合头脑风暴优化的混合蚁群优化算法[J]. 计算机应用, 2021, 41(8): 2412-2417.
[4]	贾鹤鸣, 姜子超, 李瑶, 孙康健. 基于改进斑点鬣狗优化算法的同步优化特征选择[J]. 计算机应用, 2021, 41(5): 1290-1298.
[5]	林筠超, 万源. 基于图结构优化的自适应多度量非监督特征选择方法[J]. 计算机应用, 2021, 41(5): 1282-1289.
[6]	张志浩, 林耀进, 卢舜, 郭晨, 王晨曦. 缺失标记下基于类属属性的多标记特征选择[J]. 计算机应用, 2021, 41(10): 2849-2857.
[7]	黄学雨, 徐浩特, 陶剑文. 具有特征选择的多源自适应分类框架[J]. 计算机应用, 2020, 40(9): 2499-2506.
[8]	顾桐, 许国良, 李万林, 李家浩, 王志愿, 雒江涛. 基于集成LightGBM和贝叶斯优化策略的房价智能评估模型[J]. 计算机应用, 2020, 40(9): 2762-2767.
[9]	刘丹, 姚立霜, 王云锋, 裴作飞. 面向类不平衡流量数据的分类模型[J]. 计算机应用, 2020, 40(8): 2327-2333.
[10]	肖跃雷, 张云娇. 基于特征选择和超参数优化的恐怖袭击组织预测方法[J]. 计算机应用, 2020, 40(8): 2262-2267.
[11]	汪志远, 降爱莲, 奥斯曼·穆罕默德. 基于正则互表示的无监督特征选择方法[J]. 计算机应用, 2020, 40(7): 1896-1900.
[12]	曹堉, 王成, 王鑫, 高悦尔. 基于时空节点选择和深度学习的城市道路短时交通流预测[J]. 计算机应用, 2020, 40(5): 1488-1493.
[13]	谢琪, 徐旭, 程耕国, 陈和平. 基于新的森林优化算法的特征选择算法[J]. 计算机应用, 2020, 40(5): 1266-1271.
[14]	章夏杰, 朱敬华, 陈杨. Spark下的分布式粗糙集属性约简算法[J]. 计算机应用, 2020, 40(2): 518-523.
[15]	程玉胜, 宋帆, 王一宾, 钱坤. 基于专家特征的条件互信息多标记特征选择算法[J]. 计算机应用, 2020, 40(2): 503-509.