Doubly feature-weighted fuzzy support vector machine

doi:10.11772/j.issn.1001-9081.2021040760

Abstract

Abstract:

Concerning the shortcoming that the current feature-weighted Fuzzy Support Vector Machines （FSVM） only consider the influence of feature weights on the membership functions but ignore the application of feature weights to the kernel functions calculation during sample training， a new FSVM algorithm that considers the influence of feature weights on the membership function and the kernel function calculation simultaneously was proposed， namely Doubly Feature-Weighted FSVM （DFW-FSVM）. Firstly， relative weight of each feature was calculated by using Information Gain （IG）. Secondly， the weighted Euclidean distance between the sample and the class center was calculated in the original space based on the feature weights， and then the membership function was constructed by applying the weighted Euclidean distance； at the same time， the feature weights were applied to the calculation of the kernel function in the sample training process. Finally， DFW-FSVM algorithm was constructed according to the weighted membership functions and kernel functions. In this way， DFW-FSVM is able to avoid being dominated by trivial relevant or irrelevant features. The comparative experiments were carried out on eight UCI datasets， and the results show that compared with the best results of SVM， FSVM， Feature-Weighted SVM （FWSVM）， Feature-Weighted FSVM （FWFSVM） and FSVM based on Centered Kernel Alignment （CKA-FSVM）， the accuracy and F1 value of the DFW-FSVM algorithm increase by 2.33 and 5.07 percentage points， respectively， indicating that the proposed DFW-FSVM has good classification performance.

Key words: Fuzzy Support Vector Machine (FSVM), feature-weighted, Information Gain (IG), kernel function, membership function

摘要：

针对当前基于特征加权的模糊支持向量机（FSVM）只考虑特征权重对隶属度函数的影响，而没有考虑在样本训练过程中将特征权重应用到核函数计算中的缺陷，提出了同时考虑特征加权对隶属度函数和核函数计算的影响的模糊支持向量机算法——双重特征加权模糊支持向量机（DFW-FSVM）。首先，利用信息增益（IG）计算出每个特征的权重；然后，在原始空间中基于特征权重计算出样本到类中心的加权欧氏距离，进而应用该加权欧氏距离构造隶属度函数，并在样本训练过程中将特征权重应用到核函数的计算中；最后，根据加权的隶属度函数和核函数构造出DFW-FSVM算法。该方法避免了在计算过程中被弱相关或不相关的特征所支配。在8个UCI数据集上进行对比实验，结果显示DFW-FSVM算法的准确率和F1值较5个对比算法（SVM、FSVM、特征加权SVM（FWSVM）、特征加权FSVM（FWFSVM）、基于中心核对齐的FSVM（CKA-FSVM））中的最好结果分别提升了2.33和5.07个百分点，具有较好的分类性能。

关键词: 模糊支持向量机, 特征加权, 信息增益, 核函数, 隶属度函数

CLC Number:

TP18

Yunzhi QIU, Tinghua WANG, Xiaolu DAI. Doubly feature-weighted fuzzy support vector machine[J]. Journal of Computer Applications, 2022, 42(3): 683-687.

邱云志, 汪廷华, 戴小路. 双重特征加权模糊支持向量机[J]. 《计算机应用》唯一官方网站, 2022, 42(3): 683-687.

Figures/Tables 2

References 15

1	VAPNIK V N. The Nature of Statistical Learning Theory［M］. New York： Springer， 1995： 123-245. 10.1007/978-1-4757-2440-0_7
2	SHAWE-TAYLOR J， CRISTIANINI N. Kernel Methods for Pattern Analysis［M］. New York： Cambridge University Press， 2004： 116-231. 10.1017/cbo9780511809682
3	WANG M， WAN Y， YE Z， et al. Remote sensing image classification based on the optimal support vector machine and modified binary coded ant colony optimization algorithm［J］. Information Sciences， 2017，402： 50-68. 10.1016/j.ins.2017.03.027
4	WANG T， LI W， LIU F， et al. Sprinkled semantic diffusion kernel for word sense disambiguation［J］. Engineering Applications of Artificial Intelligence， 2017，64： 43-51. 10.1016/j.engappai.2017.05.010
5	KUMAR M， RATH S K. Classification of microarray using MapReduce based proximal support vector machine classifier［J］. Knowledge-Based Systems， 2015，89： 584-602. 10.1016/j.knosys.2015.09.005
6	汪廷华，田盛丰，黄厚宽. 特征加权支持向量机［J］. 电子与信息学报， 2009， 31（3）： 514-518.
	WANG T H， TIAN S F， HUANG H K. Feature weighted support vector machine［J］. Journal of Electronics and Information Technology， 2009， 31（3）： 514-518.
7	LIN C-F， WANG S-D. Fuzzy support vector machines［J］. IEEE Transactions on Neural Networks， 2002， 13（2）： 464-471. 10.1109/72.991432
8	JIANG X， ZHANG Y， LUY J C. Fuzzy SVM with a new fuzzy membership function［J］. Neural Computing and Applications， 2006， 15： 268-276. 10.1007/s00521-006-0028-z
9	AN W， LIANG M. Fuzzy support vector machine based on within-class scatter for classification problems with outliers or noises［J］. Neurocomputing， 2013，110： 101-110. 10.1016/j.neucom.2012.11.023
10	XIONG S-W， LIU H-B， NIU X-X. Fuzzy support vector machines based on FCM clustering［C］// Proceedings of the 2005 International Conference on Machine Learning and Cybernetics. Piscataway： IEEE， 2005： 2608-2613. 10.1109/icmlc.2005.1527384
11	WANG Y， WANG S， LAI K. A new fuzzy support vector machine to evaluate credit risk［J］. IEEE Transactions on Fuzzy Systems， 2005， 13（6）： 820-831. 10.1109/tfuzz.2005.859320
12	WANG T， QIU Y， HUA J. Centered kernel alignment inspired fuzzy support vector machine［J］. Fuzzy Sets and Systems， 2020，394： 110-123. 10.1016/j.fss.2019.09.017
13	左喻灏，贾连印，游进国，等. 基于Relief-F特征加权的模糊支持向量机的分类算法［J］.化工自动化及仪表， 2019， 46（10）： 834-838. 10.3969/j.issn.1000-3932.2019.10.011
	ZUO Y H， JIA L Y， YOU J G， et al. Classification algorithm of fuzzy support vector machine based on Relief-F feature weighting［J］. Control and Instruments in Chemical Industry， 2019， 46（10）： 834-838. 10.3969/j.issn.1000-3932.2019.10.011
14	CHANG C-C， LIN C-J. LIBSVM： a library for support vector machines［J］. ACM Transactions on Intelligent Systems and Technology， 2011，2（3）： Article No. 27. 10.1145/1961189.1961199
15	BACHE K， LICHMAN M. UCI machine learning repository ［DB/OL］. ［2020-06-20］. .

数据集名称（简称）	正样本数	负样本数	特征数
Ionosphere	126	225	34
Hepatitis	32	123	19
Breast Cancer （BC）	201	85	9
Climate Model Simulation Crashes （CMSC）	46	494	18
Australian Credit Approval （ACA）	307	383	15
Stalog Heart （ST）	150	120	13
Vertebral Column （VC）	210	100	6
Breast Cancer Wisconsin （Original）（WBC）	444	239	9

数据集名称（简称）	正样本数	负样本数	特征数
Ionosphere	126	225	34
Hepatitis	32	123	19
Breast Cancer （BC）	201	85	9
Climate Model Simulation Crashes （CMSC）	46	494	18
Australian Credit Approval （ACA）	307	383	15
Stalog Heart （ST）	150	120	13
Vertebral Column （VC）	210	100	6
Breast Cancer Wisconsin （Original）（WBC）	444	239	9

算法	Ionosphere		Hepatitis		BC		CMSC		ACA		ST		VC		WBC
算法	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值
SVM	91.51	86.96	80.85	40.00	68.60	78.74	92.59	45.45	76.81	75.00	79.01	81.32	82.80	88.06	95.54	96.60
FSVM	93.40	90.14	80.85	57.14	74.42	83.33	93.21	42.11	71.50	67.04	74.07	76.92	86.02	90.65	95.05	96.24
FWSVM	95.28	92.11	76.60	47.62	72.09	81.54	93.83	44.44	82.13	82.30	80.25	82.98	87.10	90.91	96.04	96.97
FWFSVM	94.34	92.11	82.98	55.56	75.58	84.21	94.44	52.63	75.85	74.23	72.84	76.09	87.10	91.30	95.54	96.60
CKA-FSVM	92.45	88.89	85.11	36.36	69.77	82.19	94.44	66.67	68.12	69.44	76.54	78.65	86.02	90.08	95.05	96.24
DFW-FSVM	95.28	93.15	82.98	60.00	76.74	85.07	95.68	66.67	84.06	84.36	81.48	84.21	89.25	92.42	96.53	97.36

算法	Ionosphere		Hepatitis		BC		CMSC		ACA		ST		VC		WBC
算法	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值	准确率	F1值
SVM	91.51	86.96	80.85	40.00	68.60	78.74	92.59	45.45	76.81	75.00	79.01	81.32	82.80	88.06	95.54	96.60
FSVM	93.40	90.14	80.85	57.14	74.42	83.33	93.21	42.11	71.50	67.04	74.07	76.92	86.02	90.65	95.05	96.24
FWSVM	95.28	92.11	76.60	47.62	72.09	81.54	93.83	44.44	82.13	82.30	80.25	82.98	87.10	90.91	96.04	96.97
FWFSVM	94.34	92.11	82.98	55.56	75.58	84.21	94.44	52.63	75.85	74.23	72.84	76.09	87.10	91.30	95.54	96.60
CKA-FSVM	92.45	88.89	85.11	36.36	69.77	82.19	94.44	66.67	68.12	69.44	76.54	78.65	86.02	90.08	95.05	96.24
DFW-FSVM	95.28	93.15	82.98	60.00	76.74	85.07	95.68	66.67	84.06	84.36	81.48	84.21	89.25	92.42	96.53	97.36

[1]	Xiangzhou QI, Hongjie XING. Centered kernel alignment based multiple kernel one-class support vector machine [J]. Journal of Computer Applications, 2022, 42(2): 349-356.
[2]	XIAO Qi, YIN Zengshan, GAO Shuang. Extremely dim target search algorithm based on detection and tracking mutual iteration [J]. Journal of Computer Applications, 2021, 41(10): 3017-3024.
[3]	SUN Shilei, WANG Chao, ZHAO Yuandi. Parameter independent clustering of air traffic trajectory based on silhouette coefficient [J]. Journal of Computer Applications, 2019, 39(11): 3293-3297.
[4]	FAN Jun, WANG Xin, XU Hui. Prediction method of tectonic coal thickness based on particle swarm optimized hybrid kernel extreme learning machine [J]. Journal of Computer Applications, 2018, 38(6): 1820-1825.
[5]	HU Lisha, WANG Suzhen, CHEN Yiqiang, HU Chunyu, JIANG Xinlong, CHEN Zhenyu, GAO Xingyu. Objective equilibrium measurement based kernelized incremental learning method for fall detection [J]. Journal of Computer Applications, 2018, 38(4): 928-934.
[6]	ZHANG Leyuan, LI Jiaye, LI Pengqing. Low rank non-linear feature selection algorithm [J]. Journal of Computer Applications, 2018, 38(12): 3444-3449.
[7]	WANG Jiaqing, ZHU Yan, CHEN Tung-shou, CHANG Chin-chen. Optimum feature selection based on genetic algorithm under Web spam detection [J]. Journal of Computer Applications, 2018, 38(1): 295-299.
[8]	NAN Jingchang, CUI Hongyan. Modeling of power amplifier based on dynamic X-parameter of new two-dimensional kernel function [J]. Journal of Computer Applications, 2017, 37(8): 2421-2426.
[9]	HUO Weigang, QU Feng, CHENG Zhen. Incremental fuzzy associative classification method based on evolving vector quantization clustering algorithm [J]. Journal of Computer Applications, 2017, 37(11): 3075-3079.
[10]	LI Hua, LI Deyu, WANG Suge, ZHANG Jing. Kernel improvement of multi-label feature extraction method [J]. Journal of Computer Applications, 2015, 35(7): 1939-1944.
[11]	WANG Weidong, LIU Bing, GUAN Hongjie, ZHOU Yong, XIA Shixiong. Spectral embedded clustering algorithm based on kernel function [J]. Journal of Computer Applications, 2015, 35(3): 761-765.
[12]	ZHANG Cheng, LIU Yadong, LI Yuan. Nonlinear feature extraction based on discriminant diffusion map analysis [J]. Journal of Computer Applications, 2015, 35(2): 470-475.
[13]	HU Zhaohua, XING Weiguo, HE Jun, ZHANG Xiuzai. Real-time object tracking method based on multi-channel kernel correlation filter [J]. Journal of Computer Applications, 2015, 35(12): 3544-3549.
[14]	ZHANG Yingchao GUO Dong XIONG Xiong HE Lei. Compensation method for abnormal temperature data of automatic weather station [J]. Journal of Computer Applications, 2014, 34(3): 888-891.
[15]	XU Kaiyong FANG Ming YANG Tianchi MENG Fanwei HUANG Huixin. Value-at-risk quantitative method about password chip under differential power analysis attacks [J]. Journal of Computer Applications, 2013, 33(06): 1642-1645.