关键词: 欺诈网页检测, 特征选择, 遗传算法, 信息增益, 最佳最小特征集合

Abstract: Web spam has been destructive to Search Engines and Internet Security. Researches on Web spam detection are popular and essential, which focus on integrating new features and improving classification algorithms. However, basic features of web page typically used in spam detection are high-dimensional and redundant, which cause classifier ‘overhead’ and influence the detection efficiency, feature dimensionality reduction is necessary, proposed an improved feature selection algorithm based on Kullback-Leibler Divergence and Genetic Algorithm (IFSBKGA). Considering that the algorithm has a certain randomness, increased the number of experimental iterations to generate an Optimal Minimum Feature Set (OMFS). Comparative experiments are conducted based on Native Bayes and Random Forest classifiers, and the detection results can still reach or be better than that use hundreds of features, as well as the detection time is dramatically reduced. The experiments verify that IFSBKGA can reduce feature dimension, decrease the classification computational cost, while ensuring detection robustness.

Key words: Web spam detection, feature selection, Genetic Algorithm (GA), Kullback-Leibler Divergence (KLD), the optimal minimum feature set