欺诈网页检测中基于遗传算法的特征优选

doi:10.11772/j.issn.1001-9081.2017061560

计算机应用 ›› 2018, Vol. 38 ›› Issue (1): 295-299.DOI: 10.11772/j.issn.1001-9081.2017061560

• 应用前沿、交叉与综合 • 上一篇下一篇

欺诈网页检测中基于遗传算法的特征优选

王嘉卿¹, 朱焱¹, 陈同孝², 张真诚³

1. 西南交通大学信息科学与技术学院, 成都 611756;
2. 台中科技大学资讯工程系, 台湾台中 404;
3. 逢甲大学资讯工程系, 台湾台中 407

收稿日期:2017-06-26 修回日期:2017-08-20 发布日期:2018-01-22 出版日期:2018-01-10
通讯作者: 朱焱
作者简介:王嘉卿(1993-),女,浙江金华人,硕士研究生,主要研究方向:Web数据挖掘;朱焱(1965-),女,广西桂林人,教授,博士,CCF会员,主要研究方向:数据挖掘、Web异常发现、大数据管理与智能分析;陈同孝(1964-),男,安徽霍邱人,教授,博士,主要研究方向:影像处理、资料勘探、资讯安全;张真诚(1954-),男,台湾台中人,教授,博士,主要研究方向:资料库设计、电子商务安全、电子多媒体影像技术、密码学。
基金资助:
四川省学术和技术带头人后备人选科研基金资助项目（WZ0100112371408，YH1500411031402）；四川省学术和技术带头人科研基金资助项目（WZ0100112371601/004）；四川省科技服务业示范项目（2016GFW0166）。

Optimum feature selection based on genetic algorithm under Web spam detection

WANG Jiaqing¹, ZHU Yan¹, CHEN Tung-shou², CHANG Chin-chen³

1. College of Information Science and Technology, Southwest Jiaotong University, Chengdu Sichuan 611756, China;
2. Department of Computer Science and Information Engineering, Taichung University of Science and Technology, Taichung Taiwan 404, China;
3. Department of Information Engineering and Computer Science, Feng Chia University, Taichung Taiwan 407, China

Received:2017-06-26 Revised:2017-08-20 Online:2018-01-22 Published:2018-01-10
Supported by:
This work is partially supported by the Academic and Technological Leadership Training Foundation of Sichuan Province (WZ0100112371408, YH1500411031402), the Academic and Technological Leadership Research Foundation of Sichuan Province (WZ0100112371601/004), the Demonstration Project in Technology Service Industry of Sichuan Province (2016GFW0166).

摘要/Abstract

摘要： 针对网页欺诈检测中特征的高维、冗余问题，提出一个基于信息增益和遗传算法的改进特征选择算法（IFS-BIGGA）。首先，通过信息增益（IG）给出特征重要性排序，设定动态阈值减少冗余特征；其次，改进遗传算法（GA）中染色体编码函数和选择算子，并结合随机森林（RF）的受试者工作特征曲线面积（AUC）作为适应度函数，选择高辨识度特征；最后，增加实验迭代次数避免算法随机性，产生最佳最小的特征集合（OMFS）。实验验证表明，应用IFS-BIGGA生成的OMFS与高维特征集合相比，尽管RF下的AUC减小了2%，但是真阳性率（TPR）提高了21%，并且特征维度减少了92%；同时多个常用分类器的平均检测时间减少了83%；另外，IFS-BIGGA的F1值相比传统的遗传算法（TGA）和帝国主义竞争算法（ICA）分别提高了4.2%和3.5%。实验结果表明，IFS-BIGGA可以进行高效特征降维，在实际的网页检测工程中，有效减少计算代价，提高检测效率。

关键词: 特征选择, 遗传算法, 信息增益, 随机森林算法, 欺诈网页检测

Abstract: Focusing on the issue that features used in Web spam detection are always high-dimensional and redundant, an Improved Feature Selection method Based on Information Gain and Genetic Algorithm (IFS-BIGGA) was proposed. Firstly, the priorities of features were ranked by Information Gain (IG), and dynamic threshold was set to get rid of redundant features. Secondly, the function of chromosome encoding was modified and the selection operator was improved in Genetic Algorithm (GA). After that, the Area Under receiver operating Characteristic (AUC) of Random Forest (RF) classifier was utilized as the fitness function to pick up the features with high degree of identification. Finally, the Optimal Minimum Feature Set (OMFS) was obtained by increasing the experimental iteration to avoid the randomness of the proposed algorithm. The experimental results show that OMFS, compared to the high-dimensional feature set, although the AUC under RF is decreased by 2%, the True Positive Rate (TPR) is increased by 21% and the feature dimension is reduced by 92%. And the average detecting time is decreased by 83%; moreover, by comparing to the Traditional GA (TGA) and Imperialist Competitive Algorithm (ICA), the F1 score under Bayes Net (BN) is increased by 4.2% and 3.5% respectively. The experimental results that the IFS-BIGGA can effectively reduce the dimension of features, which means it can effectively reduce the calculation cost, improves the detection efficiency in the actual Web spam detection inspection project.

Key words: feature selection, Genetic Algorithm (GA), Information Gain (IG), Random Forest (RF) algorithm, Web spam detection

中图分类号:

王嘉卿, 朱焱, 陈同孝, 张真诚. 欺诈网页检测中基于遗传算法的特征优选[J]. 计算机应用, 2018, 38(1): 295-299.

WANG Jiaqing, ZHU Yan, CHEN Tung-shou, CHANG Chin-chen. Optimum feature selection based on genetic algorithm under Web spam detection[J]. Journal of Computer Applications, 2018, 38(1): 295-299.

参考文献

[1] GOH L, SINGH A K, LIM K H. Multilayer perceptrons neural network based Web spam detection application[C]//Proceedings of the 1st IEEE China Summit and International Conference on Signal and Information Processing. Piscataway, NJ:IEEE, 2013:636-640.
[2] PAGE L. The PageRank citation ranking:bringing order to the Web[J]. Stanford Digital Libraries Working Paper, 1998, 9(1):1-14.
[3] 李智超,余慧佳,刘奕群,等.网页作弊与反作弊技术综述[J].山东大学学报(理学版),2011,46(5):1-8.(LI Z C, YU H J, LIU Y Q, et al. A survey of Web spam and anti-spam[J]. Journal of Shandong University (Natural Science), 2011, 46(5):1-8.)
[4] ARAUJO L, MARTINEZ-ROMO J. Web spam detection:new classification features based on qualified link analysis and language models[J]. IEEE Transactions on Information Forensics & Security, 2010, 5(3):581-590.
[5] NTOULAS A, NAJORK M, MANASSE M, et al. Detecting spam Web pages through content analysis[C]//Proceedings of the 15th International Conference on World Wide Web. New York:ACM, 2006:83-92.
[6] KUMAR S, GAO X, WELCH I, et al. A machine learning based Web spam filtering approach[C]//Proceedings of the 30th IEEE International Conference on Advanced Information Networking & Applications. Piscataway, NJ:IEEE, 2016:973-980.
[7] 李法良,朱焱,曾俊东.集成PCA降维与分类算法的垃圾网页检测[J].计算机应用与软件,2014,31(10):269-272.(LI F L, ZHU Y, ZENG J D. Spam Webpage detection combining PCA dimensionality reduction and classifier algorithm[J]. Journal of Computer Applications and Software, 2014, 31(10):269-272)
[8] 刘卫红,方卫东,董守斌,等.基于内容与链接特征的中文垃圾网页分类[J].微计算机信息,2010,26(9):6-8.(LIU W H, FANG W D, DONG S B, et al. Chinese Web spam identification through content and hyperlinks[J]. Journal of Microcomputer Information, 2010, 26(9):6-8.)
[9] 韦莎,朱焱.主题相似度与链接权重相结合的垃圾网页排序检测[J].计算机应用,2016,36(3):735-739.(WEI S, ZHU Y. Combining topic similarity with link weight for Web spam ranking detection[J]. Journal of Computer Applications, 2016, 36(3):735-739.)
[10] GOH K L, SINGH A K. Comprehensive literature review on machine learning structures for Web spam classification[J]. Procedia Computer Science, 2015, 70:434-441.
[11] KARIMPOUR J, NOROOZI A A, ABADI A. The impact of feature selection on Web spam detection[J]. International Journal of Intelligent Systems and Applications, 2012, 4(9):59-64.
[12] LUCKNER M, GAD M, SOBKOWIAK P. Stable Web spam detection using features based on lexical items[J]. Computers and Security, 2014, 46:79-93.
[13] KULLBACK, S, LEIBLER, R A. On information and sufficiency[J]. Annals of Mathematical Statistics, 1951, 22(1):79-86.
[14] GREFENSTETTE J J. Optimization of control parameters for genetic algorithms[J]. IEEE Transactions on Systems Man and Cybernetics, 1986, 16(1):122-128.
[15] CASTILLO C, DONATO D, GIONIS A, et al. Know your neighbors:Web spam detection using the Web topology[C]//Proceedings of the 2007 International ACM SIGIR Conference on Research and Development in Information Retrieval. New York:ACM, 2007:423-430.

欺诈网页检测中基于遗传算法的特征优选

Optimum feature selection based on genetic algorithm under Web spam detection

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

[1]	刘栋, 李晨航, 吴长茂, 茹法鑫, 夏媛媛. 基于可校正强化搜索遗传算法的光学系统自动设计[J]. 《计算机应用》唯一官方网站, 2024, 44(9): 2838-2847.
[2]	陈虹, 齐兵, 金海波, 武聪, 张立昂. 融合1D-CNN与BiGRU的类不平衡流量异常检测[J]. 《计算机应用》唯一官方网站, 2024, 44(8): 2493-2499.
[3]	高麟, 周宇, 邝得互. 进化双层自适应局部特征选择[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1408-1414.
[4]	李炫锋, 刘晟材, 唐珂. 机会约束的多选择背包问题的遗传算法求解[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1378-1385.
[5]	雷明珠, 王浩, 贾蓉, 白琳, 潘晓英. 基于特征间关系合成少数类样本的过采样算法[J]. 《计算机应用》唯一官方网站, 2024, 44(5): 1428-1436.
[6]	佘维, 李阳, 钟李红, 孔德锋, 田钊. 基于改进实数编码遗传算法的神经网络超参数优化[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 671-676.
[7]	孙林, 刘梦含. 基于自适应布谷鸟优化特征选择的K-means聚类[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 831-841.
[8]	徐大鹏, 侯新民. 基于网络结构设计的图神经网络特征选择方法[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 663-670.
[9]	孟圣洁, 于万钧, 陈颖. 最大相关和最大差异的高维数据特征选择算法[J]. 《计算机应用》唯一官方网站, 2024, 44(3): 767-771.
[10]	刘晶鑫, 黄雯静, 徐亮胜, 黄冲, 吴建生. 字典学习与样本关联保持结合的无监督特征选择模型[J]. 《计算机应用》唯一官方网站, 2024, 44(12): 3766-3775.
[11]	何添, 沈宗鑫, 黄倩倩, 黄雁勇. 基于自适应学习的多视图无监督特征选择方法[J]. 《计算机应用》唯一官方网站, 2023, 43(9): 2657-2664.
[12]	梁军, 洪泽泓, 余松森. 基于改进粒子群优化算法和遗传变异的图像分割模型[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1743-1749.
[13]	孙林, 黄金旭, 徐久成. 基于邻域容差互信息和鲸鱼优化算法的非平衡数据特征选择[J]. 《计算机应用》唯一官方网站, 2023, 43(6): 1842-1854.
[14]	王彬, 向甜, 吕艺东, 王晓帆. 基于NSGA‑Ⅱ的自适应多尺度特征通道分组优化算法[J]. 《计算机应用》唯一官方网站, 2023, 43(5): 1401-1408.
[15]	于振华, 刘争气, 刘颖, 郭城. 基于自适应混合粒子群优化的软件缺陷预测特征选择方法[J]. 《计算机应用》唯一官方网站, 2023, 43(4): 1206-1213.