关键词: 不良网页, 文本分类, K最近邻分类算法, Hadoop, MapReduce

Abstract: There are many illegal Web pages on the Internet, which may have pornographic, violent, gambling or reactionary content. Without being filtered effectively, they will exercise a malign influence on the searching services. An improved K-Nearest Neighbors (KNN) classification algorithm to promote the recognition accuracy was proposed and implemented on a virtualized platform following the MapReduce model provided by the open source software Hadoop, which made it distributed and parallel. Through experiments and comparison with the existing work, it is proved that the proposed recognition method improves the accuracy and efficiency greatly. The algorithm is implemented on a virtualized platform following the MapReduce model provided by the open source software Hadoop, which makes it distributed and parallel. Through experiments and comparison with existing work, it is proved that the recognition method we propose improves the accuracy and efficiency greatly.

Key words: illegal Web page, text classification, K-Nearest Neighbors (KNN) classification algorithm, Hadoop, MapReduce