关键词: 特征选择, 文本分类, 文档频, 二进制可辨矩阵, 粗糙集, 属性约简, feature selection, text categorization, document frequency, binary discernibility matrix, Rough Set (RS), attribution reduction

Abstract: Feature selection is a core research topic in text categorization. Several classic feature selection methods were analyzed and their deficiencies were summarized. A new document frequency was proposed, and Rough Set (RS) theory was adopted to provide an attribute reduction algorithm based on binary discernibility matrix. Based on the attribute reduction algorithm and the new document frequency, a comprehensive feature selection method was given. The comprehensive method firstly used the new document frequency to select features to filter out some terms, and then employed the attribute reduction algorithm to eliminate redundancy. The experimental results on data of 8 classes, 300 documents each class from http://www.people.com.cn show that the comprehensive method has higher accuracy and recall rate compared with Mutual Information (MI), CHI value and Information Gain (IG) methods.