Abstract: The inherent sparse features and unbalanced sample of the short text make it difficult for short text to use traditional weight of long text mechanically. To resolve this problem, an approach of short text feature weight named Integrated Category (IC) was proposed. This approach introduced the concept of inverse document frequency and relevancy frequency, and integrated the distribution of sample in positive category and negative category. The experimental results show that, compared with other feature weight methods, the micro-average and macro-average of this method are above 90%, and it can enhance the sample categories distinguishing ability in negative category, and improve the precision and recall of short text categorization.

Key words: short text, feature weight, unbalanced sample, text categorization

摘要： 短文本固有的特征稀疏和样本高度不均衡等特点，使得传统长文本的加权方法难以直接套用。针对此问题，提出一种针对短文本的特征权重计算方法——综合类别法。该方法引入反文档频和相关性频率的概念，综合考虑了样本在正类和负类中的分布情况。实验结果表明，相对于其他特征权重方法，该方法的微平均和宏平均值均在90%以上，能增强样本在负类中的类别区分能力，改善短文本分类的查准率和查全率。

关键词: 短文本, 特征权重, 不均衡样本, 文本分类