Abstract: Aiming at the problem of data sparsity, poor portability and lack of recognition of multiple words (more than three words) in micro-blog new word discovery algorithm, a new word discovery algorithm based on improved Mutual Information (MI) and Branch Entropy (BE), named MBN-Gram, was proposed. Firstly, the N-Gram was used to extract the candidate terms of new words, and the rules of using frequency and stop words were used to filter the candidates. Then the improved MI and BE were used to expand and filter the candidates again. Finally, the corresponding dictionary was used to screen, so as to get new words. Theoretical and experimental analysis show that the accuracy rate, recall rate and F value of MBN-Gram algorithm were improved. Experimental results shows that the MBN-Gram algorithm is effective and feasible.

Key words: new word discovery, multi-word, N-gram, Mutual Information (MI), Branch Entropy (BE)

摘要： 针对目前微博新词发现算法中的数据稀疏、可移植性较差以及缺乏对多字词（大于三字）识别的问题，提出了基于改进互信息（MI）和邻接熵（BE）的微博新词发现算法——MBN-Gram。首先，利用N元递增算法（N-Gram）提取新词的候选项，对提取出来的候选新词使用频率和停用字等规则进行过滤；接着再利用改进MI和BE对候选项进行扩展及再过滤；最后，结合相应词典进行筛选，从而得到新词。通过理论及实验分析，MBN-Gram算法在准确率、召回率及F值上均有一定提高。实验结果表明，MBN-Gram算法是有效可行的。

关键词: 新词发现, 多字词, N-Gram, 互信息, 邻接熵