关键词: 短文本, 隐马尔可夫模型, 命名实体识别, 拼音同指关系库, 词性

Abstract: Aiming at the urgent task of named entity recognition for short text, a fast and effective method was proposed. The method comprised three steps: Firstly, according to the disturbance of non-standard expression in short text, the elimination of interferential characters and text simplification were adopted. Secondly, according to the non-integrity of short text, Hidden Markov Model (HMM) was employed to preliminarily name entity recognition, in which the part of speech was used as observed value. In the end, by means of the preliminary recognition result, a pinyin co-referential relation library was established to identify the potential entity. The experiment on the test-set including 8464 short texts shows that this method has better performance to named entity recognition for short text.

Key words: short text, HMM, named entity recognition, pinyin co-referential relation library, part of speech