关键词: 转载识别, 聚团词, 特征选择, 扩展线性比较, 向量空间模型

Abstract: Document forward detection is that to find out article collection of the same or close content from a large-scale text library. It has widespread demand in popular articles exploring, results organizing of search engine, copy detection and so on. To meet the growing diverse forms of Internet text forward and improve system efficiency, this paper discussed certain text features and researched some comparison algorithms. Then, the large-scale document forward detection algorithm based on agglomerate-term was introduced. Its principle is: first, detect and extract the agglomerate-term according to the term's distribution, and make it a key feature to characterize the text; then, set an extensive linear comparison and a multi-dimensional comparison on it; finally, compute the ultimate results of the forward detection. The experimental results show that the agglomerate-term algorithm has a better integrated performance of precision, recall and speed.

Key words: forward detection, Agglomerate-Term (AgT), feature selection, extensive linear comparison, Vector Space Model (VSM)