Abstract: To solve the low performance and efficiency issues of the traditional centralized index when processing large-scale unstructured knowledge, the authors proposed the retrieval algorithm based on text clustering. The algorithm used text clustering algorithm to improve the existing index distribution method, and reduced the search range by judging the query intent through the distance of query and clusters. The experimental results show that the proposed scheme can effectively alleviate the pressure of indexing and retrieval in handling large-scale data. It greatly improves the performance of distributed retrieval, and it still maintains relatively high accuracy rate and recall rate.

Key words: unstructured knowledge, distributed index, text clustering, full-text search, parallel retrieval

摘要： 针对传统集中式索引处理大规模数据的性能和效率问题，提出了一种基于文本聚类的检索算法。利用文本聚类算法改进现有的索引划分方案，根据查询与聚类结果的距离计算判断查询意图，缩减查询范围。实验结果表明，所提方案能够有效地缓解大规模数据建索引和检索的压力，大幅提高分布式检索性能，同时保持着较高的准确率和查全率。

关键词: 非结构化知识, 分布式索引, 文本聚类, 全文检索, 并行检索