关键词: 分布式搜索, 索引量估计, 采样-再采样, 捕获-再捕获

Abstract: Distributed search is an effective way to search the Deep Web, while collection size is an important feature in collection representation and selection in distributed search. To estimate collection size in uncooperative environments, the two novel algorithms were proposed in this paper. High frequent resample algorithm first samples collections with random queries, then resamples with high frequent queries in the sample set. Heterogeneous capture algorithm, based on the assumption of different capture probabilities among documents, uses Logistic functions and conditional maximum likelihood. Experimental results show that the algorithms outperform both sample-resample and capture-recapture algorithms.

Key words: distributed search, collection size estimation, sample-resample, capture-recapture