Compact indexing and judicious searching for billion-scale microblog retrieval

Publication Type:
Journal Article
ACM Transactions on Information Systems, 2017, 35 (3)
Issue Date:
Filename Description Size
a27-zhang.pdfPublished Version652.32 kB
Adobe PDF
Full metadata record
© 2017 ACM. In this article, we study the problem of efficient top-k disjunctive query processing in a huge microblog dataset. In terms of compact indexing, we categorize the keywords into rare terms and common terms based on inverse document frequency (idf) and propose tailored block-oriented organization to save memory consumption. In terms of fast searching, we classify the queries into three types based on term category and judiciously design an efficient search algorithm for each type. We conducted extensive experiments on a billion-scale Twitter dataset and examined the performance with both simple and more advanced ranking functions. The results showed that with much smaller index size, our search algorithm achieves a factor of 2-3 times faster speedup over state-of-the-art solutions in both ranking scenarios.
Please use this identifier to cite or link to this item: