Entropy-Based Query Performance Prediction for Neural Information Retrieval Systems

Performance prediction is an important aspect of Information Retrieval (IR), as determining the effectiveness of search results without human relevance judgments has many important applications. We propose a novel Query Performance Prediction (QPP) method to predict the effectiveness of neural reranking models. Our approach uses the retrieval score distribution for a query and a set of highest-scoring documents to estimate the likelihood of effectiveness. This method is both efficient and unsupervised, making it possible to use in production retrieval systems. The new method uses entropy, which is the key measure in information theory. The core idea is simple but novel – measure the entropy of the retrieval scores for a reranking model while using no training data or corpus related statistics. Our empirical experiments show the effectiveness of our proposed method, which is comparable with traditional state-of-the-art QPP methods in terms of both prediction quality and computational efficiency.

Conference paper