# Breaking Flat: A Generalised Query Performance Prediction Evaluation Framework

arXiv: https://arxiv.org/abs/2601.17359 · HTML: https://mlsanity.com/paper/2601.17359/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2026.

## Built on by (most cited first, 1 of 1)

- [Can QPP Choose the Right Query Variant? Evaluating Query Variant Selection for RAG Pipelines](https://mlsanity.com/paper/2604.22661/) (2026)

## Abstract (arXiv)

The traditional use-case of query performance prediction (QPP) is to identify which queries perform well and which perform poorly for a given ranking model. A more fine-grained and arguably more challenging extension of this task is to determine which ranking models are most effective for a given query. In this work, we generalize the QPP task and its evaluation into three settings: (i) SingleRanker MultiQuery (SRMQ-PP), corresponding to the standard use case; (ii) MultiRanker SingleQuery (MRSQ-PP), which evaluates a QPP model's ability to select the most effective ranker for a query; and (iii) MultiRanker MultiQuery (MRMQ-PP), which considers predictions jointly across all query ranker pairs. Our results show that (a) the relative effectiveness of QPP models varies substantially across tasks (SRMQ-PP vs. MRSQ-PP), and (b) predicting the best ranker for a query is considerably more difficult than predicting the relative difficulty of queries for a given ranker.