# Evaluating Interpolation and Extrapolation Performance of Neural Retrieval Models

arXiv: https://arxiv.org/abs/2204.11447 · HTML: https://mlsanity.com/paper/2204.11447/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2022.

## Built on by (most cited first, 9 of 9)

- [Information Retrieval Meets Large Language Models: A Strategic Report from Chinese IR Community](https://mlsanity.com/paper/2307.09751/) (2023)
- [Reducing the Footprint of Multi-Vector Retrieval with Minimal Performance Impact via Token Pooling](https://mlsanity.com/paper/2409.14683/) (2024)
- [Resources for Brewing BEIR: Reproducible Reference Models and an Official Leaderboard](https://mlsanity.com/paper/2306.07471/) (2023)
- [Disentangled Modeling of Domain and Relevance for Adaptable Dense Retrieval](https://mlsanity.com/paper/2208.05753/) (2022)
- [A Theoretical Framework for OOD Robustness in Transformers using Gevrey Classes](https://mlsanity.com/paper/2504.12991/) (2025)
- [On Generalization for Generative Flow Networks](https://mlsanity.com/paper/2407.03105/) (2024)
- [Annotating Data for Fine-Tuning a Neural Ranker? Current Active Learning Strategies are not Better than Random Selection](https://mlsanity.com/paper/2309.06131/) (2023)
- [GEM: A Native Graph-based Index for Multi-Vector Retrieval](https://mlsanity.com/paper/2603.20336/) (2026)
- [Domain Adaptation of Multilingual Semantic Search -- Literature Review](https://mlsanity.com/paper/2402.02932/) (2024)

## Abstract (arXiv)

A retrieval model should not only interpolate the training data but also extrapolate well to the queries that are different from the training data. While neural retrieval models have demonstrated impressive performance on ad-hoc search benchmarks, we still know little about how they perform in terms of interpolation and extrapolation. In this paper, we demonstrate the importance of separately evaluating the two capabilities of neural retrieval models. Firstly, we examine existing ad-hoc search benchmarks from the two perspectives. We investigate the distribution of training and test data and find a considerable overlap in query entities, query intent, and relevance labels. This finding implies that the evaluation on these test sets is biased toward interpolation and cannot accurately reflect the extrapolation capacity. Secondly, we propose a novel evaluation protocol to separately evaluate the interpolation and extrapolation performance on existing benchmark datasets. It resamples the training and test data based on query similarity and utilizes the resampled dataset for training and evaluation. Finally, we leverage the proposed evaluation protocol to comprehensively revisit a number of widely-adopted neural retrieval models. Results show models perform differently when moving from interpolation to extrapolation. For example, representation-based retrieval models perform almost as well as interaction-based retrieval models in terms of interpolation but not extrapolation. Therefore, it is necessary to separately evaluate both interpolation and extrapolation performance and the proposed resampling method serves as a simple yet effective evaluation tool for future IR studies.