# Declarative Experimentation in Information Retrieval using PyTerrier

arXiv: https://arxiv.org/abs/2007.14271 · HTML: https://mlsanity.com/paper/2007.14271/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jul 2020.

## Built on by (most cited first, 30 of 30)

- [Simplified Data Wrangling with ir_datasets](https://mlsanity.com/paper/2103.02280/) (2021)
- [MILL: Mutual Verification with Large Language Models for Zero-Shot Query Expansion](https://mlsanity.com/paper/2310.19056/) (2023)
- [Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval](https://mlsanity.com/paper/2212.10526/) (2022)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [Cross-Lingual Open-Domain Question Answering with Answer Sentence Generation](https://mlsanity.com/paper/2110.07150/) (2021)
- [Improving Content Retrievability in Search with Controllable Query Generation](https://mlsanity.com/paper/2303.11648/) (2023)
- [Generative Product Recommendations for Implicit Superlative Queries](https://mlsanity.com/paper/2504.18748/) (2025)
- [Constructing and Evaluating Declarative RAG Pipelines in PyTerrier](https://mlsanity.com/paper/2506.10802/) (2025)
- [QueryGym: A Toolkit for Reproducible LLM-Based Query Reformulation](https://mlsanity.com/paper/2511.15996/) (2025)
- [Utilizing ChatGPT to Enhance Clinical Trial Enrollment](https://mlsanity.com/paper/2306.02077/) (2023)
- [Retrieval Augmented Zero-Shot Text Classification](https://mlsanity.com/paper/2406.15241/) (2024)
- [Crafting the Path: Robust Query Rewriting for Information Retrieval](https://mlsanity.com/paper/2407.12529/) (2024)
- [On Single and Multiple Representations in Dense Passage Retrieval](https://mlsanity.com/paper/2108.06279/) (2021)
- [Text Simplification of Scientific Texts for Non-Expert Readers](https://mlsanity.com/paper/2307.03569/) (2023)
- [Combining Deep Neural Reranking and Unsupervised Extraction for Multi-Query Focused Summarization](https://mlsanity.com/paper/2302.01148/) (2023)
- [SPRINT: A Unified Toolkit for Evaluating and Demystifying Zero-shot Neural Sparse Retrieval](https://mlsanity.com/paper/2307.10488/) (2023)
- [Towards Proactive Information Retrieval in Noisy Text with Wikipedia Concepts](https://mlsanity.com/paper/2210.09877/) (2022)
- [Multi-Query Focused Disaster Summarization via Instruction-Based Prompting](https://mlsanity.com/paper/2402.09008/) (2024)
- [Which Neurons Matter in IR? Applying Integrated Gradients-based Methods to Understand Cross-Encoders](https://mlsanity.com/paper/2406.19309/) (2024)
- [UNIMIB at TREC 2021 Clinical Trials Track](https://mlsanity.com/paper/2207.13514/) (2022)
- [Executive Voiced Laughter and Social Approval: An Explorative Machine Learning Study](https://mlsanity.com/paper/2305.09485/) (2023)
- [Jean-Luc Picard at Touch'e 2023: Comparing Image Generation, Stance Detection and Feature Matching for Image Retrieval for Arguments](https://mlsanity.com/paper/2307.09172/) (2023)
- [Coarse-Tuning for Ad-hoc Document Retrieval Using Pre-trained Language Models](https://mlsanity.com/paper/2403.16915/) (2024)
- [THM@SimpleText 2025 -- Task 1.1: Revisiting Text Simplification based on Complex Terms for Non-Experts](https://mlsanity.com/paper/2507.04414/) (2025)
- [Trove: A Flexible Toolkit for Dense Retrieval](https://mlsanity.com/paper/2511.01857/) (2025)
- [Orcheo: A Modular Full-Stack Platform for Conversational Search](https://mlsanity.com/paper/2602.14710/) (2026)
- [JobMatchAI-An Intelligent Job Matching Platform Using Knowledge Graphs, Semantic Search and Explainable AI](https://mlsanity.com/paper/2603.14558/) (2026)
- [flexvec: SQL Vector Retrieval with Programmatic Embedding Modulation](https://mlsanity.com/paper/2603.22587/) (2026)
- [Robustness of IR Models to Collection Growth](https://mlsanity.com/paper/2608.23419/) (2026)
- [Benchmarking Literature Retrieval for a Model Organism: A Dictyostelium Case Study](https://mlsanity.com/paper/2610.03130/) (2026)

## Abstract (arXiv)

The advent of deep machine learning platforms such as Tensorflow and Pytorch, developed in expressive high-level languages such as Python, have allowed more expressive representations of deep neural network architectures. We argue that such a powerful formalism is missing in information retrieval (IR), and propose a framework called PyTerrier that allows advanced retrieval pipelines to be expressed, and evaluated, in a declarative manner close to their conceptual design. Like the aforementioned frameworks that compile deep learning experiments into primitive GPU operations, our framework targets IR platforms as backends in order to execute and evaluate retrieval pipelines. Further, we can automatically optimise the retrieval pipelines to increase their efficiency to suite a particular IR platform backend. Our experiments, conducted on TREC Robust and ClueWeb09 test collections, demonstrate the efficiency benefits of these optimisations for retrieval pipelines involving both the Anserini and Terrier IR platforms.