# Streamlining Evaluation with ir-measures

arXiv: https://arxiv.org/abs/2111.13466 · HTML: https://mlsanity.com/paper/2111.13466/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2021.

## Built on by (most cited first, 10 of 10)

- [Neural Approaches to Multilingual Information Retrieval](https://mlsanity.com/paper/2209.01335/) (2022)
- [Multimodal Learned Sparse Retrieval with Probabilistic Expansion Control](https://mlsanity.com/paper/2402.17535/) (2024)
- [Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data](https://mlsanity.com/paper/2305.05295/) (2023)
- [Shallow Cross-Encoders for Low-Latency Retrieval](https://mlsanity.com/paper/2403.20222/) (2024)
- [Responsible Retrieval Augmented Generation for Climate Decision Making from Documents](https://mlsanity.com/paper/2410.23902/) (2024)
- [Ranger: A Toolkit for Effect-Size Based Multi-Task Evaluation](https://mlsanity.com/paper/2305.15048/) (2023)
- [Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers](https://mlsanity.com/paper/2503.22672/) (2025)
- [Beyond Relevance: On the Relationship Between Retrieval and RAG Information Coverage](https://mlsanity.com/paper/2603.08819/) (2026)
- [MeSH Suggester: A Library and System for MeSH Term Suggestion for Systematic Review Boolean Query Construction](https://mlsanity.com/paper/2212.09018/) (2022)
- [MIDR: Enrichment-Augmented Indexing for Multimodal Document Retrieval](https://mlsanity.com/paper/2609.01316/) (2026)

## Abstract (arXiv)

We present ir-measures, a new tool that makes it convenient to calculate a diverse set of evaluation measures used in information retrieval. Rather than implementing its own measure calculations, ir-measures provides a common interface to a handful of evaluation tools. The necessary tools are automatically invoked (potentially multiple times) to calculate all the desired metrics, simplifying the evaluation process for the user. The tool also makes it easier for researchers to use recently-proposed measures (such as those from the C/W/L framework) alongside traditional measures, potentially encouraging their adoption.