# mRobust04: A Multilingual Version of the TREC Robust 2004 Benchmark

arXiv: https://arxiv.org/abs/2209.13738 · HTML: https://mlsanity.com/paper/2209.13738/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 22 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 8 of 8)

- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Pretrained Transformers for Text Ranking: BERT and Beyond](https://mlsanity.com/paper/2010.06467/) (2020)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [TREC Deep Learning Track: Reusable Test Collections in the Large Data Regime](https://mlsanity.com/paper/2104.09399/) (2021)
- [No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval](https://mlsanity.com/paper/2206.02873/) (2022)
- [HC4: A New Suite of Test Collections for Ad Hoc CLIR](https://mlsanity.com/paper/2201.09992/) (2022)
- [Can Old TREC Collections Reliably Evaluate Modern Neural Retrieval Models?](https://mlsanity.com/paper/2201.11086/) (2022)

## Built on by (most cited first, 3 of 3)

- [ptt5-v2: A Closer Look at Continued Pretraining of T5 Models for the Portuguese Language](https://mlsanity.com/paper/2406.10806/) (2024)
- [Learning to Route Queries to Heads for Attention-based Re-ranking with Large Language Models](https://mlsanity.com/paper/2604.24608/) (2026)
- [moBERTo: A Modern Encoder for Portuguese via Continued Pretraining of ModernBERT](https://mlsanity.com/paper/2606.22722/) (2026)

## Abstract (arXiv)

Robust 2004 is an information retrieval benchmark whose large number of judgments per query make it a reliable evaluation dataset. In this paper, we present mRobust04, a multilingual version of Robust04 that was translated to 8 languages using Google Translate. We also provide results of three different multilingual retrievers on this dataset. The dataset is available at https://huggingface.co/datasets/unicamp-dl/mrobust