# Quati: A Brazilian Portuguese Information Retrieval Dataset from Native Speakers

arXiv: https://arxiv.org/abs/2404.06976 · HTML: https://mlsanity.com/paper/2404.06976/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2024.

## Built on by (most cited first, 5 of 5)

- [JurisTCU: A Brazilian Portuguese Information Retrieval Dataset with Query Relevance Judgments](https://mlsanity.com/paper/2503.08379/) (2025)
- [When LLMs Disagree: Diagnosing Relevance Filtering Bias and Retrieval Divergence in SDG Search](https://mlsanity.com/paper/2507.02139/) (2025)
- [JU'A -- A Benchmark for Information Retrieval in Brazilian Legal Text Collections](https://mlsanity.com/paper/2604.06098/) (2026)
- [moBERTo: A Modern Encoder for Portuguese via Continued Pretraining of ModernBERT](https://mlsanity.com/paper/2606.22722/) (2026)
- [MTEB-BR: A Text Embedding Benchmark for Brazilian Portuguese](https://mlsanity.com/paper/2607.04581/) (2026)

## Abstract (arXiv)

Despite Portuguese being one of the most spoken languages in the world, there is a lack of high-quality information retrieval datasets in that language. We present Quati, a dataset specifically designed for the Brazilian Portuguese language. It comprises a collection of queries formulated by native speakers and a curated set of documents sourced from a selection of high-quality Brazilian Portuguese websites. These websites are frequented more likely by real users compared to those randomly scraped, ensuring a more representative and relevant corpus. To label the query-document pairs, we use a state-of-the-art LLM, which shows inter-annotator agreement levels comparable to human performance in our assessments. We provide a detailed description of our annotation methodology to enable others to create similar datasets for other languages, providing a cost-effective way of creating high-quality IR datasets with an arbitrary number of labeled documents per query. Finally, we evaluate a diverse range of open-source and commercial retrievers to serve as baseline systems. Quati is publicly available at https://huggingface.co/datasets/unicamp-dl/quati and all scripts at https://github.com/unicamp-dl/quati .