mlsanityData to 7 Oct 2026

Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval

arXiv 2311.01870 ↗First posted 3 Nov 2023cs.CL cs.AI cs.IRcited by 4

Not on the map. Why?

Abstract (arXiv)

We present Multi-EuP, a new multilingual benchmark dataset, comprising 22K multi-lingual documents collected from the European Parliament, spanning 24 languages. This dataset is designed to investigate fairness in a multilingual information retrieval (IR) context to analyze both language and demographic bias in a ranking context. It boasts an authentic multilingual corpus, featuring topics translated into all 24 languages, as well as cross-lingual relevance judgments. Furthermore, it offers rich demographic information associated with its documents, facilitating the study of demographic bias. We report the effectiveness of Multi-EuP for benchmarking both monolingual and multilingual IR. We also conduct a preliminary experiment on language bias caused by the choice of tokenization strategy.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

11 of 18 references.

References · 11

  1. 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
  2. 2016MS MARCO: A Human Generated MAchine Reading COmprehension Datasetcited by 1,591
  3. 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
  4. 2020TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languagescited by 486
  5. 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
  6. 2021mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Datasetcited by 116
  7. 2022PLAID: An Efficient Engine for Late Interaction Retrievalcited by 95
  8. 2021MultiEURLEX -- A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfercited by 89
  9. 2016Personalized Machine Translation: Preserving Original Author Traitscited by 47
  10. 2022Neural Approaches to Multilingual Information Retrievalcited by 18
  11. 2023BEIR-PL: Zero Shot Information Retrieval Benchmark for the Polish Languagecited by 15

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2024MessIRve: A Large-Scale Spanish Information Retrieval Datasetcited by 3
  2. 2025Language Bias in Information Retrieval: The Nature of the Beast and Mitigation Methodscited by 2
  3. 2025Benchmarking Gender and Political Bias in Large Language Modelscited by 0
  4. 2025Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Datasetcited by 0