# Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling

arXiv: https://arxiv.org/abs/2403.03516 · HTML: https://mlsanity.com/paper/2403.03516/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 36 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [TyDi QA: A Benchmark for Information-Seeking Question Answering in Typologically Diverse Languages](https://mlsanity.com/paper/2003.05002/) (2020)
- [RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2010.08191/) (2020)
- ["Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection](https://mlsanity.com/paper/1705.00648/) (2017)
- [Fake News Detection on Social Media: A Data Mining Perspective](https://mlsanity.com/paper/1708.01967/) (2017)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://mlsanity.com/paper/2209.11755/) (2022)
- [Distilling Knowledge from Reader to Retriever for Question Answering](https://mlsanity.com/paper/2012.04584/) (2020)
- [Improving Passage Retrieval with Zero-Shot Question Generation](https://mlsanity.com/paper/2204.07496/) (2022)
- [Improving Efficient Neural Ranking Models with Cross-Architecture Knowledge Distillation](https://mlsanity.com/paper/2010.02666/) (2020)
- [Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup](https://mlsanity.com/paper/2101.06983/) (2021)
- [XOR QA: Cross-lingual Open-Retrieval Question Answering](https://mlsanity.com/paper/2010.11856/) (2020)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [Questions Are All You Need to Train a Dense Passage Retriever](https://mlsanity.com/paper/2206.10658/) (2022)
- [One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval](https://mlsanity.com/paper/2107.11976/) (2021)
- [Cross-lingual Information Retrieval with BERT](https://mlsanity.com/paper/2004.13005/) (2020)
- [No Parameter Left Behind: How Distillation and Model Size Affect Zero-Shot Retrieval](https://mlsanity.com/paper/2206.02873/) (2022)
- [Learning Cross-Lingual IR from an English Retriever](https://mlsanity.com/paper/2112.08185/) (2021)
- [CONVERSER: Few-Shot Conversational Dense Retrieval with Synthetic Data Generation](https://mlsanity.com/paper/2309.06748/) (2023)
- [Empowering Dual-Encoder with Query Generator for Cross-Lingual Dense Retrieval](https://mlsanity.com/paper/2303.14991/) (2023)

## Built on by (most cited first, 3 of 3)

- [PairDistill: Pairwise Relevance Distillation for Dense Retrieval](https://mlsanity.com/paper/2410.01383/) (2024)
- [Generative Query Expansion with Multilingual LLMs for Cross-Lingual Information Retrieval](https://mlsanity.com/paper/2511.19325/) (2025)
- [L"etzCross: A Cross-Lingual Page-Level Benchmark for Multimodal Retrieval over Luxembourgish Documents](https://mlsanity.com/paper/2608.21714/) (2026)

## Abstract (arXiv)

Dense retrieval methods have demonstrated promising performance in multilingual information retrieval, where queries and documents can be in different languages. However, dense retrievers typically require a substantial amount of paired data, which poses even greater challenges in multilingual scenarios. This paper introduces UMR, an Unsupervised Multilingual dense Retriever trained without any paired data. Our approach leverages the sequence likelihood estimation capabilities of multilingual language models to acquire pseudo labels for training dense retrievers. We propose a two-stage framework which iteratively improves the performance of multilingual dense retrievers. Experimental results on two benchmark datasets show that UMR outperforms supervised baselines, showcasing the potential of training multilingual retrievers without paired data, thereby enhancing their practicality. Our source code, data, and models are publicly available at https://github.com/MiuLab/UMR