# Synergistic Approach for Simultaneous Optimization of Monolingual, Cross-lingual, and Multilingual Information Retrieval

arXiv: https://arxiv.org/abs/2408.10536 · HTML: https://mlsanity.com/paper/2408.10536/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 33 references that are in the map.
- First posted on arXiv 20 Aug 2024.

## Builds on (most cited first, 28 of 28)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Dense Passage Retrieval for Open-Domain Question Answering](https://mlsanity.com/paper/2004.04906/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://mlsanity.com/paper/2212.03533/) (2022)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [On the Cross-lingual Transferability of Monolingual Representations](https://mlsanity.com/paper/1910.11856/) (2019)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Large Dual Encoders Are Generalizable Retrievers](https://mlsanity.com/paper/2112.07899/) (2021)
- [MLQA: Evaluating Cross-lingual Extractive Question Answering](https://mlsanity.com/paper/1910.07475/) (2019)
- [NewsQA: A Machine Comprehension Dataset](https://mlsanity.com/paper/1611.09830/) (2016)
- [Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://mlsanity.com/paper/2104.06967/) (2021)
- [InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training](https://mlsanity.com/paper/2007.07834/) (2020)
- [Simple Entity-Centric Questions Challenge Dense Retrievers](https://mlsanity.com/paper/2109.08535/) (2021)
- [Mr. TyDi: A Multi-lingual Benchmark for Dense Retrieval](https://mlsanity.com/paper/2108.08787/) (2021)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [The Effect of Natural Distribution Shift on Question Answering Models](https://mlsanity.com/paper/2004.14444/) (2020)
- [Scalable and Efficient MoE Training for Multitask Multilingual Models](https://mlsanity.com/paper/2109.10465/) (2021)
- [DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension](https://mlsanity.com/paper/1804.07927/) (2018)
- [Making a MIRACL: Multilingual Information Retrieval Across a Continuum of Languages](https://mlsanity.com/paper/2210.09984/) (2022)
- [LAReQA: Language-agnostic answer retrieval from a multilingual pool](https://mlsanity.com/paper/2004.05484/) (2020)
- [Towards Best Practices for Training Multilingual Dense Retrieval Models](https://mlsanity.com/paper/2204.02363/) (2022)
- [Neural Approaches to Multilingual Information Retrieval](https://mlsanity.com/paper/2209.01335/) (2022)
- [MFAQ: a Multilingual FAQ Dataset](https://mlsanity.com/paper/2109.12870/) (2021)
- [Aggretriever: A Simple Approach to Aggregate Textual Representations for Robust Dense Passage Retrieval](https://mlsanity.com/paper/2208.00511/) (2022)
- [Soft Prompt Decoding for Multilingual Dense Retrieval](https://mlsanity.com/paper/2305.09025/) (2023)
- [Language Agnostic Multilingual Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2210.06633/) (2022)

## Built on by (most cited first, 2 of 2)

- [SandboxAQ's submission to MRL 2024 Shared Task on Multi-lingual Multi-task Information Retrieval](https://mlsanity.com/paper/2410.21501/) (2024)
- [Query-aware routing for Cross-lingual performance gains in Encoders](https://mlsanity.com/paper/2610.02875/) (2026)

## Abstract (arXiv)

Information retrieval across different languages is an increasingly important challenge in natural language processing. Recent approaches based on multilingual pre-trained language models have achieved remarkable success, yet they often optimize for either monolingual, cross-lingual, or multilingual retrieval performance at the expense of others. This paper proposes a novel hybrid batch training strategy to simultaneously improve zero-shot retrieval performance across monolingual, cross-lingual, and multilingual settings while mitigating language bias. The approach fine-tunes multilingual language models using a mix of monolingual and cross-lingual question-answer pair batches sampled based on dataset size. Experiments on XQuAD-R, MLQA-R, and MIRACL benchmark datasets show that the proposed method consistently achieves comparable or superior results in zero-shot retrieval across various languages and retrieval tasks compared to monolingual-only or cross-lingual-only training. Hybrid batch training also substantially reduces language bias in multilingual retrieval compared to monolingual training. These results demonstrate the effectiveness of the proposed approach for learning language-agnostic representations that enable strong zero-shot retrieval performance across diverse languages.