# Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data

arXiv: https://arxiv.org/abs/2305.05295 · HTML: https://mlsanity.com/paper/2305.05295/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 9 May 2023.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Exploiting Similarities among Languages for Machine Translation](https://mlsanity.com/paper/1309.4168/) (2013)
- [mMARCO: A Multilingual Version of the MS MARCO Passage Ranking Dataset](https://mlsanity.com/paper/2108.13897/) (2021)
- [Translation Artifacts in Cross-lingual Transfer Learning](https://mlsanity.com/paper/2004.04721/) (2020)
- [Revisiting the Primacy of English in Zero-shot Cross-lingual Transfer](https://mlsanity.com/paper/2106.16171/) (2021)
- [LAReQA: Language-agnostic answer retrieval from a multilingual pool](https://mlsanity.com/paper/2004.05484/) (2020)
- [XLDA: Cross-Lingual Data Augmentation for Natural Language Inference and Question Answering](https://mlsanity.com/paper/1905.11471/) (2019)
- [From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Text](https://mlsanity.com/paper/2107.06483/) (2021)
- [Cross-lingual Information Retrieval with BERT](https://mlsanity.com/paper/2004.13005/) (2020)
- [Code-Mixing on Sesame Street: Dawn of the Adversarial Polyglots](https://mlsanity.com/paper/2103.09593/) (2021)
- [On Cross-Lingual Retrieval with Multilingual Text Encoders](https://mlsanity.com/paper/2112.11031/) (2021)
- [Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval](https://mlsanity.com/paper/2204.02292/) (2022)
- [Teaching a New Dog Old Tricks: Resurrecting Multilingual Retrieval Using Zero-shot Learning](https://mlsanity.com/paper/1912.13080/) (2019)
- [Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport Distillation](https://mlsanity.com/paper/2301.12566/) (2023)
- [C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval](https://mlsanity.com/paper/2204.11989/) (2022)
- [Streamlining Evaluation with ir-measures](https://mlsanity.com/paper/2111.13466/) (2021)
- [El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing](https://mlsanity.com/paper/2101.10524/) (2021)
- [Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections](https://mlsanity.com/paper/1906.02656/) (2019)
- [A cost-benefit analysis of cross-lingual transfer methods](https://mlsanity.com/paper/2105.06813/) (2021)

## Built on by (most cited first, 4 of 4)

- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [Disentangled Contrastive Learning for Zero-Shot Multilingual Dense Retrieval](https://mlsanity.com/paper/2608.02189/) (2026)
- [MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries](https://mlsanity.com/paper/2505.16631/) (2025)
- [When Does Mixing Help? Analyzing Query Embedding Interpolation in Multilingual Dense Retrieval](https://mlsanity.com/paper/2606.13537/) (2026)

## Abstract (arXiv)

Transferring information retrieval (IR) models from a high-resource language (typically English) to other languages in a zero-shot fashion has become a widely adopted approach. In this work, we show that the effectiveness of zero-shot rankers diminishes when queries and documents are present in different languages. Motivated by this, we propose to train ranking models on artificially code-switched data instead, which we generate by utilizing bilingual lexicons. To this end, we experiment with lexicons induced from (1) cross-lingual word embeddings and (2) parallel Wikipedia page titles. We use the mMARCO dataset to extensively evaluate reranking models on 36 language pairs spanning Monolingual IR (MoIR), Cross-lingual IR (CLIR), and Multilingual IR (MLIR). Our results show that code-switching can yield consistent and substantial gains of 5.1 MRR@10 in CLIR and 3.9 MRR@10 in MLIR, while maintaining stable performance in MoIR. Encouragingly, the gains are especially pronounced for distant languages (up to 2x absolute gain). We further show that our approach is robust towards the ratio of code-switched tokens and also extends to unseen languages. Our results demonstrate that training on code-switched data is a cheap and effective way of generalizing zero-shot rankers for cross-lingual and multilingual retrieval.