Distillation for Multilingual Information Retrieval
Not on the map. Why?
Abstract (arXiv)
Recent work in cross-language information retrieval (CLIR), where queries and documents are in different languages, has shown the benefit of the Translate-Distill framework that trains a cross-language neural dual-encoder model using translation and distillation. However, Translate-Distill only supports a single document language. Multilingual information retrieval (MLIR), which ranks a multilingual document collection, is harder to train than CLIR because the model must assign comparable relevance scores to documents in different languages. This work extends Translate-Distill and propose Multilingual Translate-Distill (MTD) for MLIR. We show that ColBERT-X models trained with MTD outperform their counterparts trained ith Multilingual Translate-Train, which is the previous state-of-the-art training approach, by 5% to 25% in nDCG@20 and 15% to 45% in MAP. We also show that the model is robust to the way languages are mixed in training batches. Our implementation is available on GitHub.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 55 references that are in the map. First posted on arXiv 2 May 2024.
Builds on
29 of 55 references.
References · 29
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
- 2020Dense Passage Retrieval for Open-Domain Question Answeringcited by 3,779
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,160
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
- 2021ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interactioncited by 464
- 2020RocketQA: An Optimized Training Approach to Dense Passage Retrieval for Open-Domain Question Answeringcited by 410
- 2020Document Ranking with a Pretrained Sequence-to-Sequence Modelcited by 373
- 2021SPLADE: Sparse Lexical and Expansion Model for First Stage Rankingcited by 283
17 more
- 2021Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Samplingcited by 238
- 2021Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrievalcited by 203
- 2021Mr. TyDi: A Multi-lingual Benchmark for Dense Retrievalcited by 118
- 2019Deeper Text Understanding for IR with Contextual Neural Language Modelingcited by 111
- 2022Transfer Learning Approaches for Building Cross-Language Dense Retrieval Modelscited by 36
- 2024Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillationcited by 22
- 2022CITADEL: Conditional Token Interaction via Dynamic Lexical Routing for Efficient and Effective Multi-Vector Retrievalcited by 19
- 2022Neural Approaches to Multilingual Information Retrievalcited by 18
- 2024Overview of the TREC 2023 NeuCLIR Trackcited by 17
- 2020The Sockeye 2 Neural Machine Translation Toolkit at AMTA 2020cited by 16
- 2021Learning Cross-Lingual IR from an English Retrievercited by 16
- 2022C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrievalcited by 14
- 2023SLIM: Sparsified Late Interaction for Multi-Vector Retrieval with Inverted Indexescited by 12
- 2022HC4: A New Suite of Test Collections for Ad Hoc CLIRcited by 10
- 2023Soft Prompt Decoding for Multilingual Dense Retrievalcited by 10
- 2019Cross-Lingual Relevance Transfer for Document Retrievalcited by 9
- 2023NeuralMind-UNICAMP at 2022 TREC NeuCLIR: Large Boring Rerankers for Cross-lingual Retrievalcited by 7
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.