# Cross-lingual Information Retrieval with BERT

arXiv: https://arxiv.org/abs/2004.13005 · HTML: https://mlsanity.com/paper/2004.13005/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Apr 2020.

## Built on by (most cited first, 27 of 27)

- [Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models](https://mlsanity.com/paper/2201.08471/) (2022)
- [On Cross-Lingual Retrieval with Multilingual Text Encoders](https://mlsanity.com/paper/2112.11031/) (2021)
- [Parameter-Efficient Neural Reranking for Cross-Lingual and Multilingual Retrieval](https://mlsanity.com/paper/2204.02292/) (2022)
- [Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval](https://mlsanity.com/paper/2101.08370/) (2021)
- [Learning Cross-Lingual IR from an English Retriever](https://mlsanity.com/paper/2112.08185/) (2021)
- [C3: Continued Pretraining with Contrastive Weak Supervision for Cross Language Ad-Hoc Retrieval](https://mlsanity.com/paper/2204.11989/) (2022)
- [CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval](https://mlsanity.com/paper/2209.02071/) (2022)
- [X-METRA-ADA: Cross-lingual Meta-Transfer Learning Adaptation to Natural Language Understanding and Question Answering](https://mlsanity.com/paper/2104.09696/) (2021)
- [Graph-based Multilingual Product Retrieval in E-commerce Search](https://mlsanity.com/paper/2105.02978/) (2021)
- [Mixed Attention Transformer for Leveraging Word-Level Knowledge to Neural Cross-Lingual Information Retrieval](https://mlsanity.com/paper/2109.02789/) (2021)
- [Mind the Gap: Cross-Lingual Information Retrieval with Hierarchical Knowledge Enhancement](https://mlsanity.com/paper/2112.13510/) (2021)
- [Generative Retrieval as Multi-Vector Dense Retrieval](https://mlsanity.com/paper/2404.00684/) (2024)
- [Cross-Lingual Training with Dense Retrieval for Document Retrieval](https://mlsanity.com/paper/2109.01628/) (2021)
- [Machine Translation Impact in E-commerce Multilingual Search](https://mlsanity.com/paper/2302.00119/) (2023)
- [Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data](https://mlsanity.com/paper/2305.05295/) (2023)
- [Unified Embedding Based Personalized Retrieval in Etsy Search](https://mlsanity.com/paper/2306.04833/) (2023)
- [Hubness Reduction Improves Sentence-BERT Semantic Spaces](https://mlsanity.com/paper/2311.18364/) (2023)
- [ZC3: Zero-Shot Cross-Language Code Clone Detection](https://mlsanity.com/paper/2308.13754/) (2023)
- [Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling](https://mlsanity.com/paper/2403.03516/) (2024)
- [CLIRudit: Cross-Lingual Information Retrieval of Scientific Documents](https://mlsanity.com/paper/2504.16264/) (2025)
- [Anveshana: A New Benchmark Dataset for Cross-Lingual Information Retrieval On English Queries and Sanskrit Documents](https://mlsanity.com/paper/2505.19494/) (2025)
- [Leveraging Advantages of Interactive and Non-Interactive Models for Vector-Based Cross-Lingual Information Retrieval](https://mlsanity.com/paper/2111.01992/) (2021)
- [CDA: a Cost Efficient Content-based Multilingual Web Document Aligner](https://mlsanity.com/paper/2102.10246/) (2021)
- [Cross-language Information Retrieval](https://mlsanity.com/paper/2111.05988/) (2021)
- [Enhancing E-commerce Product Title Translation with Retrieval-Augmented Generation and Large Language Models](https://mlsanity.com/paper/2409.12880/) (2024)
- [Exploring new Approaches for Information Retrieval through Natural Language Processing](https://mlsanity.com/paper/2505.02199/) (2025)
- [Enhancing Generative Information Extraction with Two-step Validation: A Product Attribute Use Case](https://mlsanity.com/paper/2607.26780/) (2026)

## Abstract (arXiv)

Multiple neural language models have been developed recently, e.g., BERT and XLNet, and achieved impressive results in various NLP tasks including sentence classification, question answering and document ranking. In this paper, we explore the use of the popular bidirectional language model, BERT, to model and learn the relevance between English queries and foreign-language documents in the task of cross-lingual information retrieval. A deep relevance matching model based on BERT is introduced and trained by finetuning a pretrained multilingual BERT model with weak supervision, using home-made CLIR training data derived from parallel corpora. Experimental results of the retrieval of Lithuanian documents against short English queries show that our model is effective and outperforms the competitive baseline approaches.