# MiLQ: Benchmarking IR Models for Bilingual Web Search with Mixed Language Queries

arXiv: https://arxiv.org/abs/2505.16631 · HTML: https://mlsanity.com/paper/2505.16631/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 17 of 17)

- [MS MARCO: A Human Generated MAchine Reading COmprehension Dataset](https://mlsanity.com/paper/1611.09268/) (2016)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://mlsanity.com/paper/2112.09118/) (2021)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [FastText.zip: Compressing text classification models](https://mlsanity.com/paper/1612.03651/) (2016)
- [XOR QA: Cross-lingual Open-Retrieval Question Answering](https://mlsanity.com/paper/2010.11856/) (2020)
- [Deeper Text Understanding for IR with Contextual Neural Language Modeling](https://mlsanity.com/paper/1905.09217/) (2019)
- [A Survey of Code-switched Speech and Language Processing](https://mlsanity.com/paper/1904.00784/) (2019)
- [Pyserini: An Easy-to-Use Python Toolkit to Support Replicable IR Research with Sparse and Dense Representations](https://mlsanity.com/paper/2102.10073/) (2021)
- [Prompting Multilingual Large Language Models to Generate Code-Mixed Texts: The Case of South East Asian Languages](https://mlsanity.com/paper/2303.13592/) (2023)
- [Transfer Learning Approaches for Building Cross-Language Dense Retrieval Models](https://mlsanity.com/paper/2201.08471/) (2022)
- [Translate-Distill: Learning Cross-Language Dense Retrieval by Translation and Distillation](https://mlsanity.com/paper/2401.04810/) (2024)
- [Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport Distillation](https://mlsanity.com/paper/2301.12566/) (2023)
- [Overview of the TREC 2022 NeuCLIR Track](https://mlsanity.com/paper/2304.12367/) (2023)
- [Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages](https://mlsanity.com/paper/2412.12806/) (2024)
- [An Analysis on Matching Mechanisms and Token Pruning for Late-interaction Models](https://mlsanity.com/paper/2403.13291/) (2024)
- [Boosting Zero-shot Cross-lingual Retrieval by Training on Artificially Code-Switched Data](https://mlsanity.com/paper/2305.05295/) (2023)

## Abstract (arXiv)

Despite bilingual speakers frequently using mixed-language queries in web searches, Information Retrieval (IR) research on them remains scarce. To address this, we introduce MiLQ, Mixed-Language Query test set, the first public benchmark of mixed-language queries, qualified as realistic and relatively preferred. Experiments show that multilingual IR models perform moderately on MiLQ and inconsistently across native, English, and mixed-language queries, also suggesting code-switched training data's potential for robust IR models handling such queries. Meanwhile, intentional English mixing in queries proves an effective strategy for bilinguals searching English documents, which our analysis attributes to enhanced token matching compared to native queries.