# DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching

arXiv: https://arxiv.org/abs/2509.17768 · HTML: https://mlsanity.com/paper/2509.17768/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 43 references that are in the map.
- First posted on arXiv 22 Sep 2025.

## Builds on (most cited first, 27 of 27)

- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [GLUECoS : An Evaluation Benchmark for Code-Switched NLP](https://mlsanity.com/paper/2004.12376/) (2020)
- [LinCE: A Centralized Benchmark for Linguistic Code-switching Evaluation](https://mlsanity.com/paper/2005.04322/) (2020)
- [Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus](https://mlsanity.com/paper/2010.14571/) (2020)
- [Corpus Creation for Sentiment Analysis in Code-Mixed Tamil-English Text](https://mlsanity.com/paper/2006.00206/) (2020)
- [SemEval-2020 Task 9: Overview of Sentiment Analysis of Code-Mixed Tweets](https://mlsanity.com/paper/2008.04277/) (2020)
- [A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies](https://mlsanity.com/paper/2301.01967/) (2023)
- [A Sentiment Analysis Dataset for Code-Mixed Malayalam-English](https://mlsanity.com/paper/2006.00210/) (2020)
- [An Open Dataset and Model for Language Identification](https://mlsanity.com/paper/2305.13820/) (2023)
- [ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation](https://mlsanity.com/paper/2112.06223/) (2021)
- [Automatic Language Identification in Texts: A Survey](https://mlsanity.com/paper/1804.08186/) (2018)
- [AfroLID: A Neural Language Identification Tool for African Languages](https://mlsanity.com/paper/2210.11744/) (2022)
- [GlotLID: Language Identification for Low-Resource Languages](https://mlsanity.com/paper/2310.16248/) (2023)
- [The Decades Progress on Code-Switching Research in NLP: A Systematic Survey on Trends and Challenges](https://mlsanity.com/paper/2212.09660/) (2022)
- [HinGE: A Dataset for Generation and Evaluation of Code-Mixed Hinglish Text](https://mlsanity.com/paper/2107.03760/) (2021)
- [Bloom Library: Multimodal Datasets in 300+ Languages for a Variety of Downstream Tasks](https://mlsanity.com/paper/2210.14712/) (2022)
- [ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - English](https://mlsanity.com/paper/2211.12000/) (2022)
- [MaskLID: Code-Switching Language Identification through Iterative Masking](https://mlsanity.com/paper/2406.06263/) (2024)
- [ZAEBUC-Spoken: A Multilingual Multidialectal Arabic-English Speech Corpus](https://mlsanity.com/paper/2403.18182/) (2024)
- [The WiLI benchmark dataset for written language identification](https://mlsanity.com/paper/1801.07779/) (2018)
- [CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarization](https://mlsanity.com/paper/2303.04092/) (2023)
- [ConLID: Supervised Contrastive Learning for Low-Resource Language Identification](https://mlsanity.com/paper/2506.15304/) (2025)
- [Code-Switched Language Identification is Harder Than You Think](https://mlsanity.com/paper/2402.01505/) (2024)

## Built on by (most cited first, 3 of 3)

- [What Language is This? Ask Your Tokenizer](https://mlsanity.com/paper/2602.17655/) (2026)
- [PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues](https://mlsanity.com/paper/2601.17277/) (2026)
- [AfroScope: A Framework for Studying the Linguistic Landscape of Africa](https://mlsanity.com/paper/2601.13346/) (2026)

## Abstract (arXiv)

Language Identification (LID) is a core task in multilingual NLP, yet current systems often overfit to clean, monolingual data. This work introduces DIVERS-BENCH, a comprehensive evaluation of state-of-the-art LID models across diverse domains, including speech transcripts, web text, social media texts, children's stories, and code-switched text. Our findings reveal that while models achieve high accuracy on curated datasets, performance degrades sharply on noisy and informal inputs. We also introduce DIVERS-CS, a diverse code-switching benchmark dataset spanning 10 language pairs, and show that existing models struggle to detect multiple languages within the same sentence. These results highlight the need for more robust and inclusive LID systems in real-world settings.