# Automatic Language Identification in Texts: A Survey

arXiv: https://arxiv.org/abs/1804.08186 · HTML: https://mlsanity.com/paper/1804.08186/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2018.

## Built on by (most cited first, 30 of 41)

- [An Open Dataset and Model for Language Identification](https://mlsanity.com/paper/2305.13820/) (2023)
- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [AfroLID: A Neural Language Identification Tool for African Languages](https://mlsanity.com/paper/2210.11744/) (2022)
- [GlotLID: Language Identification for Low-Resource Languages](https://mlsanity.com/paper/2310.16248/) (2023)
- [Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data](https://mlsanity.com/paper/2105.15071/) (2021)
- [GlotCC: An Open Broad-Coverage CommonCrawl Corpus and Pipeline for Minority Languages](https://mlsanity.com/paper/2410.23825/) (2024)
- [Findings of the VarDial Evaluation Campaign 2023](https://mlsanity.com/paper/2305.20080/) (2023)
- [Language Variety Identification with True Labels](https://mlsanity.com/paper/2303.01490/) (2023)
- [Comparing Approaches to Dravidian Language Identification](https://mlsanity.com/paper/2103.05552/) (2021)
- [MaskLID: Code-Switching Language Identification through Iterative Masking](https://mlsanity.com/paper/2406.06263/) (2024)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [PALI: A Language Identification Benchmark for Perso-Arabic Scripts](https://mlsanity.com/paper/2304.01322/) (2023)
- [Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech](https://mlsanity.com/paper/2202.12163/) (2022)
- [LIMIT: Language Identification, Misidentification, and Translation using Hierarchical Models in 350+ Languages](https://mlsanity.com/paper/2305.14263/) (2023)
- [Fumbling in Babel: An Investigation into ChatGPT's Language Identification Ability](https://mlsanity.com/paper/2311.09696/) (2023)
- [OpenLID-v3: Improving the Precision of Closely Related Language Identification -- An Experience Report](https://mlsanity.com/paper/2602.13139/) (2026)
- [A reproduction of Apple's bi-directional LSTM models for language identification in short strings](https://mlsanity.com/paper/2102.06282/) (2021)
- [Language Identification for Austronesian Languages](https://mlsanity.com/paper/2206.04327/) (2022)
- [Code-Switched Language Identification is Harder Than You Think](https://mlsanity.com/paper/2402.01505/) (2024)
- [Extracting Lexical Features from Dialects via Interpretable Dialect Classifiers](https://mlsanity.com/paper/2402.17914/) (2024)
- [DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching](https://mlsanity.com/paper/2509.17768/) (2025)
- [Cross-Lingual Citations in English Papers: A Large-Scale Analysis of Prevalence, Usage, and Impact](https://mlsanity.com/paper/2111.05097/) (2021)
- [ML-On-Rails: Safeguarding Machine Learning Models in Software Systems A Case Study](https://mlsanity.com/paper/2401.06513/) (2024)
- [Multi-label Scandinavian Language Identification (SLIDE)](https://mlsanity.com/paper/2502.06692/) (2025)
- [What Language is This? Ask Your Tokenizer](https://mlsanity.com/paper/2602.17655/) (2026)
- [Detecting Structured Language Alternations in Historical Documents by Combining Language Identification with Fourier Analysis](https://mlsanity.com/paper/2401.14569/) (2024)
- [Explainability of machine learning approaches in forensic linguistics: a case study in geolinguistic authorship profiling](https://mlsanity.com/paper/2404.18510/) (2024)
- [Moly'e: A Corpus-based Approach to Language Contact in Colonial France](https://mlsanity.com/paper/2408.04554/) (2024)
- [Improving Informally Romanized Language Identification](https://mlsanity.com/paper/2504.21540/) (2025)
- [Language Identification with a Reciprocal Rank Classifier](https://mlsanity.com/paper/2109.09862/) (2021)

## Abstract (arXiv)

Language identification (LI) is the problem of determining the natural language that a document or part thereof is written in. Automatic LI has been extensively researched for over fifty years. Today, LI is a key part of many text processing pipelines, as text processing techniques generally assume that the language of the input text is known. Research in this area has recently been especially active. This article provides a brief history of LI research, and an extensive survey of the features and methods used so far in the LI literature. For describing the features and methods we introduce a unified notation. We discuss evaluation methods, applications of LI, as well as off-the-shelf LI systems that do not require training by the end user. Finally, we identify open issues, survey the work to date on each issue, and propose future directions for research in LI.