# LanideNN: Multilingual Language Identification on Character Window

arXiv: https://arxiv.org/abs/1701.03338 · HTML: https://mlsanity.com/paper/1701.03338/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jan 2017.

## Built on by (most cited first, 10 of 10)

- [GlotLID: Language Identification for Low-Resource Languages](https://mlsanity.com/paper/2310.16248/) (2023)
- [Comparing Approaches to Dravidian Language Identification](https://mlsanity.com/paper/2103.05552/) (2021)
- [MaskLID: Code-Switching Language Identification through Iterative Masking](https://mlsanity.com/paper/2406.06263/) (2024)
- [Geographically-Informed Language Identification](https://mlsanity.com/paper/2403.09892/) (2024)
- [OpenLID-v3: Improving the Precision of Closely Related Language Identification -- An Experience Report](https://mlsanity.com/paper/2602.13139/) (2026)
- [A reproduction of Apple's bi-directional LSTM models for language identification in short strings](https://mlsanity.com/paper/2102.06282/) (2021)
- [Language Identification for Austronesian Languages](https://mlsanity.com/paper/2206.04327/) (2022)
- [Code-Switched Language Identification is Harder Than You Think](https://mlsanity.com/paper/2402.01505/) (2024)
- [What Language is This? Ask Your Tokenizer](https://mlsanity.com/paper/2602.17655/) (2026)
- [Improving Language Identification for Code-Switched Utterances with Integer Linear Programming](https://mlsanity.com/paper/2609.05099/) (2026)

## Abstract (arXiv)

In language identification, a common first step in natural language processing, we want to automatically determine the language of some input text. Monolingual language identification assumes that the given document is written in one language. In multilingual language identification, the document is usually in two or three languages and we just want their names. We aim one step further and propose a method for textual language identification where languages can change arbitrarily and the goal is to identify the spans of each of the languages. Our method is based on Bidirectional Recurrent Neural Networks and it performs well in monolingual and multilingual language identification tasks on six datasets covering 131 languages. The method keeps the accuracy also for short documents and across domains, so it is ideal for off-the-shelf use without preparation of training data.