# Language Variety Identification with True Labels

arXiv: https://arxiv.org/abs/2303.01490 · HTML: https://mlsanity.com/paper/2303.01490/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 57 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Builds on (most cited first, 10 of 10)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Automatic Language Identification in Texts: A Survey](https://mlsanity.com/paper/1804.08186/) (2018)
- [Overview for the Second Shared Task on Language Identification in Code-Switched Data](https://mlsanity.com/paper/1909.13016/) (2019)
- [Lexical Simplification Benchmarks for English, Portuguese, and Spanish](https://mlsanity.com/paper/2209.05301/) (2022)
- [What's in a Domain? Learning Domain-Robust Text Representations using Adversarial Training](https://mlsanity.com/paper/1805.06088/) (2018)
- [Predicting Lexical Complexity in English Texts: The Complex 2.0 Dataset](https://mlsanity.com/paper/2102.08773/) (2021)
- [Discriminating Similar Languages: Evaluations and Explorations](https://mlsanity.com/paper/1610.00031/) (2016)
- [Language Model Adaptation for Language and Dialect Identification of Text](https://mlsanity.com/paper/1903.10915/) (2019)

## Built on by (most cited first, 12 of 12)

- [DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages](https://mlsanity.com/paper/2403.11009/) (2024)
- [Findings of the VarDial Evaluation Campaign 2023](https://mlsanity.com/paper/2305.20080/) (2023)
- [DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules](https://mlsanity.com/paper/2305.13406/) (2023)
- [Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification](https://mlsanity.com/paper/2310.13661/) (2023)
- [Enhancing Portuguese Variety Identification with Cross-Domain Approaches](https://mlsanity.com/paper/2502.14394/) (2025)
- [Two-stage Pipeline for Multilingual Dialect Detection](https://mlsanity.com/paper/2303.03487/) (2023)
- [Common Ground, Diverse Roots: The Difficulty of Classifying Common Examples in Spanish Varieties](https://mlsanity.com/paper/2412.11750/) (2024)
- [Multi-label Scandinavian Language Identification (SLIDE)](https://mlsanity.com/paper/2502.06692/) (2025)
- [Revisiting Common Assumptions about Arabic Dialects in NLP](https://mlsanity.com/paper/2505.21816/) (2025)
- [Lost in Variation? Evaluating NLI Performance in Basque and Spanish Geographical Variants](https://mlsanity.com/paper/2506.15239/) (2025)
- [IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation](https://mlsanity.com/paper/2604.04704/) (2026)
- [RoBERTweet: A BERT Language Model for Romanian Tweets](https://mlsanity.com/paper/2306.06598/) (2023)

## Abstract (arXiv)

Language identification is an important first step in many IR and NLP applications. Most publicly available language identification datasets, however, are compiled under the assumption that the gold label of each instance is determined by where texts are retrieved from. Research has shown that this is a problematic assumption, particularly in the case of very similar languages (e.g., Croatian and Serbian) and national language varieties (e.g., Brazilian and European Portuguese), where texts may contain no distinctive marker of the particular language or variety. To overcome this important limitation, this paper presents DSL True Labels (DSL-TL), the first human-annotated multilingual dataset for language variety identification. DSL-TL contains a total of 12,900 instances in Portuguese, split between European Portuguese and Brazilian Portuguese; Spanish, split between Argentine Spanish and Castilian Spanish; and English, split between American English and British English. We trained multiple models to discriminate between these language varieties, and we present the results in detail. The data and models presented in this paper provide a reliable benchmark toward the development of robust and fairer language variety identification systems. We make DSL-TL freely available to the research community.