# A Survey of Corpora for Germanic Low-Resource Languages and Dialects

arXiv: https://arxiv.org/abs/2304.09805 · HTML: https://mlsanity.com/paper/2304.09805/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 150 references that are in the map.
- First posted on arXiv 19 Apr 2023.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://mlsanity.com/paper/1911.00359/) (2019)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [Towards a Cleaner Document-Oriented Multilingual Crawled Corpus](https://mlsanity.com/paper/2201.06642/) (2022)
- [One Country, 700+ Languages: NLP Challenges for Underrepresented Languages and Dialects in Indonesia](https://mlsanity.com/paper/2203.13357/) (2022)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [From Masked Language Modeling to Translation: Non-English Auxiliary Tasks Improve Zero-shot Spoken Language Understanding](https://mlsanity.com/paper/2105.07316/) (2021)
- [Challenges of language technologies for the indigenous languages of the Americas](https://mlsanity.com/paper/1806.04291/) (2018)
- [What a Creole Wants, What a Creole Needs](https://mlsanity.com/paper/2206.00437/) (2022)
- [SwissDial: Parallel Multidialectal Corpus of Spoken Swiss German](https://mlsanity.com/paper/2103.11401/) (2021)
- [SDS-200: A Swiss German Speech to Standard German Text Corpus](https://mlsanity.com/paper/2205.09501/) (2022)
- [Arabic natural language processing: An overview](https://mlsanity.com/paper/1903.02784/) (2019)
- [The Danish Gigaword Project](https://mlsanity.com/paper/2005.03521/) (2020)
- [Swiss Parliaments Corpus, an Automatically Aligned Swiss German Speech to Standard German Text Corpus](https://mlsanity.com/paper/2010.02810/) (2020)
- [Does Corpus Quality Really Matter for Low-Resource Languages?](https://mlsanity.com/paper/2203.08111/) (2022)
- [The Norwegian Parliamentary Speech Corpus](https://mlsanity.com/paper/2201.10881/) (2022)
- [NorDial: A Preliminary Corpus of Written Norwegian Dialect Use](https://mlsanity.com/paper/2104.04989/) (2021)
- [Automatic Creation of Text Corpora for Low-Resource Languages from the Internet: The Case of Swiss German](https://mlsanity.com/paper/1912.00159/) (2019)
- [European Language Grid: An Overview](https://mlsanity.com/paper/2003.13551/) (2020)
- [Normalization of Different Swedish Dialects Spoken in Finland](https://mlsanity.com/paper/2012.05318/) (2020)
- [POS tagging, lemmatization and dependency parsing of West Frisian](https://mlsanity.com/paper/2107.07974/) (2021)
- [Annotating Norwegian Language Varieties on Twitter for Part-of-Speech](https://mlsanity.com/paper/2210.06150/) (2022)

## Built on by (most cited first, 16 of 16)

- [Aya Dataset: An Open-Access Collection for Multilingual Instruction Tuning](https://mlsanity.com/paper/2402.06619/) (2024)
- [RLHF Can Speak Many Languages: Unlocking Multilingual Preference Optimization for LLMs](https://mlsanity.com/paper/2407.02552/) (2024)
- [GlotLID: Language Identification for Low-Resource Languages](https://mlsanity.com/paper/2310.16248/) (2023)
- [What Do Dialect Speakers Want? A Survey of Attitudes Towards Language Technology for German Dialects](https://mlsanity.com/paper/2402.11968/) (2024)
- [Voices Unheard: NLP Resources and Models for Yor`ub'a Regional Dialects](https://mlsanity.com/paper/2406.19564/) (2024)
- [Sebastian, Basti, Wastl?! Recognizing Named Entities in Bavarian Dialectal Data](https://mlsanity.com/paper/2403.12749/) (2024)
- [MaiBaam: A Multi-Dialectal Bavarian Universal Dependency Treebank](https://mlsanity.com/paper/2403.10293/) (2024)
- [A Multi-Dialectal Dataset for German Dialect ASR and Dialect-to-Standard Speech Translation](https://mlsanity.com/paper/2506.02894/) (2025)
- [CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data](https://mlsanity.com/paper/2601.18026/) (2026)
- [Improving Dialectal Slot and Intent Detection with Auxiliary Tasks: A Multi-Dialectal Bavarian Case Study](https://mlsanity.com/paper/2501.03863/) (2025)
- [Make Every Letter Count: Building Dialect Variation Dictionaries from Monolingual Corpora](https://mlsanity.com/paper/2509.17855/) (2025)
- [AlbNews: A Corpus of Headlines for Topic Modeling in Albanian](https://mlsanity.com/paper/2402.04028/) (2024)
- [Indigenous Languages Spoken in Argentina: A Survey of NLP and Speech Resources](https://mlsanity.com/paper/2501.09943/) (2025)
- [Meenz bleibt Meenz, but Large Language Models Do Not Speak Its Dialect](https://mlsanity.com/paper/2602.16852/) (2026)
- [Beyond Subtokens: A Rich Character Embedding for Low-resource and Morphologically Complex Languages](https://mlsanity.com/paper/2602.21377/) (2026)
- [textitVersteasch du mi? Computational and Socio-Linguistic Perspectives on GenAI, LLMs, and Non-Standard Language](https://mlsanity.com/paper/2603.28213/) (2026)

## Abstract (arXiv)

Despite much progress in recent years, the vast majority of work in natural language processing (NLP) is on standard languages with many speakers. In this work, we instead focus on low-resource languages and in particular non-standardized low-resource languages. Even within branches of major language families, often considered well-researched, little is known about the extent and type of available resources and what the major NLP challenges are for these language varieties. The first step to address this situation is a systematic survey of available corpora (most importantly, annotated corpora, which are particularly valuable for NLP research). Focusing on Germanic low-resource language varieties, we provide such a survey in this paper. Except for geolocation (origin of speaker or document), we find that manually annotated linguistic resources are sparse and, if they exist, mostly cover morphosyntax. Despite this lack of resources, we observe that interest in this area is increasing: there is active development and a growing research community. To facilitate research, we make our overview of over 80 corpora publicly available. We share a companion website of this overview at https://github.com/mainlp/germanic-lrl-corpora .