# Resource-Lean Lexicon Induction for German Dialects

arXiv: https://arxiv.org/abs/2604.23824 · HTML: https://mlsanity.com/paper/2604.23824/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 37 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [Dictionary-based Phrase-level Prompting of Large Language Models for Machine Translation](https://mlsanity.com/paper/2302.07856/) (2023)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [Expanding Pretrained Models to Thousands More Languages via Lexicon-based Adaptation](https://mlsanity.com/paper/2203.09435/) (2022)
- [Does Manipulating Tokenization Aid Cross-Lingual Transfer? A Study on POS Tagging for Non-Standardized Languages](https://mlsanity.com/paper/2304.10158/) (2023)
- [Unsupervised Cross-Lingual Information Retrieval using Monolingual Data Only](https://mlsanity.com/paper/1805.00879/) (2018)
- [Evaluating Pixel Language Models on Non-Standardized Languages](https://mlsanity.com/paper/2412.09084/) (2024)
- [Incorporating Bilingual Dictionaries for Low Resource Semi-Supervised Neural Machine Translation](https://mlsanity.com/paper/2004.02071/) (2020)
- [Cross-Dialect Information Retrieval: Information Access in Low-Resource and High-Variance Languages](https://mlsanity.com/paper/2412.12806/) (2024)
- [We're Calling an Intervention: Exploring Fundamental Hurdles in Adapting Language Models to Nonstandard Text](https://mlsanity.com/paper/2404.07304/) (2024)
- [On the Effects of Regional Spelling Conventions in Retrieval Models](https://mlsanity.com/paper/2308.00480/) (2023)
- [MessIRve: A Large-Scale Spanish Information Retrieval Dataset](https://mlsanity.com/paper/2409.05994/) (2024)
- [Make Every Letter Count: Building Dialect Variation Dictionaries from Monolingual Corpora](https://mlsanity.com/paper/2509.17855/) (2025)

## Abstract (arXiv)

Automatic induction of high-quality dictionaries is essential for building lexical resources, yet low-resource languages and dialects pose several challenges: limited access to annotators, high degree of spelling variations, and poor performance of large language models (LLMs). We empirically show that statistical models (random forests) trained on string similarity features are surprisingly effective for inducing German dialect lexicons. They outperform LLMs, enable cross-dialect transfer, and offer a lightweight data-driven alternative. We evaluate our models intrinsically on bilingual lexicon induction (BLI) and extrinsically on dialect information retrieval (IR). On BLI, random forests outperform Mistral-123b while being more resource-lean. On dialect IR with BM25, using our dialect dictionaries for query expansion yields relative improvements of up to 28.9% in nDCG@10 and 50.7% in Recall@100. Motivated by the resource scarcity in dialects, we further investigate the extent to which models transfer across different German dialects, and their performance under varying amounts of training data.