# DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation

arXiv: https://arxiv.org/abs/2607.07669 · HTML: https://mlsanity.com/paper/2607.07669/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 45 references that are in the map.
- First posted on arXiv 8 Jul 2026.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://mlsanity.com/paper/2311.12022/) (2023)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://mlsanity.com/paper/2406.01574/) (2024)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://mlsanity.com/paper/2004.09095/) (2020)
- [GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection](https://mlsanity.com/paper/2403.03507/) (2024)
- [UltraFeedback: Boosting Language Models with Scaled AI Feedback](https://mlsanity.com/paper/2310.01377/) (2023)
- [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://mlsanity.com/paper/1608.08868/) (2016)
- [Linguistic Bias in ChatGPT: Language Models Reinforce Dialect Discrimination](https://mlsanity.com/paper/2406.08818/) (2024)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [DIALECTBENCH: A NLP Benchmark for Dialects, Varieties, and Closely-Related Languages](https://mlsanity.com/paper/2403.11009/) (2024)
- [VALUE: Understanding Dialect Disparity in NLU](https://mlsanity.com/paper/2204.03031/) (2022)
- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [Dialect prejudice predicts AI decisions about people's character, employability, and criminality](https://mlsanity.com/paper/2403.00742/) (2024)
- [Rejected Dialects: Biases Against African American Language in Reward Models](https://mlsanity.com/paper/2502.12858/) (2025)
- [TADA: Task-Agnostic Dialect Adapters for English](https://mlsanity.com/paper/2305.16651/) (2023)
- [Dynamic Multi-Reward Weighting for Multi-Style Controllable Generation](https://mlsanity.com/paper/2402.14146/) (2024)
- [Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models](https://mlsanity.com/paper/2409.00358/) (2024)
- [BESSTIE: A Benchmark for Sentiment and Sarcasm Classification for Varieties of English](https://mlsanity.com/paper/2412.04726/) (2024)
- [Low-Resource Dialect Adaptation of Large Language Models: A French Dialect Case-Study](https://mlsanity.com/paper/2510.22747/) (2025)

## Built on by (most cited first, 1 of 1)

- [Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness](https://mlsanity.com/paper/2608.05510/) (2026)

## Abstract (arXiv)

Large language models increasingly understand dialectal English, yet still produce only standard, US-leaning English, leaving dialectal generation, the harder half of the problem, largely unaddressed. We introduce DiaLLM, which continually pretrains three open-weight language model families on the International Corpus of English and applies implicit and explicit post-training paradigms, each combined with three model alignment strategies, giving the first controlled comparison of these components across Australian, Indian, and Northern British English. Our results reveal a robustness-generation gap: benchmarks are shaped by continual pretraining and SFT, while alignment visibly reshapes generation in ways benchmarks do not capture. Explicit variety-targeted adaptation produces output reliably recognised as dialectal and judged more dialectal than broad alignment, yet where human judgement was directly assessed, the method that most aggressively optimises the dialectal reward is not the one judged most dialectal. Independent linguistic analysis corroborates this reward-quality gap, most clearly on two of the three families. No single alignment method dominates, and closing the gap will require richer reward designs and continued investment in dialectal resources. We release all code, checkpoints, and preference datasets.