# TADA: Task-Agnostic Dialect Adapters for English

arXiv: https://arxiv.org/abs/2305.16651 · HTML: https://mlsanity.com/paper/2305.16651/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 56 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [MAD-X: An Adapter-Based Framework for Multi-Task Cross-Lingual Transfer](https://mlsanity.com/paper/2005.00052/) (2020)
- [WiC: the Word-in-Context Dataset for Evaluating Context-Sensitive Meaning Representations](https://mlsanity.com/paper/1808.09121/) (2018)
- [Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1604.02201/) (2016)
- [What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?](https://mlsanity.com/paper/2204.05832/) (2022)
- [A robust self-learning method for fully unsupervised cross-lingual mappings of word embeddings](https://mlsanity.com/paper/1805.06297/) (2018)
- [Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning](https://mlsanity.com/paper/2004.03829/) (2020)
- [Unsupervised Alignment of Embeddings with Wasserstein Procrustes](https://mlsanity.com/paper/1805.11222/) (2018)
- [Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection](https://mlsanity.com/paper/2201.10474/) (2022)
- [Multi-VALUE: A Framework for Cross-Dialectal English NLP](https://mlsanity.com/paper/2212.08011/) (2022)
- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation](https://mlsanity.com/paper/2110.03036/) (2021)
- [VALUE: Understanding Dialect Disparity in NLU](https://mlsanity.com/paper/2204.03031/) (2022)
- [Learning to Recognize Dialect Features](https://mlsanity.com/paper/2010.12707/) (2020)
- [ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - English](https://mlsanity.com/paper/2211.12000/) (2022)
- [Learning Crosslingual Word Embeddings without Bilingual Corpora](https://mlsanity.com/paper/1606.09403/) (2016)

## Built on by (most cited first, 15 of 15)

- [Natural Language Processing for Dialects of a Language: A Survey](https://mlsanity.com/paper/2401.05632/) (2024)
- [Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art](https://mlsanity.com/paper/2406.03930/) (2024)
- [Evaluating Dialect Robustness of Language Models via Conversation Understanding](https://mlsanity.com/paper/2405.05688/) (2024)
- [Task-Agnostic Low-Rank Adapters for Unseen English Dialects](https://mlsanity.com/paper/2311.00915/) (2023)
- [BERTwich: Extending BERT's Capabilities to Model Dialectal and Noisy Text](https://mlsanity.com/paper/2311.00116/) (2023)
- [We're Calling an Intervention: Exploring Fundamental Hurdles in Adapting Language Models to Nonstandard Text](https://mlsanity.com/paper/2404.07304/) (2024)
- [Predicting the Target Word of Game-playing Conversations using a Low-Rank Dialect Adapter for Decoder Models](https://mlsanity.com/paper/2409.00358/) (2024)
- [Exploring the Robustness of Task-oriented Dialogue Systems for Colloquial German Varieties](https://mlsanity.com/paper/2402.02078/) (2024)
- [A Multi-Agent Framework for Mitigating Dialect Biases in Privacy Policy Question-Answering Systems](https://mlsanity.com/paper/2506.02998/) (2025)
- [Harnessing Test-time Adaptation for NLU tasks Involving Dialects of English](https://mlsanity.com/paper/2503.12858/) (2025)
- [DiaLLM: An Investigation into the Robustness-Generation Gap in English Dialect Adaptation](https://mlsanity.com/paper/2607.07669/) (2026)
- [DialectLLM: A Dialect-Aware Dialog[ue] Generation Framework Beyond Standard American English](https://mlsanity.com/paper/2601.22888/) (2026)
- [AI systems and the reproduction of (standard) language ideologies in World Englishes](https://mlsanity.com/paper/2607.28528/) (2026)
- [Different Perturbations, Different Mechanisms: Understanding Continued Pre-training for Zero-Shot Dialect Robustness](https://mlsanity.com/paper/2608.05510/) (2026)
- [Not Safe for All: Auditing the Dialect Penalty in Text-to-Image Safety Pipelines](https://mlsanity.com/paper/2608.29589/) (2026)

## Abstract (arXiv)

Large Language Models, the dominant starting point for Natural Language Processing (NLP) applications, fail at a higher rate for speakers of English dialects other than Standard American English (SAE). Prior work addresses this using task-specific data or synthetic data augmentation, both of which require intervention for each dialect and task pair. This poses a scalability issue that prevents the broad adoption of robust dialectal English NLP. We introduce a simple yet effective method for task-agnostic dialect adaptation by aligning non-SAE dialects using adapters and composing them with task-specific adapters from SAE. Task-Agnostic Dialect Adapters (TADA) improve dialectal robustness on 4 dialectal variants of the GLUE benchmark without task-specific supervision.