# Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Model

arXiv: https://arxiv.org/abs/1712.01541 · HTML: https://mlsanity.com/paper/1712.01541/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2017.

## Built on by (most cited first, 30 of 37)

- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [Scaling End-to-End Models for Large-Scale Multilingual ASR](https://mlsanity.com/paper/2104.14830/) (2021)
- [LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR](https://mlsanity.com/paper/2406.06619/) (2024)
- [Can Whisper perform speech-based in-context learning?](https://mlsanity.com/paper/2309.07081/) (2023)
- [Transformer Based Deliberation for Two-Pass Speech Recognition](https://mlsanity.com/paper/2101.11577/) (2021)
- [A Configurable Multilingual Model is All You Need to Recognize All Languages](https://mlsanity.com/paper/2107.05876/) (2021)
- [SpeechMoE2: Mixture-of-Experts Model with Improved Routing](https://mlsanity.com/paper/2111.11831/) (2021)
- [Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities](https://mlsanity.com/paper/2211.05756/) (2022)
- [Accented Speech Recognition: A Survey](https://mlsanity.com/paper/2104.10747/) (2021)
- [Improving Fairness and Robustness in End-to-End Speech Recognition through unsupervised clustering](https://mlsanity.com/paper/2306.06083/) (2023)
- [SQuId: Measuring Speech Naturalness in Many Languages](https://mlsanity.com/paper/2210.06324/) (2022)
- [A Language Agnostic Multilingual Streaming On-Device ASR System](https://mlsanity.com/paper/2208.13916/) (2022)
- [Distilling a Pretrained Language Model to a Multilingual ASR Model](https://mlsanity.com/paper/2206.12638/) (2022)
- [AfriNames: Most ASR models "butcher" African Names](https://mlsanity.com/paper/2306.00253/) (2023)
- [Accented Speech Recognition With Accent-specific Codebooks](https://mlsanity.com/paper/2310.15970/) (2023)
- [Learning Cross-lingual Visual Speech Representations](https://mlsanity.com/paper/2303.09455/) (2023)
- [Towards dialect-inclusive recognition in a low-resource language: are balanced corpora the answer?](https://mlsanity.com/paper/2307.07295/) (2023)
- [Mixture of LoRA Experts for Low-Resourced Multi-Accent Automatic Speech Recognition](https://mlsanity.com/paper/2505.20006/) (2025)
- [Multilingual Speech Recognition using Knowledge Transfer across Learning Processes](https://mlsanity.com/paper/2110.07909/) (2021)
- [Flexi-Transducer: Optimizing Latency, Accuracy and Compute forMulti-Domain On-Device Scenarios](https://mlsanity.com/paper/2104.02232/) (2021)
- [A Highly Adaptive Acoustic Model for Accurate Multi-Dialect Speech Recognition](https://mlsanity.com/paper/2205.03027/) (2022)
- [Multi-pass Training and Cross-information Fusion for Low-resource End-to-end Accented Speech Recognition](https://mlsanity.com/paper/2306.11309/) (2023)
- [AccentFold: A Journey through African Accents for Zero-Shot ASR Adaptation to Target Accents](https://mlsanity.com/paper/2402.01152/) (2024)
- [Low-resource speech recognition and dialect identification of Irish in a multi-task framework](https://mlsanity.com/paper/2405.01293/) (2024)
- [Rapid Language Adaptation for Multilingual E2E Speech Recognition Using Encoder Prompting](https://mlsanity.com/paper/2406.12611/) (2024)
- [Transducer-based language embedding for spoken language identification](https://mlsanity.com/paper/2204.03888/) (2022)
- [Improving Multilingual ASR in the Wild Using Simple N-best Re-ranking](https://mlsanity.com/paper/2409.18428/) (2024)
- [Integrating Categorical Features in End-to-End ASR](https://mlsanity.com/paper/2110.03047/) (2021)
- [Generative linguistic representation for spoken language identification](https://mlsanity.com/paper/2312.10964/) (2023)
- [Where Are You From? Let Me Guess! Subdialect Recognition of Speeches in Sorani Kurdish](https://mlsanity.com/paper/2404.00124/) (2024)

## Abstract (arXiv)

Sequence-to-sequence models provide a simple and elegant solution for building speech recognition systems by folding separate components of a typical system, namely acoustic (AM), pronunciation (PM) and language (LM) models into a single neural network. In this work, we look at one such sequence-to-sequence model, namely listen, attend and spell (LAS), and explore the possibility of training a single model to serve different English dialects, which simplifies the process of training multi-dialect systems without the need for separate AM, PM and LMs for each dialect. We show that simply pooling the data from all dialects into one LAS model falls behind the performance of a model fine-tuned on each dialect. We then look at incorporating dialect-specific information into the model, both by modifying the training targets by inserting the dialect symbol at the end of the original grapheme sequence and also feeding a 1-hot representation of the dialect information into all layers of the model. Experimental results on seven English dialects show that our proposed system is effective in modeling dialect variations within a single LAS model, outperforming a LAS model trained individually on each of the seven dialects by 3.1 ~ 16.5% relative.