# Svarah: Evaluating English ASR Systems on Indian Accents

arXiv: https://arxiv.org/abs/2305.15760 · HTML: https://mlsanity.com/paper/2305.15760/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 15 references that are in the map.
- First posted on arXiv 25 May 2023.

## Builds on (most cited first, 8 of 8)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages](https://mlsanity.com/paper/2208.11761/) (2022)
- [AccentDB: A Database of Non-Native English Accents to Assist Neural Speech Recognition](https://mlsanity.com/paper/2005.07973/) (2020)
- [SMS-WSJ: Database, performance measures, and baseline recipe for multi-channel source separation and recognition](https://mlsanity.com/paper/1910.13934/) (2019)

## Built on by (most cited first, 16 of 16)

- [IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages](https://mlsanity.com/paper/2403.01926/) (2024)
- [Distilling an End-to-End Voice Assistant Without Instruction Training Data](https://mlsanity.com/paper/2410.02678/) (2024)
- [LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems](https://mlsanity.com/paper/2408.11440/) (2024)
- [SlideAVSR: A Dataset of Paper Explanation Videos for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2401.09759/) (2024)
- [Do LLM Decoders Listen Fairly? Benchmarking How Language Model Priors Shape Bias in Speech Recognition](https://mlsanity.com/paper/2604.21276/) (2026)
- [ASR Under the Stethoscope: Evaluating Biases in Clinical Speech Recognition across Indian Languages](https://mlsanity.com/paper/2512.10967/) (2025)
- [What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels](https://mlsanity.com/paper/2512.16832/) (2025)
- [NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data](https://mlsanity.com/paper/2507.00534/) (2025)
- [Task-Lens: Cross-Task Utility Based Speech Dataset Profiling for Low-Resource Indian Languages](https://mlsanity.com/paper/2602.23388/) (2026)
- [Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India](https://mlsanity.com/paper/2604.19151/) (2026)
- [MoDiCoL: A Modular Diagnostic Continual Learning Dataset for Robust Speech Recognition](https://mlsanity.com/paper/2606.14459/) (2026)
- [SamaVaani: Auditing and Debiasing Multilingual Clinical ASR for Indian Languages](https://mlsanity.com/paper/2606.26901/) (2026)
- [Position: AI Leaderboards Are Underserving the Global South: A Case Study from India](https://mlsanity.com/paper/2608.18117/) (2026)
- [Vimarsha: Faithful ASR Evaluation for Indian Languages with Demographic Diversity, In-the-Wild Audio and Spelling Variations](https://mlsanity.com/paper/2609.24199/) (2026)
- [Fairness Beyond a Single Run: Training-Seed Variability in Speech LLM Adaptation](https://mlsanity.com/paper/2609.38976/) (2026)
- [Anuvani-LID: Robust and Calibrated Spoken Language Identification for Indian Languages](https://mlsanity.com/paper/2610.20263/) (2026)

## Abstract (arXiv)

India is the second largest English-speaking country in the world with a speaker base of roughly 130 million. Thus, it is imperative that automatic speech recognition (ASR) systems for English should be evaluated on Indian accents. Unfortunately, Indian speakers find a very poor representation in existing English ASR benchmarks such as LibriSpeech, Switchboard, Speech Accent Archive, etc. In this work, we address this gap by creating Svarah, a benchmark that contains 9.6 hours of transcribed English audio from 117 speakers across 65 geographic locations throughout India, resulting in a diverse range of accents. Svarah comprises both read speech and spontaneous conversational data, covering various domains, such as history, culture, tourism, etc., ensuring a diverse vocabulary. We evaluate 6 open source ASR models and 2 commercial ASR systems on Svarah and show that there is clear scope for improvement on Indian accents. Svarah as well as all our code will be publicly available.