# Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR

arXiv: https://arxiv.org/abs/2305.15386 · HTML: https://mlsanity.com/paper/2305.15386/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 12 of 12)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [NeMo: a toolkit for building AI applications using Neural Modules](https://mlsanity.com/paper/1909.09577/) (2019)
- [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021)
- [Towards Building ASR Systems for the Next Billion Users](https://mlsanity.com/paper/2111.03945/) (2021)
- [IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages](https://mlsanity.com/paper/2208.11761/) (2022)
- [Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages](https://mlsanity.com/paper/2208.12666/) (2022)
- [Automatic Speech Recognition in Sanskrit: A New Speech Corpus and Modelling Insights](https://mlsanity.com/paper/2106.05852/) (2021)
- [Subword Dictionary Learning and Segmentation Techniques for Automatic Speech Recognition in Tamil and Kannada](https://mlsanity.com/paper/2207.13331/) (2022)

## Built on by (most cited first, 21 of 21)

- [LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems](https://mlsanity.com/paper/2408.11440/) (2024)
- [Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling](https://mlsanity.com/paper/2408.14026/) (2024)
- [BERSting at the Screams: A Benchmark for Distanced, Emotional and Shouted Speech Recognition](https://mlsanity.com/paper/2505.00059/) (2025)
- [Towards Orthographically-Informed Evaluation of Speech Recognition Systems for Indian Languages](https://mlsanity.com/paper/2603.00941/) (2026)
- [To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation](https://mlsanity.com/paper/2406.04512/) (2024)
- [Whisper Finetuning on Nepali Language](https://mlsanity.com/paper/2411.12587/) (2024)
- [What Do Prosody and Text Convey? Characterizing How Meaningful Information is Distributed Across Multiple Channels](https://mlsanity.com/paper/2512.16832/) (2025)
- [Vedavani: A Benchmark Corpus for ASR on Vedic Sanskrit Poetry](https://mlsanity.com/paper/2506.00145/) (2025)
- [NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data](https://mlsanity.com/paper/2507.00534/) (2025)
- [Structured Sparsity and Weight-adaptive Pruning for Memory and Compute efficient Whisper models](https://mlsanity.com/paper/2510.12666/) (2025)
- [Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition](https://mlsanity.com/paper/2601.19451/) (2026)
- [Transsion TSUP's speech recognition system for ASRU 2023 MADASR Challenge](https://mlsanity.com/paper/2307.11778/) (2023)
- [Voice of India: A Large-Scale Benchmark for Real-World Speech Recognition in India](https://mlsanity.com/paper/2604.19151/) (2026)
- [The TTS-STT Flywheel: Synthetic Entity-Dense Audio Closes the Indic ASR Gap Where Commercial and Open-Source Systems Fail](https://mlsanity.com/paper/2605.03073/) (2026)
- [Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition](https://mlsanity.com/paper/2605.13087/) (2026)
- [SCRIBE: Diagnostic Evaluation and Rich Transcription Models for Indic ASR](https://mlsanity.com/paper/2605.20712/) (2026)
- [Position: AI Leaderboards Are Underserving the Global South: A Case Study from India](https://mlsanity.com/paper/2608.18117/) (2026)
- [BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models](https://mlsanity.com/paper/2609.09554/) (2026)
- [V=akQA: A Benchmark and Evaluation Study for Telugu Spoken Factoid Question Answering](https://mlsanity.com/paper/2609.19879/) (2026)
- [Vimarsha: Faithful ASR Evaluation for Indian Languages with Demographic Diversity, In-the-Wild Audio and Spelling Variations](https://mlsanity.com/paper/2609.24199/) (2026)
- [Harmonizing Spectral Evolution in Conditional Flow Matching for TTS](https://mlsanity.com/paper/2609.34431/) (2026)

## Abstract (arXiv)

Improving ASR systems is necessary to make new LLM-based use-cases accessible to people across the globe. In this paper, we focus on Indian languages, and make the case that diverse benchmarks are required to evaluate and improve ASR systems for Indian languages. To address this, we collate Vistaar as a set of 59 benchmarks across various language and domain combinations, on which we evaluate 3 publicly available ASR systems and 2 commercial systems. We also train IndicWhisper models by fine-tuning the Whisper models on publicly available training datasets across 12 Indian languages totalling to 10.7K hours. We show that IndicWhisper significantly improves on considered ASR systems on the Vistaar benchmark. Indeed, IndicWhisper has the lowest WER in 39 out of the 59 benchmarks, with an average reduction of 4.1 WER. We open-source all datasets, code and models.