# NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data

arXiv: https://arxiv.org/abs/2507.00534 · HTML: https://mlsanity.com/paper/2507.00534/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 1 Jul 2025.

## Builds on (most cited first, 30 of 37)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Progressive Neural Networks](https://mlsanity.com/paper/1606.04671/) (2016)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Experience Replay for Continual Learning](https://mlsanity.com/paper/1811.11682/) (2018)
- [Efficient Lifelong Learning with A-GEM](https://mlsanity.com/paper/1812.00420/) (2018)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Learning multiple visual domains with residual adapters](https://mlsanity.com/paper/1705.08045/) (2017)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [CORe50: a New Dataset and Benchmark for Continuous Object Recognition](https://mlsanity.com/paper/1705.03550/) (2017)
- [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages](https://mlsanity.com/paper/2303.01037/) (2023)
- [Overcoming Catastrophic Forgetting in Graph Neural Networks](https://mlsanity.com/paper/2012.06002/) (2020)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [A Wholistic View of Continual Learning with Deep Neural Networks: Forgotten Lessons and the Bridge to Active and Open World Learning](https://mlsanity.com/paper/2009.01797/) (2020)
- [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021)
- [IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languages](https://mlsanity.com/paper/2403.01926/) (2024)
- [IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages](https://mlsanity.com/paper/2208.11761/) (2022)
- [Learn Continually, Generalize Rapidly: Lifelong Knowledge Accumulation for Few-shot Learning](https://mlsanity.com/paper/2104.08808/) (2021)
- [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2312.17279/) (2023)
- [Pseudo-Labeling for Massively Multilingual Speech Recognition](https://mlsanity.com/paper/2111.00161/) (2021)
- [Towards Lifelong Learning of End-to-end ASR](https://mlsanity.com/paper/2104.01616/) (2021)
- [Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR](https://mlsanity.com/paper/2305.15386/) (2023)
- [Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Training](https://mlsanity.com/paper/2109.02284/) (2021)
- [CL-MASR: A Continual Learning Benchmark for Multilingual ASR](https://mlsanity.com/paper/2310.16931/) (2023)
- [Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages](https://mlsanity.com/paper/2208.12666/) (2022)
- [Svarah: Evaluating English ASR Systems on Indian Accents](https://mlsanity.com/paper/2305.15760/) (2023)

## Built on by (most cited first, 2 of 2)

- [Navigating the Reality Gap: On-Device Continual Adaptation of ASR for Clinical Telephony](https://mlsanity.com/paper/2512.16401/) (2025)
- [Adapting Text LLMs to Speech via Multimodal Depth Up-Scaling](https://mlsanity.com/paper/2604.00489/) (2026)

## Abstract (arXiv)

We introduce Nirantar, a comprehensive framework for evaluating continual learning (CL) in multilingual and multi-domain ASR. Designed to reflect real-world CL challenges, Nirantar leverages data collected incrementally across 22 languages and 208 districts in India through natural episodes. This enables evaluation across Language-Incremental (LIL), Domain-Incremental (DIL), and the novel Language-Incremental Domain-Incremental Learning (LIDIL) scenarios. Unlike prior work that relies on simulated episodes, Nirantar presents dynamic, non-uniform language and domain shifts, making it an ideal testbed for CL research. With 3250 hours of human-transcribed speech, including 1720 hours newly introduced in this work, our framework enables systematic benchmarking of CL methods. We evaluate existing approaches and demonstrate that no single method performs consistently well, underscoring the need for more robust CL strategies.