# TorchAudio 2.1: Advancing speech recognition, self-supervised learning, and audio processing components for PyTorch

arXiv: https://arxiv.org/abs/2310.17864 · HTML: https://mlsanity.com/paper/2310.17864/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2023.

## Built on by (most cited first, 27 of 27)

- [Open-Source Conversational AI with SpeechBrain 1.0](https://mlsanity.com/paper/2407.00463/) (2024)
- [GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement](https://mlsanity.com/paper/2406.11546/) (2024)
- [Less Peaky and More Accurate CTC Forced Alignment by Label Priors](https://mlsanity.com/paper/2406.02560/) (2024)
- [Data-Juicer Sandbox: A Feedback-Driven Suite for Multimodal Data-Model Co-development](https://mlsanity.com/paper/2407.11784/) (2024)
- [SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervision](https://mlsanity.com/paper/2512.20308/) (2025)
- [Flow-SLM: Joint Learning of Linguistic and Acoustic Information for Spoken Language Modeling](https://mlsanity.com/paper/2508.09350/) (2025)
- [ProGRes: Prompted Generative Rescoring on ASR n-Best](https://mlsanity.com/paper/2409.00217/) (2024)
- [SLIDE: Integrating Speech Language Model with LLM for Spontaneous Spoken Dialogue Generation](https://mlsanity.com/paper/2501.00805/) (2025)
- [Lotus: Creating Short Videos From Long Videos With Abstractive and Extractive Summarization](https://mlsanity.com/paper/2502.07096/) (2025)
- [SKILL: Similarity-aware Knowledge distILLation for Speech Self-Supervised Learning](https://mlsanity.com/paper/2402.16830/) (2024)
- [Vibravox: A Dataset of French Speech Captured with Body-conduction Audio Sensors](https://mlsanity.com/paper/2407.11828/) (2024)
- [BabyHuBERT: Multilingual Self-Supervised Learning for Segmenting Speakers in Child-Centered Long-Form Recordings](https://mlsanity.com/paper/2509.15001/) (2025)
- [Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction](https://mlsanity.com/paper/2312.15616/) (2023)
- [Audio Codec Augmentation for Robust Collaborative Watermarking of Speech Synthesis](https://mlsanity.com/paper/2409.13382/) (2024)
- [Open-Amp: Synthetic Data Framework for Audio Effect Foundation Models](https://mlsanity.com/paper/2411.14972/) (2024)
- [SleepNetZero: Zero-Burden Zero-Shot Reliable Sleep Staging With Neural Networks Based on Ballistocardiograms](https://mlsanity.com/paper/2410.22646/) (2024)
- [From Large-scale Audio Tagging to Real-Time Explainable Emergency Vehicle Sirens Detection](https://mlsanity.com/paper/2506.23437/) (2025)
- [SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Models](https://mlsanity.com/paper/2406.08445/) (2024)
- [Reproducible Machine Learning-based Voice Pathology Detection: Introducing the Pitch Difference Feature](https://mlsanity.com/paper/2410.10537/) (2024)
- [Inverse Reconstruction of Shock Time Series from Shock Response Spectrum Curves using Machine Learning](https://mlsanity.com/paper/2603.03229/) (2026)
- [Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs](https://mlsanity.com/paper/2607.06831/) (2026)
- [Quantifying the Generation Modality Gap in Speech-Text Language Models](https://mlsanity.com/paper/2609.14743/) (2026)
- [Objective and subjective evaluation of speech enhancement methods in the UDASE task of the 7th CHiME challenge](https://mlsanity.com/paper/2402.01413/) (2024)
- [Coherence in the brain unfolds across separable temporal regimes](https://mlsanity.com/paper/2512.20481/) (2025)
- [A Toolkit for Detecting Spurious Correlations in Speech Datasets](https://mlsanity.com/paper/2604.26676/) (2026)
- [Distilling Text LMs into Speech-to-Speech LMs by Matching Internal Text Predictions](https://mlsanity.com/paper/2610.17461/) (2026)
- [Where Words Begin and End](https://mlsanity.com/paper/2610.17782/) (2026)

## Abstract (arXiv)

TorchAudio is an open-source audio and speech processing library built for PyTorch. It aims to accelerate the research and development of audio and speech technologies by providing well-designed, easy-to-use, and performant PyTorch components. Its contributors routinely engage with users to understand their needs and fulfill them by developing impactful features. Here, we survey TorchAudio's development principles and contents and highlight key features we include in its latest version (2.1): self-supervised learning pre-trained pipelines and training recipes, high-performance CTC decoders, speech recognition models and training recipes, advanced media I/O capabilities, and tools for performing forced alignment, multi-channel speech enhancement, and reference-less speech assessment. For a selection of these features, through empirical studies, we demonstrate their efficacy and show that they achieve competitive or state-of-the-art performance.