TorchAudio: Building Blocks for Audio and Speech Processing
Not on the map. Why?
Abstract (arXiv)
This document describes version 0.10 of TorchAudio: building blocks for machine learning applications in the audio and speech processing domain. The objective of TorchAudio is to accelerate the development and deployment of machine learning applications for researchers and engineers by providing off-the-shelf building blocks. The building blocks are designed to be GPU-compatible, automatically differentiable, and production-ready. TorchAudio can be easily installed from Python Package Index repository and the source code is publicly available under a BSD-2-Clause License (as of September 2021) at https://github.com/pytorch/audio. In this document, we provide an overview of the design principles, functionalities, and benchmarks of TorchAudio. We also benchmark our implementation of several audio and speech operations and models. We verify through the benchmarks that our implementations of various operations and models are valid and perform similarly to other publicly available implementations.
Key facts. Cited by 83 papers in the map (data through 13 Oct 2026). First posted on arXiv 28 Oct 2021.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 1; 2022: 19; 2023: 25; 2024: 26; 2025: 5; 2026: 7.
83 papers.
Sort:
By area:
- 2023Scaling Speech Technology to 1,000+ Languagescited by 402
- 2022Decoding speech perception from non-invasive brain recordingscited by 129
- 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 75
- 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 74
- 2021TorchGeo: Deep Learning With Geospatial Datacited by 73
- 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
- 2022Perfectly Secure Steganography Using Minimum Entropy Couplingcited by 44
- 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 40
- 2022AugLy: Data Augmentations for Robustnesscited by 34
- 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 33
- 2023Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentationcited by 33
- 2023DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Modelscited by 33
38 more
- 2024TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learningcited by 29
- 2022The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editingcited by 24
- 2023Automatic Piano Transcription with Hierarchical Frequency-Time Transformercited by 22
- 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillationcited by 22
- 2023Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Computecited by 21
- 2022Words are all you need? Language as an approximation for human similarity judgmentscited by 18
- 2022Accelerating RNN-T Training and Inference Using CTC guidancecited by 17
- 2024REACT 2024: the Second Multiple Appropriate Facial Reaction Generation Challengecited by 16
- 2023Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editingcited by 15
- 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
- 2023Allophant: Cross-lingual Phoneme Recognition with Articulatory Attributescited by 13
- 2022Music Enhancement via Image Translation and Vocodingcited by 11
- 2023Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio Learnerscited by 11
- 2023ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkitcited by 10
- 2023AnuraSet: A dataset for benchmarking Neotropical anuran calls identification in passive acoustic monitoringcited by 10
- 2024AudioProtoPNet: An interpretable deep learning model for bird sound classificationcited by 10
- 2022GANStrument: Adversarial Instrument Sound Synthesis with Pitch-invariant Instance Conditioningcited by 9
- 2025AV-Flow: Transforming Text to Audio-Visual Human-like Interactionscited by 9
- 2022ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understandingcited by 8
- 2024DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Modelscited by 8
- 2023Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Featurescited by 7
- 2023R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Piecescited by 7
- 2022Jointist: Joint Learning for Multi-instrument Transcription and Its Applicationscited by 6
- 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognitioncited by 6
- 2023Automatic Disfluency Detection from Untranscribed Speechcited by 6
- 2023REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challengecited by 5
- 2023AV-CPL: Continuous Pseudo-Labeling for Audio-Visual Speech Recognitioncited by 5
- 2024Weakly-supervised Audio Separation via Bi-modal Semantic Similaritycited by 5
- 2024Learn from Real: Reality Defender's Submission to ASVspoof5 Challengecited by 5
- 2022Continual Learning for On-Device Speech Recognition using Disentangled Conformerscited by 4
- 2022Efficient Speech Representation Learning with Low-Bit Quantizationcited by 4
- 2023Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Predictioncited by 4
- 2024Whisper Finetuning on Nepali Languagecited by 4
- 2024MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyondcited by 4
- 2025The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary studycited by 4
- 2026Zero-shot Forecasting by Simulation Alonecited by 4
- 2022Generating Diverse Vocal Bursts with StyleGAN2 and MEL-Spectrogramscited by 3
- 2022Blank Collapse: Compressing CTC emission for the faster decodingcited by 3
50 most cited of 83 All 83 (JSON)
- 2026FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech26 Sep 2026
- 2026Scaling Forced Alignment to End-User Devices17 Sep 2026
- 2026Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs7 Jul 2026
- 2026DeFRiS: Silo-Cooperative IoT Applications Scheduling via Decentralized Federated Reinforcement Learning16 Mar 2026
- 2026ReHear: Iterative Pseudo-Label Refinement for Semi-Supervised Speech Recognition via Audio Large Language Models21 Feb 2026
- 2026Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representations28 Jan 2026
- 2026Zero-shot Forecasting by Simulation Alone2 Jan 2026
- 2025Geospatial Machine Learning Libraries2 Oct 2025
- 2025uPVC-Net: A Universal Premature Ventricular Contraction Detection Deep Learning Algorithm12 Jun 2025
- 2025The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary study19 May 2025
- 2025AV-Flow: Transforming Text to Audio-Visual Human-like Interactions18 Feb 2025
- 2025VoicePrompter: Robust Zero-Shot Voice Conversion with Voice Prompt and Conditional Flow Matching29 Jan 2025
38 more
- 2024MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyond16 Dec 2024
- 2024Representation Purification for End-to-End Speech Translation5 Dec 2024
- 2024Whisper Finetuning on Nepali Language19 Nov 2024
- 2024DC-Spin: A Speaker-invariant Speech Tokenizer for Spoken Language Models31 Oct 2024
- 2024SleepNetZero: Zero-Burden Zero-Shot Reliable Sleep Staging With Neural Networks Based on Ballistocardiograms30 Oct 2024
- 2024Joint Beamforming and Speaker-Attributed ASR for Real Distant-Microphone Meeting Transcription29 Oct 2024
- 2024Learn from Real: Reality Defender's Submission to ASVspoof5 Challenge9 Oct 2024
- 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration14 Sep 2024
- 2024Lightweight Transducer Based on Frame-Level Criterion5 Sep 2024
- 2024Direction of Arrival Correction through Speech Quality Feedback13 Aug 2024
- 2024Multimodal Input Aids a Bayesian Model of Phonetic Learning22 Jul 2024
- 2024An Automated Approach to Collecting and Labeling Time Series Data for Event Detection Using Elastic Node Hardware6 Jul 2024
- 2024WavRx: a Disease-Agnostic, Generalizable, and Privacy-Preserving Speech Health Diagnostic Model26 Jun 2024
- 2024TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning21 Jun 2024
- 2024LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks13 Jun 2024
- 2024SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Models12 Jun 2024
- 2024AdaKD: Dynamic Knowledge Distillation of ASR models using Adaptive Loss Weighting11 May 2024
- 2024Toward end-to-end interpretable convolutional neural networks for waveform signals3 May 2024
- 2024AudioProtoPNet: An interpretable deep learning model for bird sound classification16 Apr 2024
- 2024Wireless Earphone-based Real-Time Monitoring of Breathing Exercises: A Deep Learning Approach16 Apr 2024
- 2024A Large-Scale Evaluation of Speech Foundation Models15 Apr 2024
- 2024Weakly-supervised Audio Separation via Bi-modal Semantic Similarity2 Apr 2024
- 2024SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representations10 Mar 2024
- 2024Objective and subjective evaluation of speech enhancement methods in the UDASE task of the 7th CHiME challenge2 Feb 2024
- 2024Proactive Detection of Voice Cloning with Localized Watermarking30 Jan 2024
- 2024REACT 2024: the Second Multiple Appropriate Facial Reaction Generation Challenge10 Jan 2024
- 2023Uncertainty as a Predictor: Leveraging Self-Supervised Learning for Zero-Shot MOS Prediction25 Dec 2023
- 2023R-Spin: Efficient Speaker and Noise-invariant Representation Learning with Acoustic Pieces15 Nov 2023
- 2023Automatic Disfluency Detection from Untranscribed Speech1 Nov 2023
- 2023Loop Copilot: Conducting AI Ensembles for Music Generation and Iterative Editing19 Oct 2023
- 2023Sound-skwatter (Did You Mean: Sound-squatter?) AI-powered Generator for Phishing Prevention10 Oct 2023
- 2023Comparative Analysis of Transfer Learning in Deep Learning Text-to-Speech Models on a Few-Shot, Low-Resource, Customized Dataset8 Oct 2023
- 2023AV-CPL: Continuous Pseudo-Labeling for Audio-Visual Speech Recognition29 Sep 2023
- 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation19 Sep 2023
- 2023Explaining Speech Classification Models via Word-Level Audio Segments and Paralinguistic Features14 Sep 2023
- 2023CIF-T: A Novel CIF-based Transducer Architecture for Automatic Speech Recognition26 Jul 2023
- 2023AnuraSet: A dataset for benchmarking Neotropical anuran calls identification in passive acoustic monitoring11 Jul 2023
- 2023Automatic Piano Transcription with Hierarchical Frequency-Time Transformer10 Jul 2023
50 newest of 83 All 83 (JSON)
- 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillationcited by 22
- 2025AV-Flow: Transforming Text to Audio-Visual Human-like Interactionscited by 9
- 2025VoicePrompter: Robust Zero-Shot Voice Conversion with Voice Prompt and Conditional Flow Matchingcited by 3
- 2025AV-Flow: Transforming Text to Audio-Visual Human-like Interactions18 Feb 2025
- 2025VoicePrompter: Robust Zero-Shot Voice Conversion with Voice Prompt and Conditional Flow Matching29 Jan 2025
- 2023ConsistencyTTA: Accelerating Diffusion-Based Text-to-Audio Generation with Consistency Distillation19 Sep 2023
- 2023Scaling Speech Technology to 1,000+ Languagescited by 402
- 2022Decoding speech perception from non-invasive brain recordingscited by 129
- 2024Proactive Detection of Voice Cloning with Localized Watermarkingcited by 75
- 2022Imitator: Personalized Speech-driven 3D Facial Animationcited by 74
- 2021TorchGeo: Deep Learning With Geospatial Datacited by 73
- 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
- 2022Perfectly Secure Steganography Using Minimum Entropy Couplingcited by 44
- 2024A Large-Scale Evaluation of Speech Foundation Modelscited by 40
- 2022AugLy: Data Augmentations for Robustnesscited by 34
- 2022StyleTTS: A Style-Based Generative Model for Natural and Diverse Text-to-Speech Synthesiscited by 33
- 2023Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentationcited by 33
- 2023DPHuBERT: Joint Distillation and Pruning of Self-Supervised Speech Modelscited by 33
12 most cited of 77 All 77 (JSON)
- 2026FA-Bench: A Benchmark for Phone- and Word-Level Timestamp Accuracy in Forced Alignment and ASR on Clean and Noisy Speech26 Sep 2026
- 2026Scaling Forced Alignment to End-User Devices17 Sep 2026
- 2026Gradient-Based Speech-to-Text Alignment for Any ASR Model: From CTC to Speech LLMs7 Jul 2026
- 2026ReHear: Iterative Pseudo-Label Refinement for Semi-Supervised Speech Recognition via Audio Large Language Models21 Feb 2026
- 2026Position-invariant Fine-tuning of Speech Enhancement Models with Self-supervised Speech Representations28 Jan 2026
- 2026Zero-shot Forecasting by Simulation Alone2 Jan 2026
- 2025Geospatial Machine Learning Libraries2 Oct 2025
- 2025uPVC-Net: A Universal Premature Ventricular Contraction Detection Deep Learning Algorithm12 Jun 2025
- 2025The role of data partitioning on the performance of EEG-based deep learning models in supervised cross-subject analysis: a preliminary study19 May 2025
- 2024MERaLiON-SpeechEncoder: Towards a Speech Foundation Model for Singapore and Beyond16 Dec 2024
- 2024Representation Purification for End-to-End Speech Translation5 Dec 2024
- 2024Whisper Finetuning on Nepali Language19 Nov 2024
12 newest of 77 All 77 (JSON)