mlsanityData to 13 Oct 2026

SpeechBrain: A General-Purpose Speech Toolkit

arXiv 2106.04624 ↗First posted 8 Jun 2021eess.AS cs.AI cs.LGcited by 404

Not on the map. Why?

Abstract (arXiv)

SpeechBrain is an open-source and all-in-one speech toolkit. It is designed to facilitate the research and development of neural speech processing technologies by being simple, flexible, user-friendly, and well-documented. This paper describes the core architecture designed to support several tasks of common interest, allowing users to naturally conceive, compare and share novel speech processing pipelines. SpeechBrain achieves competitive or state-of-the-art performance in a wide range of speech benchmarks. It also provides training recipes, pretrained models, and inference scripts for popular speech datasets, as well as tutorials which allow anyone with basic Python proficiency to familiarize themselves with speech technologies.

Key facts. Cited by 404 papers in the map (data through 13 Oct 2026). Builds on 65 of its 113 references that are in the map. First posted on arXiv 8 Jun 2021.

Builds on

65 of 113 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,106
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,539
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,870
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,352
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,979
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,487
  8. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,366
  9. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,359
  10. 2018VoxCeleb2: Deep Speaker Recognitioncited by 864
  11. 2017VoxCeleb: a large-scale speaker identification datasetcited by 719
  12. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 657
38 more
  1. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 612
  2. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 555
  3. 2017Convolutional Sequence to Sequence Learningcited by 552
  4. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 464
  5. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 456
  6. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 402
  7. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 366
  8. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  9. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 232
  10. 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
  11. 2017Deep Complex Networkscited by 217
  12. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 212
  13. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  14. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  15. 2019Speech Model Pre-training for End-to-End Spoken Language Understandingcited by 192
  16. 2020Real Time Speech Enhancement in the Waveform Domaincited by 184
  17. 2020Attention is All You Need in Speech Separationcited by 184
  18. 2020fastai: A Layered API for Deep Learningcited by 183
  19. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  20. 2019pyannote.audio: neural building blocks for speaker diarizationcited by 178
  21. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 155
  22. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 155
  23. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 129
  24. 2021MetricGAN+: An Improved Version of MetricGAN for Speech Enhancementcited by 114
  25. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 96
  26. 2019QuartzNet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutionscited by 93
  27. 2018Towards end-to-end spoken language understandingcited by 92
  28. 2019Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modelingcited by 86
  29. 2019Jasper: An End-to-End Convolutional Neural Acoustic Modelcited by 75
  30. 2020Rethinking Evaluation in ASR: Are Our Models Robust Enough?cited by 72
  31. 2020Bayesian HMM clustering of x-vector sequences (VBx) in speaker diarization: theory, implementation and analysis on standard taskscited by 69
  32. 2018A Comparison of Techniques for Language Model Integration in Encoder-Decoder Speech Recognitioncited by 66
  33. 2018From Audio to Semantics: Approaches to end-to-end spoken language understandingcited by 63
  34. 2019WHAMR!: Noisy and Reverberant Single-Channel Speech Separationcited by 61
  35. 2019BUT System Description to VoxCeleb Speaker Recognition Challenge 2019cited by 51
  36. 2018Light Gated Recurrent Units for Speech Recognitioncited by 48
  37. 2021ECAPA-TDNN Embeddings for Speaker Diarizationcited by 47
  38. 2020Asteroid: the PyTorch-based audio source separation toolkit for researcherscited by 45

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 13; 2022: 99; 2023: 93; 2024: 88; 2025: 49; 2026: 62.

404 papers.

Sort:

By area:

  1. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 594
  2. 2023Scaling Speech Technology to 1,000+ Languagescited by 402
  3. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 294
  4. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  5. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  6. 2022SoundSpaces 2.0: A Simulation Platform for Visual-Acoustic Learningcited by 93
  7. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
  8. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  9. 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 77
  10. 2024mHuBERT-147: A Compact Multilingual HuBERT Modelcited by 64
  11. 2023Efficient Sequence Transduction by Jointly Predicting Tokens and Durationscited by 56
  12. 2024Open-Source Conversational AI with SpeechBrain 1.0cited by 55
38 more
  1. 2021ESPnet-SLU: Advancing Spoken Language Understanding through ESPnetcited by 53
  2. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 53
  3. 2022Pushing the limits of raw waveform speaker recognitioncited by 50
  4. 2022Transformer-based Multimodal Information Fusion for Facial Expression Analysiscited by 49
  5. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  6. 2022WeNet 2.0: More Productive End-to-End Speech Recognition Toolkitcited by 46
  7. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  8. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separationcited by 45
  9. 2024The VoicePrivacy 2024 Challenge Evaluation Plancited by 39
  10. 2022An efficient encoder-decoder architecture with top-down attention for speech separationcited by 38
  11. 2023MARBLE: Music Audio Representation Benchmark for Universal Evaluationcited by 38
  12. 2025M-Prometheus: A Suite of Open Multilingual LLM Judgescited by 37
  13. 2023The Edinburgh International Accents of English Corpus: Towards the Democratization of English ASRcited by 36
  14. 2021Fine-tuning wav2vec2 for speaker recognitioncited by 35
  15. 2023AfriSpeech-200: Pan-African Accented Speech Dataset for Clinical and General Domain ASRcited by 35
  16. 2024DASB - Discrete Audio and Speech Benchmarkcited by 32
  17. 2023Speech Self-Supervised Representation Benchmarking: Are We Doing it Right?cited by 28
  18. 2023My Science Tutor (MyST) -- A Large Corpus of Children's Conversational Speechcited by 28
  19. 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 27
  20. 2024FakingRecipe: Detecting Fake News on Short Video Platforms from the Perspective of Creative Processcited by 25
  21. 2024Twists, Humps, and Pebbles: Multilingual Speech Recognition Models Exhibit Gender Performance Gapscited by 24
  22. 2024How Should We Extract Discrete Audio Tokens from Self-Supervised Models?cited by 24
  23. 2022TVLT: Textless Vision-Language Transformercited by 22
  24. 2024Benchmarking Large Multimodal Models against Common Corruptionscited by 22
  25. 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
  26. 2022Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Featurescited by 20
  27. 2022Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacycited by 20
  28. 2023CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voicecited by 20
  29. 2023AV-RIR: Audio-Visual Room Impulse Response Estimationcited by 20
  30. 2024SafeEar: Content Privacy-Preserving Audio Deepfake Detectioncited by 20
  31. 2022Listen, Adapt, Better WER: Source-free Single-utterance Test-time Adaptation for Automatic Speech Recognitioncited by 19
  32. 2021MetricGAN-U: Unsupervised speech enhancement/ dereverberation based only on noisy/ reverberated speechcited by 18
  33. 2022Words are all you need? Language as an approximation for human similarity judgmentscited by 18
  34. 2023What Can an Accent Identifier Learn? Probing Phonetic and Prosodic Information in a Wav2vec2-based Accent Identification Modelcited by 18
  35. 2025FocalCodec: Low-Bitrate Speech Coding via Focal Modulation Networkscited by 18
  36. 2022Dubbing in Practice: A Large Scale Study of Human Localization With Insights for Automatic Dubbingcited by 17
  37. 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 17
  38. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 17

50 most cited of 404 All 404 (JSON)