mlsanityData to 7 Oct 2026

Recent Developments on ESPnet Toolkit Boosted by Conformer

arXiv 2010.13956 ↗First posted 26 Oct 2020eess.AS cs.SDcited by 132

Not on the map. Why?

Abstract (arXiv)

In this study, we present recent developments on ESPnet: End-to-End Speech Processing toolkit, which mainly involves a recently proposed architecture called Conformer, Convolution-augmented Transformer. This paper shows the results for a wide range of end-to-end speech processing applications, such as automatic speech recognition (ASR), speech translations (ST), speech separation (SS) and text-to-speech (TTS). Our experiments reveal various training tips and significant performance benefits obtained with the Conformer on different tasks. These results are competitive or even outperform the current state-of-art Transformer models. We are preparing to release all-in-one recipes using open source and publicly available corpora for all the above tasks with pre-trained models. Our aim for this work is to contribute to our research community by reducing the burden of preparing state-of-the-art research environments usually requiring high resources.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 32; 2022: 48; 2023: 38; 2024: 12; 2025: 2.

132 papers.

Sort:

By area:

  1. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  2. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 189
  3. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 138
  4. 2023End-to-End Speech Recognition: A Surveycited by 127
  5. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
  6. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
  7. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  8. 2021Translatotron 2: High-quality direct speech-to-speech translation with voice preservationcited by 69
  9. 2022Squeezeformer: An Efficient Transformer for Automatic Speech Recognitioncited by 50
  10. 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
  11. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  12. 2022A3T: Alignment-Aware Acoustic and Text Pretraining for Speech Synthesis and Editingcited by 40
38 more
  1. 2021An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognitioncited by 39
  2. 2022TranSpeech: Speech-to-Speech Translation With Bilateral Perturbationcited by 34
  3. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  4. 2023Improving Massively Multilingual ASR With Auxiliary CTC Objectivescited by 31
  5. 2022iSTFTNet: Fast and Lightweight Mel-Spectrogram Vocoder Incorporating Inverse Short-Time Fourier Transformcited by 30
  6. 2021A Comparative Study on Non-Autoregressive Modelings for Speech-to-Text Generationcited by 27
  7. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 26
  8. 2021Advanced Long-context End-to-end Speech Recognition Using Context-expanded Transformerscited by 25
  9. 2022CTC Alignments Improve Autoregressive Translationcited by 25
  10. 2023The Singing Voice Conversion Challenge 2023cited by 25
  11. 2021Searchable Hidden Intermediates for End-to-End Models of Decomposable Sequence Taskscited by 24
  12. 2023FluentSpeech: Stutter-Oriented Automatic Speech Editing with Context-Aware Diffusion Modelscited by 23
  13. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  14. 2023Audio-Visual Efficient Conformer for Robust Speech Recognitioncited by 22
  15. 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selectioncited by 19
  16. 2021Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorizationcited by 17
  17. 2022Towards Zero-Shot Code-Switched Speech Recognitioncited by 15
  18. 2021Dual Causal/Non-Causal Self-Attention for Streaming End-to-End Speech Recognitioncited by 14
  19. 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
  20. 2023AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representationcited by 14
  21. 2021Layer Pruning on Demand with Intermediate CTCcited by 13
  22. 2023A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Taskscited by 13
  23. 2023Visual Speech Recognition for Languages with Limited Labeled Data using Automatic Labels from Whispercited by 13
  24. 2022Conformer Based Elderly Speech Recognition System for Alzheimer's Disease Detectioncited by 12
  25. 2021Streaming End-to-End ASR based on Blockwise Non-Autoregressive Modelscited by 11
  26. 2022Combining Spectral and Self-Supervised Features for Low Resource Speech Recognition and Translationcited by 11
  27. 2022BERT Meets CTC: New Formulation of End-to-End Speech Recognition with Pre-trained Masked Language Modelcited by 11
  28. 2022Structured State Space Decoder for Speech Recognition and Synthesiscited by 11
  29. 2023SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understandingcited by 11
  30. 2022Boosting Cross-Domain Speech Recognition with Self-Supervisioncited by 10
  31. 2023ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkitcited by 10
  32. 2021Residual Energy-Based Models for End-to-End Speech Recognitioncited by 9
  33. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  34. 2021Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediatescited by 9
  35. 2023Speech collage: code-switched audio generation by collaging monolingual corporacited by 9
  36. 2022Conversational Speech Recognition By Learning Conversation-level Characteristicscited by 8
  37. 2022Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentationcited by 8
  38. 2022Two-Pass Low Latency End-to-End Spoken Language Understandingcited by 8

50 most cited of 132 All 132 (JSON)