mlsanityData to 7 Oct 2026

E-Branchformer: Branchformer with Enhanced merging for speech recognition

arXiv 2210.00077 ↗First posted 30 Sep 2022eess.AS cs.LGcited by 90

Not on the map. Why?

Abstract (arXiv)

Conformer, combining convolution and self-attention sequentially to capture both local and global information, has shown remarkable performance and is currently regarded as the state-of-the-art for automatic speech recognition (ASR). Several other studies have explored integrating convolution and self-attention but they have not managed to match Conformer's performance. The recently introduced Branchformer achieves comparable performance to Conformer by using dedicated branches of convolution and self-attention and merging local and global context from each branch. In this paper, we propose E-Branchformer, which enhances Branchformer by applying an effective merging method and stacking additional point-wise modules. E-Branchformer sets new state-of-the-art word error rates (WERs) 1.81% and 3.65% on LibriSpeech test-clean and test-other sets without using any external training data.

Key facts. Cited by 90 papers in the map (data through 7 Oct 2026). Builds on 51 of its 62 references that are in the map. First posted on arXiv 30 Sep 2022.

Builds on

51 of 62 references.

References · 51

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2016Layer Normalizationcited by 5,579
  3. 2017Squeeze-and-Excitation Networkscited by 4,763
  4. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  7. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  8. 2020Rethinking Attention with Performerscited by 1,584
  9. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  10. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  11. 2018Self-Attention with Relative Position Representationscited by 1,112
  12. 2021CvT: Introducing Convolutions to Vision Transformerscited by 946
38 more
  1. 2020Efficient Transformers: A Surveycited by 784
  2. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  3. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  4. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  5. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  6. 2014Deep Speech: Scaling up end-to-end speech recognitioncited by 354
  7. 2021W2v-BERT: Combining Contrastive Learning and Masked Language Modeling for Self-Supervised Speech Pre-Trainingcited by 340
  8. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
  9. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  10. 2019Pay Less Attention with Lightweight and Dynamic Convolutionscited by 211
  11. 2019A Comparative Study on Transformer vs RNN in Speech Applicationscited by 210
  12. 2020SLURP: A Spoken Language Understanding Resource Packagecited by 197
  13. 2019The Evolved Transformercited by 180
  14. 2022DeepNet: Scaling Transformers to 1,000 Layerscited by 176
  15. 2018QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehensioncited by 164
  16. 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
  17. 2020Recent Developments on ESPnet Toolkit Boosted by Conformercited by 132
  18. 2020Lite Transformer with Long-Short Range Attentioncited by 127
  19. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  20. 2022ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyondcited by 117
  21. 2021An Attention Free Transformercited by 111
  22. 2022Inception Transformercited by 99
  23. 2020ContextNet: Improving Convolutional Neural Networks for Automatic Speech Recognition with Global Contextcited by 96
  24. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 95
  25. 2022Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understandingcited by 90
  26. 2020ConvBERT: Improving BERT with Span-based Dynamic Convolutioncited by 88
  27. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
  28. 2019Jasper: An End-to-End Convolutional Neural Acoustic Modelcited by 75
  29. 2021On the Integration of Self-Attention and Convolutioncited by 74
  30. 2020Streaming automatic speech recognition with the transformer modelcited by 73
  31. 2021SLUE: New Benchmark Tasks for Spoken Language Understanding Evaluation on Natural Speechcited by 70
  32. 2021Fastformer: Additive Attention Can Be All You Needcited by 61
  33. 2019Improving Transformer Models by Reordering their Sublayerscited by 52
  34. 2018Improved training of end-to-end attention models for speech recognitioncited by 35
  35. 2019Semantic Mask for Transformer based End-to-End Speech Recognitioncited by 18
  36. 2020Attention based on-device streaming speech recognition with large speech corpuscited by 17
  37. 2021ASR-GLUE: A New Multi-task Benchmark for ASR-Robust Natural Language Understandingcited by 9
  38. 2021SRU++: Pioneering Fast Recurrence with Attention for Speech Recognitioncited by 6

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 19; 2024: 27; 2025: 27; 2026: 17.

90 papers.

Sort:

By area:

  1. 2023End-to-End Speech Recognition: A Surveycited by 127
  2. 2023Zipformer: A faster and better encoder for automatic speech recognitioncited by 77
  3. 2024OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformercited by 74
  4. 2023Voxtlm: unified decoder-only models for consolidating speech recognition/synthesis and speech/text continuation taskscited by 71
  5. 2024Towards Robust Speech Representation Learning for Thousands of Languagescited by 48
  6. 2023Reproducing Whisper-Style Training Using an Open-Source Toolkit and Publicly Available Datacited by 42
  7. 2023Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Studycited by 42
  8. 2024OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identificationcited by 34
  9. 2025OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaningcited by 27
  10. 2024ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasetscited by 26
  11. 2023Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Computecited by 21
  12. 2024Exploring the Capability of Mamba in Speech Applicationscited by 19
38 more
  1. 2025ZIPA: A family of efficient models for multilingual phone recognitioncited by 18
  2. 2024CR-CTC: Consistency regularization on CTC for improved speech recognitioncited by 15
  3. 2023Multi-Head State Space Model for Speech Recognitioncited by 14
  4. 2023A Comparative Study on E-Branchformer vs Conformer in Speech Recognition, Translation, and Understanding Taskscited by 13
  5. 2023SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understandingcited by 11
  6. 2023I3D: Transformer architectures with input-dependent dynamic depth for speech recognitioncited by 10
  7. 2023ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkitcited by 10
  8. 2024On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Modelscited by 10
  9. 2024DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understandingcited by 10
  10. 2023Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learningcited by 8
  11. 2024MLCA-AVSR: Multi-Layer Cross Attention Fusion based Audio-Visual Speech Recognitioncited by 8
  12. 2025Dolphin: A Large-Scale Automatic Speech Recognition Model for Eastern Languagescited by 8
  13. 2024Neural Blind Source Separation and Diarization for Distant Speech Recognitioncited by 6
  14. 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnetcited by 6
  15. 2025SpidR: Learning Fast and Stable Linguistic Units for Spoken Language Models Without Supervisioncited by 6
  16. 2023HyperConformer: Multi-head HyperMixer for Efficient Speech Recognitioncited by 5
  17. 2024Revisiting ASR Error Correction with Specialized Modelscited by 5
  18. 2024CNVSRC 2023: The First Chinese Continuous Visual Speech Recognition Challengecited by 5
  19. 2024Robust Audiovisual Speech Recognition Models with Mixture-of-Expertscited by 5
  20. 2024FastAdaSP: Multitask-Adapted Efficient Inference for Large Speech Language Modelcited by 5
  21. 2026An Empirical Recipe for Universal Phone Recognitioncited by 5
  22. 2023A New Benchmark of Aphasia Speech Recognition and Detection Based on E-Branchformer and Multi-task Learningcited by 4
  23. 2023Loss Masking Is Not Needed in Decoder-only Transformer for Discrete-token-based ASRcited by 4
  24. 2025On-device Streaming Discrete Speech Unitscited by 4
  25. 2025OWSM-Biasing: Contextualizing Open Whisper-Style Speech Models for Automatic Speech Recognition with Dynamic Vocabularycited by 4
  26. 2025Unifying Diarization, Separation, and ASR with Multi-Speaker Encodercited by 4
  27. 2025MEGConformer: Conformer-Based MEG Decoder for Robust Speech and Phoneme Classificationcited by 4
  28. 2023Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharingcited by 3
  29. 2024The NPU-ASLP-LiAuto System Description for Visual Speech Recognition in CNVSRC 2023cited by 3
  30. 2024Teaching a Multilingual Large Language Model to Understand Multilingual Speech via Multi-Instructional Trainingcited by 3
  31. 2024Low-resource speech recognition and dialect identification of Irish in a multi-task frameworkcited by 3
  32. 2024Children's Speech Recognition through Discrete Token Enhancementcited by 3
  33. 2024CAMEL: Cross-Attention Enhanced Mixture-of-Experts and Language Bias for Code-Switching Speech Recognitioncited by 3
  34. 2025SeniorTalk: A Chinese Conversation Dataset with Rich Annotations for Super-Aged Seniorscited by 3
  35. 2025Unveiling the Best Practices for Applying Speech Foundation Models to Speech Intelligibility Prediction for Hearing-Impaired Peoplecited by 3
  36. 2025Whale: Large-Scale multilingual ASR model with w2v-BERT and E-Branchformer with large speech datacited by 3
  37. 2025SSVD: Structured SVD for Parameter-Efficient Fine-Tuning and Benchmarking under Domain Shift in ASRcited by 3
  38. 2023Unimodal Aggregation for CTC-based Speech Recognitioncited by 2

50 most cited of 90 All 90 (JSON)