mlsanityData to 7 Oct 2026

Practical Conformer: Optimizing size, speed and flops of Conformer for on-Device and cloud ASR

arXiv 2304.00171 ↗First posted 31 Mar 2023cs.CL cs.SD eess.AScited by 0

Not on the map. Why?

Abstract (arXiv)

Conformer models maintain a large number of internal states, the vast majority of which are associated with self-attention layers. With limited memory bandwidth, reading these from memory at each inference step can slow down inference. In this paper, we design an optimized conformer that is small enough to meet on-device restrictions and has fast inference on TPUs. We explore various ideas to improve the execution speed, including replacing lower conformer blocks with convolution-only blocks, strategically downsizing the architecture, and utilizing an RNNAttention-Performer. Our optimized conformer can be readily incorporated into a cascaded-encoder setting, allowing a second-pass decoder to operate on its output and improve the accuracy whenever more resources are available. Altogether, we find that these optimizations can reduce latency by a factor of 6.8x, and come at a reasonable trade-off in quality. With the cascaded second-pass, we show that the recognition accuracy is completely recoverable. Thus, our proposed encoder can double as a strong standalone encoder in on device, and as the first part of a high-performance ASR pipeline.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 25 of its 33 references that are in the map. First posted on arXiv 31 Mar 2023.

Builds on

25 of 33 references.

References · 25

  1. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  2. 2020Rethinking Attention with Performerscited by 1,584
  3. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  4. 2019Fast Transformer Decoding: One Write-Head is All You Needcited by 700
  5. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  6. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
  7. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 232
  8. 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
  9. 2020Emformer: Efficient Memory Transformer Based Acoustic Model For Low Latency Streaming Speech Recognitioncited by 111
  10. 2020A Streaming On-Device End-to-End Model Surpassing Server-Side Conventional Model Quality and Latencycited by 99
  11. 2021Finetuning Pretrained Transformers into RNNscited by 94
  12. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
13 more
  1. 2019Improving RNN Transducer Modeling for End-to-End Speech Recognitioncited by 68
  2. 2020A Better and Faster End-to-End Model for Streaming ASRcited by 68
  3. 2021Beyond Fully-Connected Layers with Quaternions: Parameterization of Hypercomplex Multiplications with 1/n Parameterscited by 66
  4. 2021Fastformer: Additive Attention Can Be All You Needcited by 61
  5. 2020Cascaded encoders for unifying streaming and non-streaming ASRcited by 51
  6. 2021Efficient conformer: Progressive downsampling and grouped attention for automatic speech recognitioncited by 42
  7. 2021Tied & Reduced RNN-T Decodercited by 35
  8. 2020A New Training Pipeline for an Improved Neural Transducercited by 34
  9. 2020Efficient Knowledge Distillation for RNN-Transducer Modelscited by 29
  10. 2020Dynamic Sparsity Neural Networks for Automatic Speech Recognitioncited by 23
  11. 2018FLOPs as a Direct Optimization Objective for Learning Sparse Neural Networkscited by 7
  12. 2020Streaming Transformer ASR with Blockwise Synchronous Beam Searchcited by 6
  13. 2020Efficient End-to-End Speech Recognition Using Performers in Conformerscited by 1

Built on by

Not cited yet.