# Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognition

arXiv: https://arxiv.org/abs/2305.05084 · HTML: https://mlsanity.com/paper/2305.05084/

## Key facts

- Cited by 124 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2023.

## Built on by (most cited first, 30 of 124)

- [SALM: Speech-augmented Language Model with In-context Learning for Speech Recognition and Translation](https://mlsanity.com/paper/2310.09424/) (2023)
- [Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and AST](https://mlsanity.com/paper/2509.14128/) (2025)
- [OWSM-CTC: An Open Encoder-Only Speech Foundation Model for Speech Recognition, Translation, and Language Identification](https://mlsanity.com/paper/2402.12654/) (2024)
- [Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence](https://mlsanity.com/paper/2604.24954/) (2026)
- [Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models](https://mlsanity.com/paper/2406.10507/) (2024)
- [OWSM v4: Improving Open Whisper-Style Speech Models via Data Scaling and Cleaning](https://mlsanity.com/paper/2506.00338/) (2025)
- [Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognition](https://mlsanity.com/paper/2312.17279/) (2023)
- [Benchmarking Large Multimodal Models against Common Corruptions](https://mlsanity.com/paper/2401.11943/) (2024)
- [Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation](https://mlsanity.com/paper/2510.06961/) (2025)
- [Granary: Speech Recognition and Translation Dataset in 25 European Languages](https://mlsanity.com/paper/2505.13404/) (2025)
- [BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5](https://mlsanity.com/paper/2406.19954/) (2024)
- [Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models](https://mlsanity.com/paper/2502.12414/) (2025)
- [Speed of Light Exact Greedy Decoding for RNN-T Speech Recognition Models on GPU](https://mlsanity.com/paper/2406.03791/) (2024)
- [Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Play](https://mlsanity.com/paper/2505.02707/) (2025)
- [Label-Looping: Highly Efficient Decoding for Transducers](https://mlsanity.com/paper/2406.06220/) (2024)
- [Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representations](https://mlsanity.com/paper/2407.03495/) (2024)
- [Training and Inference Efficiency of Encoder-Decoder Speech Models](https://mlsanity.com/paper/2503.05931/) (2025)
- [Hallucination Benchmark for Speech Foundation Models](https://mlsanity.com/paper/2510.16567/) (2025)
- [Multilingual Audio-Visual Speech Recognition with Hybrid CTC/RNN-T Fast Conformer](https://mlsanity.com/paper/2405.12983/) (2024)
- [Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter](https://mlsanity.com/paper/2406.07096/) (2024)
- [LiteASR: Efficient Automatic Speech Recognition with Low-Rank Approximation](https://mlsanity.com/paper/2502.20583/) (2025)
- [NGPU-LM: GPU-Accelerated N-Gram Language Model for Context-Biasing in Greedy ASR Decoding](https://mlsanity.com/paper/2505.22857/) (2025)
- [Discrete Diffusion for Generative Modeling of Text-Aligned Speech Tokens](https://mlsanity.com/paper/2509.20060/) (2025)
- [Hearing to Translate: The Effectiveness of Speech Modality Integration into LLMs](https://mlsanity.com/paper/2512.16378/) (2025)
- [Towards Resource-Efficient Compound AI Systems](https://mlsanity.com/paper/2501.16634/) (2025)
- [The Silent Thought: Modeling Internal Cognition in Full-Duplex Spoken Dialogue Models via Latent Reasoning](https://mlsanity.com/paper/2603.17837/) (2026)
- [BERSting at the Screams: A Benchmark for Distanced, Emotional and Shouted Speech Recognition](https://mlsanity.com/paper/2505.00059/) (2025)
- [SPGISpeech 2.0: Transcribed multi-speaker financial audio for speaker-tagged transcription](https://mlsanity.com/paper/2508.05554/) (2025)
- [Pseudo-Labeling for Domain-Agnostic Bangla Automatic Speech Recognition](https://mlsanity.com/paper/2311.03196/) (2023)
- [To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation](https://mlsanity.com/paper/2406.04512/) (2024)

## Abstract (arXiv)

Conformer-based models have become the dominant end-to-end architecture for speech processing tasks. With the objective of enhancing the conformer architecture for efficient training and inference, we carefully redesigned Conformer with a novel downsampling schema. The proposed model, named Fast Conformer(FC), is 2.8x faster than the original Conformer, supports scaling to Billion parameters without any changes to the core architecture and also achieves state-of-the-art accuracy on Automatic Speech Recognition benchmarks. To enable transcription of long-form speech up to 11 hours, we replaced global attention with limited context attention post-training, while also improving accuracy through fine-tuning with the addition of a global token. Fast Conformer, when combined with a Transformer decoder also outperforms the original Conformer in accuracy and in speed for Speech Translation and Spoken Language Understanding.