# MFLA: Monotonic Finite Look-ahead Attention for Streaming Speech Recognition

arXiv: https://arxiv.org/abs/2506.03722 · HTML: https://mlsanity.com/paper/2506.03722/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 26 references that are in the map.
- First posted on arXiv 4 Jun 2025.

## Builds on (most cited first, 22 of 22)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [MLS: A Large-Scale Multilingual Dataset for Speech Research](https://mlsanity.com/paper/2012.03411/) (2020)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognition](https://mlsanity.com/paper/2206.08317/) (2022)
- [WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit](https://mlsanity.com/paper/2102.01547/) (2021)
- [Monotonic Infinite Lookback Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.05218/) (2019)
- [Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling](https://mlsanity.com/paper/2311.00430/) (2023)
- [Online and Linear-Time Attention by Enforcing Monotonic Alignments](https://mlsanity.com/paper/1704.00784/) (2017)
- [Monotonic Chunkwise Attention](https://mlsanity.com/paper/1712.05382/) (2017)
- [CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognition](https://mlsanity.com/paper/1905.11235/) (2019)
- [Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection](https://mlsanity.com/paper/2005.11185/) (2020)
- [WenetSpeech4TTS: A 12,800-hour Mandarin TTS Corpus for Large Speech Generation Model Benchmark](https://mlsanity.com/paper/2406.05763/) (2024)
- [Thinking Slow about Latency Evaluation for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.00048/) (2019)
- [Synchronous Transformers for End-to-End Speech Recognition](https://mlsanity.com/paper/1912.02958/) (2019)
- [BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5](https://mlsanity.com/paper/2406.19954/) (2024)
- [Decoder-only Architecture for Streaming End-to-end Speech Recognition](https://mlsanity.com/paper/2406.16107/) (2024)
- [Efficient Streaming LLM for Speech Recognition](https://mlsanity.com/paper/2410.03752/) (2024)
- [Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detection](https://mlsanity.com/paper/2406.10052/) (2024)
- [Whisper in Medusa's Ear: Multi-head Efficient Decoding for Transformer-based ASR](https://mlsanity.com/paper/2409.15869/) (2024)
- [A Transcription Prompt-based Efficient Audio Large Language Model for Robust Speech Recognition](https://mlsanity.com/paper/2408.09491/) (2024)

## Built on by (most cited first, 1 of 1)

- [From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Models](https://mlsanity.com/paper/2603.04592/) (2026)

## Abstract (arXiv)

Applying large pre-trained speech models like Whisper has shown promise in reducing training costs for various speech tasks. However, integrating these models into streaming systems remains a challenge. This paper presents a novel prefix-to-prefix training framework for streaming recognition by fine-tuning the Whisper. We introduce the Continuous Integrate-and-Fire mechanism to establish a quasi-monotonic alignment between continuous speech sequences and discrete text tokens. Additionally, we design Monotonic Finite Look-ahead Attention, allowing each token to attend to infinite left-context and finite right-context from the speech sequences. We also employ the wait-k decoding strategy to simplify the decoding process while ensuring consistency between training and testing. Our theoretical analysis and experiments demonstrate that this approach achieves a controllable trade-off between latency and quality, making it suitable for various streaming applications.