# Efficient Monotonic Multihead Attention

arXiv: https://arxiv.org/abs/2312.04515 · HTML: https://mlsanity.com/paper/2312.04515/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 7 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Sequence-to-Sequence Models Can Directly Translate Foreign Speech](https://mlsanity.com/paper/1703.08581/) (2017)
- [Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/1612.01744/) (2016)
- [Monotonic Infinite Lookback Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.05218/) (2019)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [SimulEval: An Evaluation Toolkit for Simultaneous Translation](https://mlsanity.com/paper/2007.16193/) (2020)
- [Monotonic Multihead Attention](https://mlsanity.com/paper/1909.12406/) (2019)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [Online and Linear-Time Attention by Enforcing Monotonic Alignments](https://mlsanity.com/paper/1704.00784/) (2017)
- [Monotonic Chunkwise Attention](https://mlsanity.com/paper/1712.05382/) (2017)
- [ESPnet-ST: All-in-One Speech Translation Toolkit](https://mlsanity.com/paper/2004.10234/) (2020)
- [SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2011.02048/) (2020)
- [End-to-End Automatic Speech Translation of Audiobooks](https://mlsanity.com/paper/1802.04200/) (2018)
- [UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units](https://mlsanity.com/paper/2212.08055/) (2022)
- [Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translation](https://mlsanity.com/paper/1806.03661/) (2018)
- [Low-Resource Speech-to-Text Translation](https://mlsanity.com/paper/1803.09164/) (2018)

## Built on by (most cited first, 8 of 8)

- [BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5](https://mlsanity.com/paper/2406.19954/) (2024)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Speech ReaLLM -- Real-time Streaming Speech Recognition with Multimodal LLMs by Teaching the Flow of Time](https://mlsanity.com/paper/2406.09569/) (2024)
- [Navigating the Minefield of MT Beam Search in Cascaded Streaming Speech Translation](https://mlsanity.com/paper/2407.11010/) (2024)
- [SimulMEGA: MoE Routers are Advanced Policy Makers for Simultaneous Speech Translation](https://mlsanity.com/paper/2509.01200/) (2025)
- [Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT](https://mlsanity.com/paper/2508.13358/) (2025)
- [Regularized Entropy Information Adaptation with Temporal-Awareness Networks for Simultaneous Speech Translation](https://mlsanity.com/paper/2604.09916/) (2026)
- [Attention-Based Adaptive Policies for Simultaneous Speech-to-Text Translation](https://mlsanity.com/paper/2609.30839/) (2026)

## Abstract (arXiv)

We introduce the Efficient Monotonic Multihead Attention (EMMA), a state-of-the-art simultaneous translation model with numerically-stable and unbiased monotonic alignment estimation. In addition, we present improved training and inference strategies, including simultaneous fine-tuning from an offline translation model and reduction of monotonic alignment variance. The experimental results demonstrate that the proposed model attains state-of-the-art performance in simultaneous speech-to-text translation on the Spanish and English translation task.