# You May Not Need Attention

arXiv: https://arxiv.org/abs/1810.13409 · HTML: https://mlsanity.com/paper/1810.13409/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2018.

## Built on by (most cited first, 7 of 7)

- [Continuous Rating as Reliable Human Evaluation of Simultaneous Speech Translation](https://mlsanity.com/paper/2203.02458/) (2022)
- [Is Encoder-Decoder Redundant for Neural Machine Translation?](https://mlsanity.com/paper/2210.11807/) (2022)
- [Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models](https://mlsanity.com/paper/2305.13085/) (2023)
- [Infusing Future Information into Monotonic Attention Through Language Models](https://mlsanity.com/paper/2109.03121/) (2021)
- [Non-Monotonic Attention-based Read/Write Policy Learning for Simultaneous Translation](https://mlsanity.com/paper/2503.22051/) (2025)
- [Full-Sentence Models Perform Better in Simultaneous Translation Using the Information Enhanced Decoding Strategy](https://mlsanity.com/paper/2105.01893/) (2021)
- [Overcoming Latency Bottlenecks in On-Device Speech Translation: A Cascaded Approach with Alignment-Based Streaming MT](https://mlsanity.com/paper/2508.13358/) (2025)

## Abstract (arXiv)

In NMT, how far can we get without attention and without separate encoding and decoding? To answer that question, we introduce a recurrent neural translation model that does not use attention and does not have a separate encoder and decoder. Our eager translation model is low-latency, writing target tokens as soon as it reads the first source token, and uses constant memory during decoding. It performs on par with the standard attention-based model of Bahdanau et al. (2014), and better on long sentences.