# Area Attention

arXiv: https://arxiv.org/abs/1810.10126 · HTML: https://mlsanity.com/paper/1810.10126/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2018.

## Built on by (most cited first, 5 of 5)

- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [Speech Emotion Recognition with Multiscale Area Attention and Data Augmentation](https://mlsanity.com/paper/2102.01813/) (2021)
- [Efficient Representation Learning via Adaptive Context Pooling](https://mlsanity.com/paper/2207.01844/) (2022)
- [Attention-based UAV Trajectory Optimization for Wireless Power Transfer-assisted IoT Systems](https://mlsanity.com/paper/2502.17517/) (2025)
- [Container damage detection using advanced computer vision model Yolov12 vs Yolov11 vs RF-DETR A comparative analysis](https://mlsanity.com/paper/2506.22517/) (2025)

## Abstract (arXiv)

Existing attention mechanisms are trained to attend to individual items in a collection (the memory) with a predefined, fixed granularity, e.g., a word token or an image grid. We propose area attention: a way to attend to areas in the memory, where each area contains a group of items that are structurally adjacent, e.g., spatially for a 2D memory such as images, or temporally for a 1D memory such as natural language sentences. Importantly, the shape and the size of an area are dynamically determined via learning, which enables a model to attend to information with varying granularity. Area attention can easily work with existing model architectures such as multi-head attention for simultaneously attending to multiple areas in the memory. We evaluate area attention on two tasks: neural machine translation (both character and token-level) and image captioning, and improve upon strong (state-of-the-art) baselines in all the cases. These improvements are obtainable with a basic form of area attention that is parameter free.