# Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling

arXiv: https://arxiv.org/abs/1801.10296 · HTML: https://mlsanity.com/paper/1801.10296/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Jan 2018.

## Built on by (most cited first, 15 of 15)

- [A General Survey on Attention Mechanisms in Deep Learning](https://mlsanity.com/paper/2203.14263/) (2022)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Visual Attention Methods in Deep Learning: An In-Depth Survey](https://mlsanity.com/paper/2204.07756/) (2022)
- [RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting](https://mlsanity.com/paper/2108.01316/) (2021)
- [Attention Mechanism in Neural Networks: Where it Comes and Where it Goes](https://mlsanity.com/paper/2204.13154/) (2022)
- [Scalable Multi-Agent Model-Based Reinforcement Learning](https://mlsanity.com/paper/2205.15023/) (2022)
- [Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement](https://mlsanity.com/paper/2412.01282/) (2024)
- [Concentration Network for Reinforcement Learning of Large-Scale Multi-Agent Systems](https://mlsanity.com/paper/2203.06416/) (2022)
- [Video Violence Recognition and Localization Using a Semi-Supervised Hard Attention Model](https://mlsanity.com/paper/2202.02212/) (2022)
- [Decentralized Graph Neural Network for Privacy-Preserving Recommendation](https://mlsanity.com/paper/2308.08072/) (2023)
- [EFCNet: Every Feature Counts for Small Medical Object Segmentation](https://mlsanity.com/paper/2406.18201/) (2024)
- [Integrating Biological and Machine Intelligence: Attention Mechanisms in Brain-Computer Interfaces](https://mlsanity.com/paper/2502.19281/) (2025)
- [Three-stream network for enriched Action Recognition](https://mlsanity.com/paper/2104.13051/) (2021)
- [An Interpretable Framework for Drug-Target Interaction with Gated Cross Attention](https://mlsanity.com/paper/2109.08360/) (2021)
- [LanguaShrink: Reducing Token Overhead with Psycholinguistics](https://mlsanity.com/paper/2409.00855/) (2024)

## Abstract (arXiv)

Many natural language processing tasks solely rely on sparse dependencies between a few tokens in a sentence. Soft attention mechanisms show promising performance in modeling local/global dependencies by soft probabilities between every two tokens, but they are not effective and efficient when applied to long sentences. By contrast, hard attention mechanisms directly select a subset of tokens but are difficult and inefficient to train due to their combinatorial nature. In this paper, we integrate both soft and hard attention into one context fusion model, "reinforced self-attention (ReSA)", for the mutual benefit of each other. In ReSA, a hard attention trims a sequence for a soft self-attention to process, while the soft attention feeds reward signals back to facilitate the training of the hard one. For this purpose, we develop a novel hard attention called "reinforced sequence sampling (RSS)", selecting tokens in parallel and trained via policy gradient. Using two RSS modules, ReSA efficiently extracts the sparse dependencies between each pair of selected tokens. We finally propose an RNN/CNN-free sentence-encoding model, "reinforced self-attention network (ReSAN)", solely based on ReSA. It achieves state-of-the-art performance on both Stanford Natural Language Inference (SNLI) and Sentences Involving Compositional Knowledge (SICK) datasets.