# Just read twice: closing the recall gap for recurrent language models

arXiv: https://arxiv.org/abs/2407.05483 · HTML: https://mlsanity.com/paper/2407.05483/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba).
- First posted on arXiv 7 Jul 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Core SSM architectures for language](https://mlsanity.com/ssm/core-ssm-architectures-for-language/)

## Built on by (most cited first, 30 of 60)

- [Gated Delta Networks: Improving Mamba2 with Delta Rule](https://mlsanity.com/paper/2412.06464/) (2024)
- [Mamba-3: Improved Sequence Modeling using State Space Principles](https://mlsanity.com/paper/2603.15569/) (2026)
- [Gated Slot Attention for Efficient Linear-Time Sequence Modeling](https://mlsanity.com/paper/2409.07146/) (2024)
- [SCBench: A KV Cache-Centric Analysis of Long-Context Methods](https://mlsanity.com/paper/2412.10319/) (2024)
- [Longhorn: State Space Models are Amortized Online Learners](https://mlsanity.com/paper/2407.14207/) (2024)
- [Falcon Mamba: The First Competitive Attention-free 7B Language Model](https://mlsanity.com/paper/2410.05355/) (2024)
- [Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layers](https://mlsanity.com/paper/2512.17351/) (2025)
- [Gated DeltaNet-2: Decoupling Erase and Write in Linear Attention](https://mlsanity.com/paper/2605.22791/) (2026)
- [Distilling to Hybrid Attention Models via KL-Guided Layer Selection](https://mlsanity.com/paper/2512.20569/) (2025)
- [Comba: Improving Bilinear RNNs with Closed-loop Control](https://mlsanity.com/paper/2506.02475/) (2025)
- [Expansion Span: Combining Fading Memory and Retrieval in Hybrid State Space Models](https://mlsanity.com/paper/2412.13328/) (2024)
- [Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences](https://mlsanity.com/paper/2604.21100/) (2026)
- [Single-Pass Document Scanning for Question Answering](https://mlsanity.com/paper/2504.03101/) (2025)
- [On State Reduction in Linear Attention](https://mlsanity.com/paper/2602.04852/) (2026)
- [Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modeling](https://mlsanity.com/paper/2510.26912/) (2025)
- [The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures](https://mlsanity.com/paper/2510.05364/) (2025)
- [Switch Attention: Towards Dynamic and Fine-grained Hybrid Transformers](https://mlsanity.com/paper/2603.26380/) (2026)
- [MDN: Parallelizing Stepwise Momentum for Delta Linear Attention](https://mlsanity.com/paper/2605.05838/) (2026)
- [Revealing and Mitigating the Local Pattern Shortcuts of Mamba](https://mlsanity.com/paper/2410.15678/) (2024)
- [Some Attention is All You Need for Retrieval](https://mlsanity.com/paper/2510.19861/) (2025)
- [Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction](https://mlsanity.com/paper/2510.20787/) (2025)
- [Controllably Efficient Language Models](https://mlsanity.com/paper/2511.05313/) (2025)
- [When to Think Fast and Slow? AMOR: Adaptive Entropy Gate for Hybrid Models](https://mlsanity.com/paper/2602.13215/) (2026)
- [Modern Transformers Are Implicit Hybrids: From Functional Differentiation to Principled Hybrid Architecture Design](https://mlsanity.com/paper/2609.02986/) (2026)
- [Nirvana: A Specialized Generalist Model With Task-Aware Memory Mechanism](https://mlsanity.com/paper/2510.26083/) (2025)
- [RAM-Net: Linear-Time Sequence Modeling with Sparsely Addressable State](https://mlsanity.com/paper/2602.11958/) (2026)
- [DART: Decoded Attention over Recurrent States for Efficient Long-Context Sequence Modeling](https://mlsanity.com/paper/2608.02032/) (2026)
- [Massive Activations in Hybrid Linear Attention Large Language Models: Pre-Attention Spikes and Inter-Spike Plateaus](https://mlsanity.com/paper/2608.12149/) (2026)
- [Birdie: Advancing State Space Models with Reward-Driven Objectives and Curricula](https://mlsanity.com/paper/2411.01030/) (2024)
- [Autoregressive Universal Video Segmentation Model](https://mlsanity.com/paper/2508.19242/) (2025)

## Abstract (arXiv)

Recurrent large language models that compete with Transformers in language modeling perplexity are emerging at a rapid rate (e.g., Mamba, RWKV). Excitingly, these architectures use a constant amount of memory during inference. However, due to the limited memory, recurrent LMs cannot recall and use all the information in long contexts leading to brittle in-context learning (ICL) quality. A key challenge for efficient LMs is selecting what information to store versus discard. In this work, we observe the order in which information is shown to the LM impacts the selection difficulty. To formalize this, we show that the hardness of information recall reduces to the hardness of a problem called set disjointness (SD), a quintessential problem in communication complexity that requires a streaming algorithm (e.g., recurrent model) to decide whether inputted sets are disjoint. We empirically and theoretically show that the recurrent memory required to solve SD changes with set order, i.e., whether the smaller set appears first in-context. Our analysis suggests, to mitigate the reliance on data order, we can put information in the right order in-context or process prompts non-causally. Towards that end, we propose: (1) JRT-Prompt, where context gets repeated multiple times in the prompt, effectively showing the model all data orders. This gives 11.0 ± 1.3 points of improvement, averaged across 16 recurrent LMs and the 6 ICL tasks, with 11.9× higher throughput than FlashAttention-2 for generation prefill (length 32k, batch size 16, NVidia H100). We then propose (2) JRT-RNN, which uses non-causal prefix-linear-attention to process prompts and provides 99% of Transformer quality at 360M params., 30B tokens and 96% at 1.3B params., 50B tokens on average across the tasks, with 19.2× higher throughput for prefill than FA2.