mlsanityData to 7 Oct 2026

SR-TTT Does Not Learn Retrieval: A Correction and Mechanistic Post-Mortem of Surprisal-Aware Residual Test-Time Training

arXiv 2603.06642 ↗First posted 26 Feb 2026cs.LG cs.AI cs.CLcited by 0

On the map: Efficient LLM inference and serving → KV cache optimization

Abstract (arXiv)

Test-Time Training (TTT) language models replace the KV-cache with fast weights updated during inference, achieving O(1) memory but suffering catastrophic failure on exact-recall tasks. Version 1 of this work proposed SR-TTT, which routes high-surprisal tokens to a sparse exact-attention Residual Cache, and reported large Needle-in-a-Haystack gains. We show those gains were evaluation artifacts: the loss and metric read logits at the answer positions rather than one position earlier, training both models to copy an answer already visible in their input (a model trained on retrieval-impossible data reaches 100% accuracy under the flawed metric); additionally, the cache attended non-causally over future tokens, including the answer itself. We release a corrected implementation with startup causality self-tests, then ask whether the SR-TTT hypothesis survives correction. It does not, and the failure decomposes into two independent, separately measured bottlenecks. Storage: surprisal gating is systematically position-biased - the TTT reconstruction loss requires burn-in before a needle becomes relatively surprising, so early-context needles are stored at near-zero rates (0-1% containment at depth 0.1) exactly where long-context memory matters most. Addressing: with storage solved by an oracle and with new trainable read-time projections, per-slot attention supervision raises addressing mass 2.5x (0.06 -> 0.15) yet token accuracy is statistically unchanged, and retrieval extracts only approx. 0.06 nats of the 2.30-nat needle; position-free content addressing cannot resolve ordered slots whose contents are near-interchangeable. Exact match remains 0% in all 2,250 paired trials across all corrected conditions. We retract the claims of v1 and offer the corrected codebase, diagnostic protocol, and negative results as a cautionary reference for surprise-gated memory architectures.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 17 of its 21 references that are in the map. First posted on arXiv 26 Feb 2026.

Builds on

17 of 21 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs3 Dec
  2. 2024Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference14 Feb

Other references · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  5. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  6. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  7. 2023TinyStories: How Small Can Language Models Be and Still Speak Coherent English?cited by 425
  8. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  9. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 329
  10. 2024Titans: Learning to Memorize at Test Timecited by 303
  11. 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 197
  12. 2023Zoology: Measuring and Improving Recall in Efficient Language Modelscited by 188
3 more
  1. 2025Test-Time Training Done Rightcited by 121
  2. 2024Dynamic Memory Compression: Retrofitting LLMs for Accelerated Inferencecited by 115
  3. 2026Gated Differentiable Working Memory for Long-Context Language Modelingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Rethinking Expressivity and Efficiency in Test-Time Training
  2. 2026TASR: Training-Free Adaptive Stopping for Iterative Retrieval
  3. 2025Scaling Test-Time Inference with Policy-Optimized, Dynamic Retrieval-Augmented Generation via KV Caching and Decoding
  4. 2026TTSR: Test-Time Self-Evolving via Reflection
  5. 2026Thinking Outside the Box: Retention and Transmission of Information in Sliding-Window KV Inference
  6. 2026Test-Time Compute for Frozen Embedding Models through Agentic Program Search
  7. 2025Retrieval as a Decision: Training-Free Adaptive Gating for Efficient RAG
  8. 2026InfoMem: Training Long-Context Memory Agents with Answer-Conditioned Information Gain