mlsanityData to 14 Oct 2026

Test-Time Training with KV Binding Is Secretly Linear Attention

arXiv 2602.21204 ↗First posted 24 Feb 2026cs.LG cs.AI cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

Test-time training (TTT) with KV binding as sequence modeling layer is commonly interpreted as a form of online meta-learning that memorizes a key-value mapping at test time. However, our analysis reveals multiple phenomena that contradict this memorization-based interpretation. Motivated by these findings, we revisit the formulation of TTT and show that a broad class of TTT architectures can be expressed as a form of learned linear attention operator. Beyond explaining previously puzzling model behaviors, this perspective yields multiple practical benefits: it enables principled architectural simplifications, admits fully parallel formulations that preserve performance while improving efficiency, and provides a systematic reduction of diverse TTT variants to a standard linear attention form. Overall, our results reframe TTT not as test-time memorization, but as learned linear attention with enhanced representational capacity. Project page: https://research.nvidia.com/labs/sil/projects/tttla/.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 43 of its 51 references that are in the map. First posted on arXiv 24 Feb 2026.

Builds on

43 of 51 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  6. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  8. 2020GLU Variants Improve Transformercited by 1,804
  9. 2025s1: Simple test-time scalingcited by 1,341
  10. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  11. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  12. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
31 more
  1. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  2. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  3. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  4. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  5. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
  6. 2024Parallelizing Linear Transformers with the Delta Rule over Sequence Lengthcited by 396
  7. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  8. 2024Titans: Learning to Memorize at Test Timecited by 328
  9. 2022Test-Time Training with Masked Autoencoderscited by 212
  10. 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
  11. 2024Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrencecited by 159
  12. 2025RWKV-7 "Goose" with Expressive Dynamic State Evolutioncited by 141
  13. 2025Test-Time Training Done Rightcited by 130
  14. 2025TTT3R: 3D Reconstruction as Test-Time Trainingcited by 129
  15. 2024HGRN2: Gated Linear RNNs with State Expansioncited by 127
  16. 2023Hierarchically Gated Recurrent Neural Network for Sequence Modelingcited by 102
  17. 2025One-Minute Video Generation with Test-Time Trainingcited by 101
  18. 2024Gated Slot Attention for Efficient Linear-Time Sequence Modelingcited by 76
  19. 2025Test-time regression: a unifying framework for designing sequence models with associative memorycited by 74
  20. 2025Nested Learning: The Illusion of Deep Learning Architecturescited by 72
  21. 2025ATLAS: Learning to Optimally Memorize the Context at Test Timecited by 66
  22. 2025It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimizationcited by 64
  23. 2025End-to-End Test-Time Training for Long Contextcited by 62
  24. 2024Longhorn: State Space Models are Amortized Online Learnerscited by 59
  25. 2018Meta-Learning Update Rules for Unsupervised Representation Learningcited by 37
  26. 2025Understanding Transformer from the Perspective of Associative Memorycited by 34
  27. 2024G3R: Gradient Guided Generalizable Reconstructioncited by 31
  28. 2025Test3R: Learning to Reconstruct 3D at Test Timecited by 20
  29. 2025Lattice: Learning to Efficiently Compress the Memorycited by 19
  30. 2025ViT3: Unlocking Test-Time Training in Visioncited by 18
  31. 2025Gated KalmaNet: A Fading Memory Layer Through Test-Time Ridge Regressioncited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026HorizonStream: Long-Horizon Attention for Streaming 3D Reconstructioncited by 9
  2. 2026Memory for Large Language Modelscited by 4
  3. 2026Fast Spatial Memory with Elastic Test-Time Trainingcited by 3
  4. 2026Training LLM Agents for Spontaneous, Reward-Free Self-Evolution via World Knowledge Explorationcited by 3
  5. 2026Can Video World Models Track Unobserved World States?cited by 2
  6. 2026Hybrid Associative Memoriescited by 1
  7. 2026Online Neural Space Time Memory for Dynamic Novel View Synthesiscited by 0
  8. 2026Black-Mamba: Biologically-Inspired Leaky Accumulation for Conceptual Knowledge under Distribution Driftcited by 0
  9. 2026Using Context Is Not Enough: Test-Time Training for Personalized Reward Modelingcited by 0
  10. 2026Copy the Same, Distill the Difference: Initializing Linear Vision Transformerscited by 0
  11. 2026DeltaTTT: Layerwise Optimization for Nonlinear Recurrent Memorycited by 0
  12. 2026Error Correction in Trained TTT Layerscited by 0
1 more
  1. 2026Encoding Context into LLM Weights: What Makes It Context?cited by 0