mlsanityData to 14 Oct 2026

Memoir: Should a Model Write to Its Memory While It Thinks?

arXiv 2607.20792 ↗First posted 22 Jul 2026cs.LG cs.NEcited by 0

Not on the map. Why?

Abstract (arXiv)

Memoir combines per-sample fast memory, shared slow parameters, variable-depth latent recurrence, and a future-latent energy objective. We test its riskiest coupling: each pondering iteration may rewrite the fast tier that the same iteration reads. On procedural associative recall with key interference, we compare a coupled arm against an otherwise identical read-only pondering arm. Both arms contain 81,738 parameters, including 76,362 trainable parameters, and use matched declared forward multiply-accumulate counts, data, optimizer, schedule, and seeds. After 240 training steps across 12 seeds, coupled recall is 0.5203 with a 95 percent interval of [0.4522, 0.5883], while read-only recall is 0.6557 with [0.5953, 0.7160]. The arms are paired per seed, and the read-only lead of 0.1354 gives a paired t of 3.23 on 11 degrees of freedom with a 95 percent interval of [0.0431, 0.2277] on the difference, winning on 10 of 12 seeds. After 960 steps across 8 seeds, both arms reach 1.0000, so the measured effect is a learning-speed penalty at a fixed budget, not a demonstrated capability penalty. That longer control is ceiling limited, leaving convergence on a non-saturating task unmeasured. A predicted failure in which memory rewriting corrupts the energy signal did not occur: the energy margin grew and held. Kernel restructuring also reduced delta-rule forward time from 0.907 ms to 0.351 ms on the stated device. Code and evidence are available at https://github.com/RightNow-AI/Memoir

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 21 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

18 of 21 references.

References · 18

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  3. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  4. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  5. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  6. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  7. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  8. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 415
  9. 2024Learning to (Learn at Test Time): RNNs with Expressive Hidden Statescited by 355
  10. 2024Titans: Learning to Memorize at Test Timecited by 328
  11. 2025Why Language Models Hallucinatecited by 220
  12. 2024The Surprising Effectiveness of Test-Time Training for Few-Shot Learningcited by 96
6 more
  1. 2025Nested Learning: The Illusion of Deep Learning Architecturescited by 72
  2. 2025ATLAS: Learning to Optimally Memorize the Context at Test Timecited by 66
  3. 2025It's All Connected: A Journey Through Test-Time Memorization, Attentional Bias, Retention, and Online Optimizationcited by 64
  4. 2025End-to-End Test-Time Training for Long Contextcited by 62
  5. 2025A Systematic Analysis of Hybrid Linear Attentioncited by 39
  6. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 31

Built on by

Not cited yet.