# Repeated Shared Access Enables Grokking, but Edit Propagation Depends on an Addressable Memory

arXiv: https://arxiv.org/abs/2606.20737 · HTML: https://mlsanity.com/paper/2606.20737/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 31 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Builds on (most cited first, 29 of 29)

- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Neural Turing Machines](https://mlsanity.com/paper/1410.5401/) (2014)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Fast Model Editing at Scale](https://mlsanity.com/paper/2110.11309/) (2021)
- [Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach](https://mlsanity.com/paper/2502.05171/) (2025)
- [Memory Networks](https://mlsanity.com/paper/1410.3916/) (2014)
- [Titans: Learning to Memorize at Test Time](https://mlsanity.com/paper/2501.00663/) (2024)
- [MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questions](https://mlsanity.com/paper/2305.14795/) (2023)
- [Reasoning with Latent Thoughts: On the Power of Looped Transformers](https://mlsanity.com/paper/2502.17416/) (2025)
- [Recurrent Memory Transformer](https://mlsanity.com/paper/2207.06881/) (2022)
- [Scaling Latent Reasoning via Looped Language Models](https://mlsanity.com/paper/2510.25741/) (2025)
- [Explaining grokking through circuit efficiency](https://mlsanity.com/paper/2309.02390/) (2023)
- [Looped Transformers for Length Generalization](https://mlsanity.com/paper/2409.15647/) (2024)
- [Lessons on Parameter Sharing across Layers in Transformers](https://mlsanity.com/paper/2104.06022/) (2021)
- [Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization](https://mlsanity.com/paper/2405.15071/) (2024)
- [SIM-CoT: Supervised Implicit Chain-of-Thought](https://mlsanity.com/paper/2509.20317/) (2025)
- [MoEUT: Mixture-of-Experts Universal Transformers](https://mlsanity.com/paper/2405.16039/) (2024)
- [Memory Layers at Scale](https://mlsanity.com/paper/2412.09764/) (2024)
- [Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers](https://mlsanity.com/paper/2604.07822/) (2026)
- [CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inference](https://mlsanity.com/paper/2310.10845/) (2023)
- [MLP Memory: A Retriever-Pretrained Memory for Large Language Models](https://mlsanity.com/paper/2508.01832/) (2025)
- [LM2: Large Memory Models](https://mlsanity.com/paper/2502.06049/) (2025)
- [Adaptive Loops and Memory in Transformers: Think Harder or Know More?](https://mlsanity.com/paper/2603.08391/) (2026)
- [Great Memory, Shallow Reasoning: Limits of kNN-LMs](https://mlsanity.com/paper/2408.11815/) (2024)
- [Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoning](https://mlsanity.com/paper/2604.21999/) (2026)

## Abstract (arXiv)

We study factual edit propagation in a controlled synthetic knowledge-graph QA setting using a 2x2 grid that crosses loop recurrence with shared-memory access: a dense transformer (Dense), a looped transformer (Loop), a dense backbone with shared memory (Dense+Mem), and a looped backbone with shared memory (loop-memory coupling, LMC). The two factors dissociate. For learning, both routes to repeated shared access -- looped recomputation and repeated memory rereading -- cross the out-of-distribution (OOD) grokking barrier that Dense fails, so repeated shared access is the behavioral regularity, not a specific architecture. For editing, the substrates split along a different axis: applying a single localized factual edit (conditioned on direct success) and measuring 2-hop propagation on a shared pre-edit-correct set, the edit propagates strongly in both memory-bearing cells (LMC 0.78-0.92, Dense+Mem 0.71-0.96) and only weakly in the memory-free ones (Loop 0.04-0.30, Dense 0.00-0.03). The split is along the memory axis, not the loop axis: every memory-bearing seed exceeds every memory-free seed, with no detectable difference between the two memory cells. Crucially Dense+Mem has no recurrence, so the propagating ingredient is an addressable site that an edit can write to and later computation rereads, not loop recomputation; Loop is at best a partial intermediate. The affordance survives coarsening the store (N=128 to N=13): propagation attenuates but the memory/no-memory split persists, so fine granularity buys precision rather than the affordance itself. These results dissociate learning competence from editing affordance -- repeated shared access suffices to grok, but edit propagation depends on whether the substrate exposes an addressable memory that the forward computation can write to and later reread, an affordance that loop recurrence provides only partially.