Repeated Shared Access Enables Grokking, but Edit Propagation Depends on an Addressable Memory
Not on the map. Why?
Abstract (arXiv)
We study factual edit propagation in a controlled synthetic knowledge-graph QA setting using a 2x2 grid that crosses loop recurrence with shared-memory access: a dense transformer (Dense), a looped transformer (Loop), a dense backbone with shared memory (Dense+Mem), and a looped backbone with shared memory (loop-memory coupling, LMC). The two factors dissociate. For learning, both routes to repeated shared access -- looped recomputation and repeated memory rereading -- cross the out-of-distribution (OOD) grokking barrier that Dense fails, so repeated shared access is the behavioral regularity, not a specific architecture. For editing, the substrates split along a different axis: applying a single localized factual edit (conditioned on direct success) and measuring 2-hop propagation on a shared pre-edit-correct set, the edit propagates strongly in both memory-bearing cells (LMC 0.78-0.92, Dense+Mem 0.71-0.96) and only weakly in the memory-free ones (Loop 0.04-0.30, Dense 0.00-0.03). The split is along the memory axis, not the loop axis: every memory-bearing seed exceeds every memory-free seed, with no detectable difference between the two memory cells. Crucially Dense+Mem has no recurrence, so the propagating ingredient is an addressable site that an edit can write to and later computation rereads, not loop recomputation; Loop is at best a partial intermediate. The affordance survives coarsening the store (N=128 to N=13): propagation attenuates but the memory/no-memory split persists, so fine granularity buys precision rather than the affordance itself. These results dissociate learning competence from editing affordance -- repeated shared access suffices to grok, but edit propagation depends on whether the substrate exposes an addressable memory that the forward computation can write to and later reread, an affordance that loop recurrence provides only partially.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 31 references that are in the map. First posted on arXiv 17 Jun 2026.
Builds on
29 of 31 references.
References · 29
- 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2022Mass-Editing Memory in a Transformercited by 976
- 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
- 2014Neural Turing Machinescited by 630
- 2018Universal Transformerscited by 613
- 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 599
- 2021Fast Model Editing at Scalecited by 491
- 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
- 2014Memory Networkscited by 354
- 2024Titans: Learning to Memorize at Test Timecited by 303
- 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 292
17 more
- 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
- 2022Recurrent Memory Transformercited by 197
- 2025Scaling Latent Reasoning via Looped Language Modelscited by 145
- 2023Explaining grokking through circuit efficiencycited by 105
- 2024Looped Transformers for Length Generalizationcited by 91
- 2021Lessons on Parameter Sharing across Layers in Transformerscited by 85
- 2024Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalizationcited by 78
- 2025SIM-CoT: Supervised Implicit Chain-of-Thoughtcited by 63
- 2024MoEUT: Mixture-of-Experts Universal Transformerscited by 55
- 2024Memory Layers at Scalecited by 39
- 2026Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformerscited by 34
- 2023CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inferencecited by 27
- 2025MLP Memory: A Retriever-Pretrained Memory for Large Language Modelscited by 19
- 2025LM2: Large Memory Modelscited by 13
- 2026Adaptive Loops and Memory in Transformers: Think Harder or Know More?cited by 11
- 2024Great Memory, Shallow Reasoning: Limits of kNN-LMscited by 8
- 2026Universal Transformers Need Memory: Depth-State Trade-offs in Adaptive Recursive Reasoningcited by 5
Built on by
Not cited yet.