Forgetting Is Not a Fix: Path Dependence in Sequential Engram Editing
Not on the map. Why?
Abstract (arXiv)
AI Engram (Kwon et al., 2026) formalizes the four engram criteria of neuroscience as a constrained inverse problem in weight space and solves it closed-form: concept-specific memory traces become linear objects that can be extracted once and combined arithmetically. Appendix F states the Compositional Memory States Hypothesis: edited models live on "a commutative manifold where the integration of A and B reaches a consistent equilibrium regardless of the learning sequence." The evidence base is single and paired edits -- in materials terms, single-cycle tests, in which fatigue accumulation is structurally invisible. Whether the hypothesis holds under sequential load is exactly the "temporal dynamics" question the paper defers to future work. We run that test on the authors' own reference implementation, at their reported best edit strength (TOFU alpha=0.6, a choice favoring the linearity hypothesis), with pre-registered predictions, across three model charges (two vendors, two architecture families). Four findings replicate across all three: (1) zero-shot composition and sequential re-calibrated editing diverge by 61-71% of the edit magnitude; (2) cut order is not interchangeable, and the effect scales with concept overlap -- in one charge the order of cutting two Paris landmarks decides whether an uninvolved third concept survives; (3) the survivors' layer-input covariances -- the method's own sufficient statistics, read as strain gauges -- drift monotonically with every further cut, in every surviving concept, in every charge; (4) erased knowledge partially returns under subsequent unrelated cuts. Appendix F's commutative-manifold hypothesis is thereby falsified for sequential editing; the single-edit results of the original paper are untouched. For unlearning-as-compliance: erasure certified today does not certify the artifact after its next edit.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 5 of its 7 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
5 of 7 references.
References · 5
- 2022Editing Models with Task Arithmeticcited by 1,120
- 2022Mass-Editing Memory in a Transformercited by 1,001
- 2022Toy Models of Superpositioncited by 971
- 2024TinyLlama: An Open-Source Small Language Modelcited by 618
- 2024TOFU: A Task of Fictitious Unlearning for LLMscited by 533
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Activation-Space Order-Swap Geometry: A Site-Asymmetry Audit26 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Information as Structural Alignment: A Dynamical Theory of Continual Learning
- 2024Navigating the Dual Facets: A Comprehensive Evaluation of Sequential Memory Editing in Large Language Models
- 2026First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers
- 2026Mind the Gap: Diagnosing Constraint Discovery Failures in Text-in-Image Editing
- 2026Cortex: Consolidating Context into Parametric Memory
- 2025Lessons from Neuroscience for AI: How integrating Actions, Compositional Structure and Episodic Memory could enable Safe, Interpretable and Human-Like AI
- 2025From Memorization to Reasoning in the Spectrum of Loss Curvature
- 2026More Edits, More Stable: Understanding the Lifelong Normalization in Sequential Model Editing