The Art of Not Forgetting
Not on the map. Why?
Abstract (arXiv)
We introduce CMP (Cognitive Memory Primitive), an architecture that represents inputs as sparse relational codes, stores them in a two-tier competitive memory, and learns entirely through local, gradient-free updates, with no backpropagation anywhere in the network. We use this architecture to test a specific hypothesis: that catastrophic forgetting, usually treated as a training-time defect to be patched with replay or regularization, is instead a structural consequence of how backpropagation assigns credit and that a learning rule that is local and sparse by construction should resist it without a patch. On a controlled domain-incremental protocol across 15 text domains, three-seed replicated, CMP's backward transfer is 15-19x better than a matched-size Transformer trained with online EWC, and the result survives a domain-order control (reported as a range, +0.24 to +0.44, rather than a single figure). We report this alongside a real, substantial accuracy gap versus the Transformer baseline, a null result on a recognized vision benchmark, and a diagnosed, unresolved failure attempting to combine this architecture with a separate mechanism that improves raw accuracy, disclosed because an honest negative result is more useful than an omitted one. The central claim is narrow and falsifiable: local, sparse, non-backpropagation learning measurably resists catastrophic forgetting better than backpropagation with its standard fix, under conditions we state precisely.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 18 references that are in the map. First posted on arXiv 20 Jul 2026.
Builds on
9 of 18 references.
References · 9
- 2017Attention Is All You Needcited by 60,361
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2017Gradient Episodic Memory for Continual Learningcited by 1,898
- 2016Progressive Neural Networkscited by 1,361
- 2018Progress & Compress: A scalable framework for continual learningcited by 520
- 2020Hopfield Networks is All You Needcited by 475
- 2024Byte Latent Transformer: Patches Scale Better Than Tokenscited by 146
Built on by
Not cited yet.