mlsanityData to 7 Oct 2026

Why the Brain Consolidates: Predictive Forgetting for Optimal Generalisation

arXiv 2603.04688 ↗First posted 5 Mar 2026q-bio.NC cs.AI cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Standard accounts of memory consolidation emphasise the stabilisation of stored representations, but struggle to explain representational drift, semanticisation, or the necessity of offline replay. Here we propose that high-capacity neocortical networks optimise stored representations for generalisation by reducing complexity via predictive forgetting, i.e. the selective retention of experienced information that predicts future outcomes or experience. We show that predictive forgetting formally improves information-theoretic generalisation bounds on stored representations. Under high-fidelity encoding constraints, such compression is generally unattainable in a single pass; high-capacity networks therefore benefit from temporally separated, iterative refinement of stored traces without re-accessing sensory input. We demonstrate this capacity dependence with simulations in autoencoder-based neocortical models, biologically plausible predictive coding circuits, and Transformer-based language models, and derive quantitative predictions for consolidation-dependent changes in neural representational geometry. These results identify a computational role for off-line consolidation beyond stabilisation, showing that outcome-conditioned compression optimises the retention-generalisation trade-off.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 18 of its 88 references that are in the map. First posted on arXiv 5 Mar 2026.

Builds on

18 of 88 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  7. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  8. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  9. 2016Deep Variational Information Bottleneckcited by 915
  10. 2021PonderNet: Learning to Pondercited by 156
  11. 2023How Does Information Bottleneck Help Deep Learning?cited by 92
  12. 2017Classification and Geometry of General Perceptual Manifoldscited by 62
6 more
  1. 2024Human-inspired Episodic Memory for Infinite Context LLMscited by 62
  2. 2025Key-value memory in the braincited by 22
  3. 2023Sample as You Infer: Predictive Coding With Langevin Dynamicscited by 11
  4. 2025Bidirectional predictive codingcited by 5
  5. 2025Bottlenecked Transformers: Periodic KV Cache Consolidation for Generalised Reasoningcited by 3
  6. 2025SuRe: Surprise-Driven Prioritised Replay for Continual LLM Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.