mlsanityData to 14 Oct 2026

Catastrophic Forgetting as Accessibility Collapse: A Three-Level Framework for Knowledge Persistence in Continual Learning

arXiv 2606.06032 ↗First posted 4 Jun 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Catastrophic forgetting is commonly interpreted as the irreversible erasure of previously acquired knowledge during sequential learning. In this work, we investigate an alternative perspective: that forgetting may arise not from complete destruction of task representations but from a loss of accessibility to preserved information. We introduce a three-level framework separating knowledge storage, representation, and accessibility, and evaluate each component through a series of continual-learning experiments on sequential CIFAR-100 classification using ResNet-18. Our analysis combines checkpoint persistence, linear probing, representation geometry, classifier-reset recovery, and layer-wise recoverability experiments. We observe complete behavioral forgetting of earlier tasks, with task accuracy collapsing from 54.8% to 0%, while linear probe performance retains approximately 76% of the original representational information. Furthermore, retraining only the final classifier restores 75.7% of the original task performance without modifying the backbone network. Layer-wise analysis reveals that early and intermediate layers preserve highly recoverable task information despite severe degradation at later stages. Projection-energy and principal-angle analyses indicate that retained knowledge persists as distributed high-dimensional representations rather than through preservation of a small dominant subspace. These findings suggest that catastrophic forgetting is better characterized as an accessibility failure than complete representational erasure, and that substantial task-relevant information remains embedded within neural representations even after functional forgetting has occurred.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 17 references that are in the map. First posted on arXiv 4 Jun 2026.

Builds on

10 of 17 references.

References · 10

  1. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  2. 2016Learning without Forgettingcited by 2,496
  3. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  4. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  5. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  6. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  7. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  8. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  9. 2017Lifelong Learning with Dynamically Expandable Networkscited by 694
  10. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning
  2. 2026Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning
  3. 2026On the Plasticity Collapse in Continual Machine Unlearning
  4. 2026Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability
  5. 2026To Retain or to Adapt? Generalizing Continual Learning
  6. 2025RECALL: REpresentation-aligned Catastrophic-forgetting ALLeviation via Hierarchical Model Merging
  7. 2026Information as Structural Alignment: A Dynamical Theory of Continual Learning
  8. 2025Mitigating Catastrophic Forgetting and Mode Collapse in Text-to-Image Diffusion via Latent Replay