Continual Learning as Shared-Manifold Continuation Under Compatible Shift
Not on the map. Why?
Abstract (arXiv)
Continual learning methods usually preserve old behavior by regularizing parameters, matching old outputs, or replaying previous examples. These strategies can reduce forgetting, but they do not directly specify how the latent representation should evolve. We study a narrower geometric alternative for the regime where old and new data should remain on the same latent support: continual learning as continuation of a shared manifold. We instantiate this view within Support-Preserving Manifold Assimilation (SPMA) and evaluate a geometry-preserving variant, SPMA-OG, that combines sparse replay, output distillation, relational geometry preservation, local smoothing, and chart-assignment regularization on old anchors. On representative compatible-shift CIFAR10 and Tiny-ImageNet runs, SPMA-OG improves over sparse replay baselines in old-task retention and representation-preservation metrics while remaining competitive on new-task accuracy. On a controlled synthetic atlas-manifold benchmark, it achieves near-perfect anchor-geometry preservation while also improving new-task accuracy over replay. These results provide evidence that geometry-aware anchor regularization is a useful inductive bias when continual learning should preserve a shared latent support rather than create a new one.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 16 references that are in the map. First posted on arXiv 20 Mar 2026.
Builds on
12 of 16 references.
References · 12
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2016Learning without Forgettingcited by 2,496
- 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
- 2017Gradient Episodic Memory for Continual Learningcited by 1,898
- 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
- 2018Experience Replay for Continual Learningcited by 1,095
- 2018Efficient Lifelong Learning with A-GEMcited by 1,058
- 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
- 2019Relational Knowledge Distillationcited by 889
- 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
- 2023Subspace Distillation for Continual Learningcited by 12
Built on by
Not cited yet.