mlsanityData to 14 Oct 2026

Neural Succession: A Mesoscopic Theory of Invasion, Coexistence, and Stabilization in Continual Learning

arXiv 2609.36375 ↗First posted 28 Sep 2026cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Continual learning is usually studied through mechanisms that preserve old knowledge. We develop Successional Learning Theory (SLT), a mesoscopic account in which the current representation is a resident community, the incoming task is an invader, forgetting is resident displacement, joint retention is coexistence, replay is resident reinforcement, and training moves from establishment toward stabilization. Its empirical coordinate is directional pre-invasion compatibility, measured on the resident model before the incoming task is learned. Across eight experiments, compatibility orders later forgetting on the 20 directed Split-CIFAR-10 transitions (three-repeat r=-0.789, incoming-task cluster 95% CI [-0.90,-0.72], every repeat alone r<=-0.67), forecasts held-out forgetting with 24% lower error than a no-information baseline, and reproduces under controlled MNIST permutations and CIFAR-10 rotations (r=-0.804, -0.718). On an 84-transition suite, compatibility separates coexistence from exclusion at every retention threshold (AUC 0.93-0.97). Replay repairs every transition with at most 325 stored examples and is most efficient where displacement is largest. Compatibility reaches |r|=0.720, while activation, representation, Jacobian, and fixed-coefficient Lotka-Volterra specializations do not. Plasticity and feature turnover fall reliably from early to late training (15/15 and 14/15 runs). We formalize a minimum habitat-modification bound, a displacement floor, a sufficient coexistence condition, an identifiability law with a range-restriction corollary, successional stabilization, and local reinforcement. The identifiability law also predicts where the coordinate loses leverage, and the prediction matches three CIFAR-100 partitions and five optimizer regimes. SLT is a pre-adaptation diagnostic that complements replay, regularization, and projection methods.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 47 of its 82 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

47 of 82 references.

References · 47

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  4. 2016Learning without Forgettingcited by 2,496
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,333
  6. 2017Gradient Episodic Memory for Continual Learningcited by 1,898
  7. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  8. 2016Progressive Neural Networkscited by 1,361
  9. 2018Continual Lifelong Learning with Neural Networks: A Reviewcited by 1,330
  10. 2017Memory Aware Synapses: Learning what (not) to forgetcited by 1,105
  11. 2018Experience Replay for Continual Learningcited by 1,095
  12. 2018Efficient Lifelong Learning with A-GEMcited by 1,058
35 more
  1. 2020Dark Experience for General Continual Learning: a Strong, Simple Baselinecited by 911
  2. 2017PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruningcited by 898
  3. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 872
  4. 2022Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distributioncited by 739
  5. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 722
  6. 2019Three scenarios for continual learningcited by 614
  7. 2016Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Lojasiewicz Conditioncited by 539
  8. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 521
  9. 2019Online Continual Learning with Maximally Interfered Retrievalcited by 379
  10. 2019Orthogonal Gradient Descent for Continual Learningcited by 369
  11. 2021Gradient Projection Memory for Continual Learningcited by 348
  12. 2018Gradient Descent Happens in a Tiny Subspacecited by 217
  13. 2020LEEP: A New Measure to Evaluate Transferability of Learned Representationscited by 188
  14. 2020Understanding the Role of Training Regimes in Continual Learningcited by 180
  15. 2018Continual Learning of Context-dependent Processing in Neural Networkscited by 179
  16. 2023Understanding plasticity in neural networkscited by 174
  17. 2021LogME: Practical Assessment of Pre-trained Models for Transfer Learningcited by 172
  18. 2020Anatomy of Catastrophic Forgetting: Hidden Representations and Task Semanticscited by 161
  19. 2020Linear Mode Connectivity in Multitask and Continual Learningcited by 145
  20. 2021Training Networks in Null Space of Feature Covariance for Continual Learningcited by 143
  21. 2019Transferability and Hardness of Supervised Classification Taskscited by 127
  22. 2020The Early Phase of Neural Network Trainingcited by 115
  23. 2020A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrixcited by 103
  24. 2021Continual Learning in the Teacher-Student Setup: Impact of Task Similaritycited by 88
  25. 2020Optimal Continual Learning has Perfect Memory and is NP-hardcited by 87
  26. 2021Continual Learning of a Mixed Sequence of Similar and Dissimilar Taskscited by 81
  27. 2022How catastrophic can catastrophic forgetting be in linear regression?cited by 81
  28. 2023Theory on Forgetting and Generalization of Continual Learningcited by 72
  29. 2019Toward Understanding Catastrophic Forgetting in Continual Learningcited by 68
  30. 2022The Effect of Task Ordering in Continual Learningcited by 31
  31. 2024Disentangling and Mitigating the Impact of Task Similarity for Continual Learningcited by 19
  32. 2025Optimal Task Order for Continual Learning of Multiple Taskscited by 14
  33. 2024Budgeted Online Continual Learning by Adaptive Layer Freezing and Frequency-based Samplingcited by 7
  34. 2019Machine Learning as Ecologycited by 1
  35. 2022Toward Sustainable Continual Learning: Detection and Knowledge Repurposing of Similar Taskscited by 1

Built on by

Not cited yet.