# Effect of Spherical Parameter Space on Number of Non-Forgetting Tasks and Capacity of Continual Learning

arXiv: https://arxiv.org/abs/2610.17453 · HTML: https://mlsanity.com/paper/2610.17453/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 12 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Continual Learning of Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2404.16789/) (2024)
- [Understanding the Role of Training Regimes in Continual Learning](https://mlsanity.com/paper/2006.06958/) (2020)
- [Linear Mode Connectivity in Multitask and Continual Learning](https://mlsanity.com/paper/2010.04495/) (2020)
- [Optimal Continual Learning has Perfect Memory and is NP-hard](https://mlsanity.com/paper/2006.05188/) (2020)
- [Theory on Forgetting and Generalization of Continual Learning](https://mlsanity.com/paper/2302.05836/) (2023)
- [Hierarchically Structured Meta-learning](https://mlsanity.com/paper/1905.05301/) (2019)
- [Continual evaluation for lifelong learning: Identifying the stability gap](https://mlsanity.com/paper/2205.13452/) (2022)
- [Exploring Data Geometry for Continual Learning](https://mlsanity.com/paper/2304.03931/) (2023)
- [Continual Learning with Guarantees via Weight Interval Constraints](https://mlsanity.com/paper/2206.07996/) (2022)
- [Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretability](https://mlsanity.com/paper/2601.22012/) (2026)
- [Subspace Geometry Governs Catastrophic Forgetting in Low-Rank Adaptation](https://mlsanity.com/paper/2603.02224/) (2026)
- [Fisher-Orthogonal Projected Natural Gradient Descent for Continual Learning](https://mlsanity.com/paper/2601.12816/) (2026)
- [Exploring the Impact of Parameter Update Magnitude on Forgetting and Generalization of Continual Learning](https://mlsanity.com/paper/2602.20796/) (2026)
- [Recovery Guarantees for Continual Learning of Dependent Tasks: Memory, Data-Dependent Regularization, and Data-Dependent Weights](https://mlsanity.com/paper/2604.17578/) (2026)

## Abstract (arXiv)

We propose a continuous learning formulation for T non-forgetting tasks with the spherical parameter space mathbbSd-1(√d) in asymptotic regime drightarrow ∞. Each task i has an optimum bmThetai, minimizing its corresponding loss function Ri(bmw) on sphere. We reduce each task i to an acceptable loss region modeled by a geodesic ball B_bmThetai(ri(d)) on mathbbSd-1(√d) centered at bmThetai with radius ri(d). To model task dependencies (similarities), task optima bmThetai are distributed by a Neyman-Scott clustered point process Phi. Cluster centers bmYjj=1Np form a homogeneous Poisson point process on sphere, while intra-cluster task optima follow a von Mises-Fisher distribution with mean bmYj and concentration kappa. Each task t∈ 1, ..., T learning algorithm is considered to be a channel Qt(dbmwt | bmwt-1, Kt-1(bmθ1:t-1), bmθt), where bmWt is the learned parameter of task t and Kt-1(bmTheta1:t-1)=bigcapi=1t-1B_bmThetai(ri(d)). Since in continual learning, the learner does not need to reach bmThetat exactly to successfully learn task t, to avoid forgetting as well as reliable learning, it only needs for the learned parameter vector bmWt to enter B_bmThetat(rt(d)) cap Kt-1(bmTheta1:t-1). Using this model, we derive an upper bound on T ensuring mathbbP[KT(bmTheta1:T) neq emptyset] rightarrow 1 as drightarrow ∞ and kappa/d rightarrow α, where α>0 is a constant. Finally, for T tasks sampled uniformly from Phi, we derive an upper bound for total information theoretic capacity under the non-forgetting constraint for each task.