# Gradient-Free Warm-Start Library Recovery: an Amortized-Regret Separation

arXiv: https://arxiv.org/abs/2606.21253 · HTML: https://mlsanity.com/paper/2606.21253/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 43 references that are in the map.
- First posted on arXiv 19 Jun 2026.

## Builds on (most cited first, 12 of 12)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [The Benefit of Multitask Representation Learning](https://mlsanity.com/paper/1505.06279/) (2015)
- [On the Theory of Transfer Learning: The Importance of Task Diversity](https://mlsanity.com/paper/2006.11650/) (2020)
- [Equipping Experts/Bandits with Long-term Memory](https://mlsanity.com/paper/1905.12950/) (2019)
- [Improved Regret Bounds for Tracking Experts with Memory](https://mlsanity.com/paper/2106.13021/) (2021)
- [Sequential algorithms for testing identity and closeness of distributions](https://mlsanity.com/paper/2205.06069/) (2022)
- [HiCL: Hippocampal-Inspired Continual Learning](https://mlsanity.com/paper/2508.16651/) (2025)
- [On the Learnability of Test-Time Adaptation: A Recovery Complexity Perspective](https://mlsanity.com/paper/2605.28057/) (2026)
- [Self-Evidencing Through Hierarchical Gradient Decomposition: A Dissipative System That Maintains Non-Equilibrium Steady-State by Minimizing Variational Free Energy](https://mlsanity.com/paper/2510.17916/) (2025)

## Abstract (arXiv)

Continual learning that is gradient-free, local, online, and append-only is attractive for edge and streaming deployment, but its value is usually argued informally. We give a provable account on recurring-regime streams. Given segmentation, a warm-start library learner attains amortized recovery cost O!big(KD/varepsilon2+(R-K)logK/Delta2big) versus a memoryless re-estimator's Theta(RD/varepsilon2), an advantage (R-K),Theta(D/varepsilon2) growing with dimension D and recurrence density. The mechanism is a decoupling: recognizing which of K seen regimes is active costs O(log K/Delta2), independent of D, whereas estimating a regime costs Theta(D/varepsilon2). We prove this is tight: matching lower bounds give recognition Theta(log K/Delta2) and a memoryless-class bound Omega(RD/varepsilon2), so each term is individually minimax-tight (the joint statement is conditional). The separation is born-immune (a memoryless learner's advantage is identically zero) and paradigm-level: it matches, and does not beat, a fair spawn-capable Bayesian baseline; the contribution is attaining this cost structure without end-to-end backprop and with zero forgetting by construction. A count-calibrated variant ties the baseline's leading constant up to a bounded, never-negative per-recurrence overshoot, hyperparameter-free and with no per-step transcendentals. We bound the scope: recognizable regimes are capped by simplex packing (walls eTheta(D)); autonomous segmentation is impossible at the packing wall (no detector escapes the false-alarm/delay frontier as regimes overlap); the advantage vanishes under overlap. The dimension-dependent separation is corroborated on synthetic streams and real k-mer genome distributions (memoryless cost propto D1.04, recognition D-independent); the one real sequential stream sits in the D=1 near-null corner.