# Geometric Foundations of Tuning without Forgetting in Neural ODEs

arXiv: https://arxiv.org/abs/2509.03474 · HTML: https://mlsanity.com/paper/2509.03474/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 21 references that are in the map.
- First posted on arXiv 3 Sep 2025.

## Builds on (most cited first, 6 of 6)

- [A Mean-Field Optimal Control Formulation of Deep Learning](https://mlsanity.com/paper/1807.01083/) (2018)
- [Deep neural networks, generic universal interpolation, and controlled ODEs](https://mlsanity.com/paper/1908.07838/) (2019)
- [A Measure Theoretical Approach to the Mean-field Maximum Principle for Training NeurODEs](https://mlsanity.com/paper/2107.08707/) (2021)
- [Control in the spaces of ensembles of points](https://mlsanity.com/paper/1907.00905/) (2019)
- [A Shooting Formulation of Deep Learning](https://mlsanity.com/paper/2006.10330/) (2020)
- [Control Theoretic Approach to Fine-Tuning and Transfer Learning](https://mlsanity.com/paper/2404.11013/) (2024)

## Built on by (most cited first, 3 of 3)

- [Control Disturbance Rejection in Neural ODEs](https://mlsanity.com/paper/2509.18034/) (2025)
- [Privacy Preserving Gossip Learning](https://mlsanity.com/paper/2609.14778/) (2026)
- [Deep learning from a control vantage point](https://mlsanity.com/paper/2610.14385/) (2026)

## Abstract (arXiv)

In our earlier work, we introduced the principle of Tuning without Forgetting (TwF) for sequential training of neural ODEs, where training samples are added iteratively and parameters are updated within the subspace of control functions that preserves the end-point mapping at previously learned samples on the manifold of output labels in the first-order approximation sense. In this letter, we prove that this parameter subspace forms a Banach submanifold of finite codimension under nonsingular controls, and we characterize its tangent space. This reveals that TwF corresponds to a continuation/deformation of the control function along the tangent space of this Banach submanifold, providing a theoretical foundation for its mapping-preserving (not forgetting) during the sequential training exactly, beyond first-order approximation.