# Control Theoretic Approach to Fine-Tuning and Transfer Learning

arXiv: https://arxiv.org/abs/2404.11013 · HTML: https://mlsanity.com/paper/2404.11013/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 17 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Explicit Inductive Bias for Transfer Learning with Convolutional Networks](https://mlsanity.com/paper/1802.01483/) (2018)
- [Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations](https://mlsanity.com/paper/1710.10121/) (2017)
- [Training Networks in Null Space of Feature Covariance for Continual Learning](https://mlsanity.com/paper/2103.07113/) (2021)
- [DELTA: DEep Learning Transfer using Feature Map with Attention for Convolutional Networks](https://mlsanity.com/paper/1901.09229/) (2019)
- [Distance-Based Regularisation of Deep Networks for Fine-Tuning](https://mlsanity.com/paper/2002.08253/) (2020)
- [Trainable Projected Gradient Method for Robust Fine-tuning](https://mlsanity.com/paper/2303.10720/) (2023)
- [Deep neural networks, generic universal interpolation, and controlled ODEs](https://mlsanity.com/paper/1908.07838/) (2019)
- [Control in the spaces of ensembles of points](https://mlsanity.com/paper/1907.00905/) (2019)

## Built on by (most cited first, 3 of 3)

- [Geometric Foundations of Tuning without Forgetting in Neural ODEs](https://mlsanity.com/paper/2509.03474/) (2025)
- [Control Disturbance Rejection in Neural ODEs](https://mlsanity.com/paper/2509.18034/) (2025)
- [Privacy Preserving Gossip Learning](https://mlsanity.com/paper/2609.14778/) (2026)

## Abstract (arXiv)

Given a training set in the form of a paired (X,Y), we say that the control system dot x = f(x,u) has learned the paired set via the control u^* if the system steers each point of X to its corresponding target in Y. If the training set is expanded, most existing methods for finding a new control u^* require starting from scratch, resulting in a quadratic increase in complexity with the number of points. To overcome this limitation, we introduce the concept of textit tuning without forgetting. We develop textitan iterative algorithm to tune the control u^* when the training set expands, whereby points already in the paired set are still matched, and new training samples are learned. At each update of our method, the control u^* is projected onto the kernel of the end-point mapping generated by the controlled dynamics at the learned samples. It ensures keeping the end-points for the previously learned samples constant while iteratively learning additional samples.