# Layerwise goal-oriented adaptivity for neural ODEs: an optimal control perspective

arXiv: https://arxiv.org/abs/2601.07397 · HTML: https://mlsanity.com/paper/2601.07397/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 12 Jan 2026.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Stable Architectures for Deep Neural Networks](https://mlsanity.com/paper/1705.03341/) (2017)
- [On Neural Differential Equations](https://mlsanity.com/paper/2202.02435/) (2022)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Neural ODE control for classification, approximation and transport](https://mlsanity.com/paper/2104.05278/) (2021)
- [Deep learning as optimal control problems: models and numerical methods](https://mlsanity.com/paper/1904.05657/) (2019)
- [Train Like a (Var)Pro: Efficient Training of Neural Networks with Variable Projection](https://mlsanity.com/paper/2007.13171/) (2020)
- [A descent algorithm for the optimal control of ReLU neural network informed PDEs based on approximate directional derivatives](https://mlsanity.com/paper/2210.07900/) (2022)
- [An Optimal Time Variable Learning Framework for Deep Neural Networks](https://mlsanity.com/paper/2204.08528/) (2022)
- [Strong stationarity for optimal control problems with non-smooth integral equation constraints: Application to continuous DNNs](https://mlsanity.com/paper/2302.05318/) (2023)
- [A multilevel proximal trust-region method for nonsmooth optimization with applications](https://mlsanity.com/paper/2512.00538/) (2025)

## Abstract (arXiv)

In this work, we propose a novel layerwise adaptive construction method for neural network architectures. Our approach is based on a goal--oriented dual-weighted residual technique for the optimal control of neural differential equations. This leads to an ordinary differential equation constrained optimization problem with controls acting as coefficients and a specific loss function. We implement our approach on the basis of a DG(0) Galerkin discretization of the neural ODE, leading to an explicit Euler time marching scheme. The resulting optimization problem is solved using the Adam algorithm and a BFGS method adapted to the H1 topology induced by the regularization term. Finally, we apply our method to the construction of neural networks for the classification of data sets, where we present results for a selection of well known examples from the literature.