# Contractivity of neural ODEs: an eigenvalue optimization problem

arXiv: https://arxiv.org/abs/2402.13092 · HTML: https://mlsanity.com/paper/2402.13092/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2024.

## Built on by (most cited first, 4 of 4)

- [Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods](https://mlsanity.com/paper/2410.05592/) (2024)
- [Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methods](https://mlsanity.com/paper/2412.01181/) (2024)
- [The Vanishing Gradient Problem for Stiff Neural Differential Equations](https://mlsanity.com/paper/2508.01519/) (2025)
- [Approximation properties of neural ODEs](https://mlsanity.com/paper/2503.15696/) (2025)

## Abstract (arXiv)

We propose a novel methodology to solve a key eigenvalue optimization problem which arises in the contractivity analysis of neural ODEs. When looking at contractivity properties of a one layer weight-tied neural ODE dotu(t)=σ(Au(t)+b) (with u,b ∈ mathbb Rn, A is a given n × n matrix, σ : mathbb R → mathbb R denotes an activation function and for a vector z ∈ mathbb Rn, σ(z) ∈ mathbb Rn has to be interpreted entry-wise), we are led to study the logarithmic norm of a set of products of type D A, where D is a diagonal matrix such that diag(D) ∈ σ'(mathbb Rn). Specifically, given a real number c (usually c=0), the problem consists in finding the largest positive interval Isubseteq mathbb [0,∞) such that the logarithmic norm μ(DA) ≤ c for all diagonal matrices D with Dii∈ I. We propose a two-level nested methodology: an inner level where, for a given I, we compute an optimizer D^star(I) by a gradient system approach, and an outer level where we tune I so that the value c is reached by μ(D^star(I)A). We extend the proposed two-level approach to the general multilayer, and possibly time-dependent, case dotu(t) = σ( Ak(t) ldots σ ( A1(t) u(t) + b1(t) ) ldots + bk(t) ) and we propose several numerical examples to illustrate its behaviour, including its stabilizing performance on a one-layer neural ODE applied to the classification of the MNIST handwritten digits dataset.