# NAIS-Net: Stable Deep Networks from Non-Autonomous Differential Equations

arXiv: https://arxiv.org/abs/1804.07209 · HTML: https://mlsanity.com/paper/1804.07209/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Apr 2018.

## Built on by (most cited first, 7 of 7)

- [Neural Flows: Efficient Alternative to Neural ODEs](https://mlsanity.com/paper/2110.13040/) (2021)
- [Signature Methods in Machine Learning](https://mlsanity.com/paper/2206.14674/) (2022)
- [End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking](https://mlsanity.com/paper/2202.05826/) (2022)
- [Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules](https://mlsanity.com/paper/2206.01649/) (2022)
- [On the Stochastic Stability of Deep Markov Models](https://mlsanity.com/paper/2111.04601/) (2021)
- [Residual networks classify inputs based on their neural transient dynamics](https://mlsanity.com/paper/2101.03009/) (2021)
- [Robust Convolution Neural ODEs via Contractivity-promoting regularization](https://mlsanity.com/paper/2508.11432/) (2025)

## Abstract (arXiv)

This paper introduces Non-Autonomous Input-Output Stable Network(NAIS-Net), a very deep architecture where each stacked processing block is derived from a time-invariant non-autonomous dynamical system. Non-autonomy is implemented by skip connections from the block input to each of the unrolled processing stages and allows stability to be enforced so that blocks can be unrolled adaptively to a pattern-dependent processing depth. NAIS-Net induces non-trivial, Lipschitz input-output maps, even for an infinite unroll length. We prove that the network is globally asymptotically stable so that for every initial condition there is exactly one input-dependent equilibrium assuming tanh units, and incrementally stable for ReL units. An efficient implementation that enforces the stability under derived conditions for both fully-connected and convolutional layers is also presented. Experimental results show how NAIS-Net exhibits stability in practice, yielding a significant reduction in generalization gap compared to ResNets.