# STEER: Simple Temporal Regularization For Neural ODEs

arXiv: https://arxiv.org/abs/2006.10711 · HTML: https://mlsanity.com/paper/2006.10711/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2020.

## Built on by (most cited first, 30 of 58)

- [Neural Flows: Efficient Alternative to Neural ODEs](https://mlsanity.com/paper/2110.13040/) (2021)
- [Stiff Neural Ordinary Differential Equations](https://mlsanity.com/paper/2103.15341/) (2021)
- [Heavy Ball Neural Ordinary Differential Equations](https://mlsanity.com/paper/2110.04840/) (2021)
- [Deconstructing the Inductive Biases of Hamiltonian Neural Networks](https://mlsanity.com/paper/2202.04836/) (2022)
- [Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics](https://mlsanity.com/paper/2105.03918/) (2021)
- [Neural Laplace: Learning diverse classes of differential equations in the Laplace domain](https://mlsanity.com/paper/2206.04843/) (2022)
- [Neural Integral Equations](https://mlsanity.com/paper/2209.15190/) (2022)
- [Comprehensive Review of Neural Differential Equations for Time Series Analysis](https://mlsanity.com/paper/2502.09885/) (2025)
- [Stabilized Neural Differential Equations for Learning Dynamics with Explicit Constraints](https://mlsanity.com/paper/2306.09739/) (2023)
- [When are dynamical systems learned from time series data statistically accurate?](https://mlsanity.com/paper/2411.06311/) (2024)
- [Homotopy-based training of NeuralODEs for accurate dynamics discovery](https://mlsanity.com/paper/2210.01407/) (2022)
- [On latent dynamics learning in nonlinear reduced order modeling](https://mlsanity.com/paper/2408.15183/) (2024)
- [Taylor-Lagrange Neural Ordinary Differential Equations: Toward Fast Training and Evaluation of Neural ODEs](https://mlsanity.com/paper/2201.05715/) (2022)
- [Neural Markov Jump Processes](https://mlsanity.com/paper/2305.19744/) (2023)
- [Second-Order Neural ODE Optimizer](https://mlsanity.com/paper/2109.14158/) (2021)
- [Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods](https://mlsanity.com/paper/2410.05592/) (2024)
- [How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies](https://mlsanity.com/paper/2110.07034/) (2021)
- [torchode: A Parallel ODE Solver for PyTorch](https://mlsanity.com/paper/2210.12375/) (2022)
- [Sparsity in Continuous-Depth Neural Networks](https://mlsanity.com/paper/2210.14672/) (2022)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Simulation-Free Training of Neural ODEs on Paired Data](https://mlsanity.com/paper/2410.22918/) (2024)
- [Unification of Symmetries Inside Neural Networks: Transformer, Feedforward and Neural ODE](https://mlsanity.com/paper/2402.02362/) (2024)
- [Learning Hamiltonian neural Koopman operator and simultaneously sustaining and discovering conservation law](https://mlsanity.com/paper/2406.02154/) (2024)
- [Bidirectional Normalizing Flow: From Data to Noise and Back](https://mlsanity.com/paper/2512.10953/) (2025)
- [Novel DNNs for Stiff ODEs with Applications to Chemically Reacting Flows](https://mlsanity.com/paper/2104.01914/) (2021)
- [Proximal Implicit ODE Solvers for Accelerating Learning Neural ODEs](https://mlsanity.com/paper/2204.08621/) (2022)
- [Continuous Deep Equilibrium Models: Training Neural ODEs faster by integrating them to Infinity](https://mlsanity.com/paper/2201.12240/) (2022)
- [LaTiM: Longitudinal representation learning in continuous-time models to predict disease progression](https://mlsanity.com/paper/2404.07091/) (2024)
- [Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methods](https://mlsanity.com/paper/2412.01181/) (2024)
- [Multi-Resolution Continuous Normalizing Flows](https://mlsanity.com/paper/2106.08462/) (2021)

## Abstract (arXiv)

Training Neural Ordinary Differential Equations (ODEs) is often computationally expensive. Indeed, computing the forward pass of such models involves solving an ODE which can become arbitrarily complex during training. Recent works have shown that regularizing the dynamics of the ODE can partially alleviate this. In this paper we propose a new regularization technique: randomly sampling the end time of the ODE during training. The proposed regularization is simple to implement, has negligible overhead and is effective across a wide variety of tasks. Further, the technique is orthogonal to several other methods proposed to regularize the dynamics of ODEs and as such can be used in conjunction with them. We show through experiments on normalizing flows, time series models and image recognition that the proposed regularization can significantly decrease training time and even improve performance over baseline models.