# Continuous-Time Piecewise-Linear Recurrent Neural Networks

arXiv: https://arxiv.org/abs/2602.15649 · HTML: https://mlsanity.com/paper/2602.15649/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 94 references that are in the map.
- First posted on arXiv 17 Feb 2026.

## Builds on (most cited first, 30 of 36)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Hamiltonian Neural Networks](https://mlsanity.com/paper/1906.01563/) (2019)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Augmented Neural ODEs](https://mlsanity.com/paper/1904.01681/) (2019)
- [Modern Koopman Theory for Dynamical Systems](https://mlsanity.com/paper/2102.12086/) (2021)
- [Scalable Gradients for Stochastic Differential Equations](https://mlsanity.com/paper/2001.01328/) (2020)
- [How to train your neural ODE: the world of Jacobian and kinetic regularization](https://mlsanity.com/paper/2002.02798/) (2020)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limit](https://mlsanity.com/paper/1905.09883/) (2019)
- [Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networks](https://mlsanity.com/paper/1802.07486/) (2018)
- [On Second Order Behaviour in Augmented Neural ODEs](https://mlsanity.com/paper/2006.07220/) (2020)
- [On the difficulty of learning chaotic dynamics with RNNs](https://mlsanity.com/paper/2110.07238/) (2021)
- [STEER: Simple Temporal Regularization For Neural ODEs](https://mlsanity.com/paper/2006.10711/) (2020)
- [Generalized Teacher Forcing for Learning Chaotic Dynamics](https://mlsanity.com/paper/2306.04406/) (2023)
- [Neural Controlled Differential Equations for Online Prediction Tasks](https://mlsanity.com/paper/2106.11028/) (2021)
- [Learning Dissipative Dynamics in Chaotic Systems](https://mlsanity.com/paper/2106.06898/) (2021)
- [Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRI](https://mlsanity.com/paper/1902.07186/) (2019)
- [Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systems](https://mlsanity.com/paper/2207.02542/) (2022)
- [Recurrent switching linear dynamical systems](https://mlsanity.com/paper/1610.08466/) (2016)
- [Out-of-Domain Generalization in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2402.18377/) (2024)
- [A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamics](https://mlsanity.com/paper/2201.08910/) (2022)
- [A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurements](https://mlsanity.com/paper/1612.07846/) (2016)
- [Generative learning for nonlinear dynamics](https://mlsanity.com/paper/2311.04128/) (2023)
- [True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statistics](https://mlsanity.com/paper/2505.13192/) (2025)
- [Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstruction](https://mlsanity.com/paper/2410.14240/) (2024)
- [DyNODE: Neural Ordinary Differential Equations for Dynamics Modeling in Continuous Control](https://mlsanity.com/paper/2009.04278/) (2020)
- [Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systems](https://mlsanity.com/paper/2111.01256/) (2021)
- [Bifurcations and loss jumps in RNN training](https://mlsanity.com/paper/2310.17561/) (2023)
- [Inferring stochastic low-rank recurrent neural networks from neural data](https://mlsanity.com/paper/2406.16749/) (2024)
- [Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamics](https://mlsanity.com/paper/2212.07892/) (2022)

## Built on by (most cited first, 4 of 4)

- [Position: A Dynamical Systems Perspective is Needed to Advance Time Series Modeling](https://mlsanity.com/paper/2602.16864/) (2026)
- [A Minimal Interpretable Architecture for Zero-Shot Reconstruction of Dynamical Systems](https://mlsanity.com/paper/2607.14937/) (2026)
- [Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training](https://mlsanity.com/paper/2608.30960/) (2026)
- [Continuous-Time Machine Learning: A Unified Mathematical Perspective](https://mlsanity.com/paper/2609.16710/) (2026)

## Abstract (arXiv)

In dynamical systems reconstruction (DSR) we aim to recover the dynamical system (DS) underlying observed time series. Specifically, we aim to learn a generative surrogate model which approximates the underlying, data-generating DS, and recreates its long-term properties (`climate statistics'). In scientific and medical areas, in particular, these models need to be mechanistically tractable -- through their mathematical analysis we would like to obtain insight into the recovered system's workings. Piecewise-linear (PL), ReLU-based RNNs (PLRNNs) have a strong track-record in this regard, representing SOTA DSR models while allowing mathematical insight by virtue of their PL design. However, all current PLRNN variants are discrete-time maps. This is in disaccord with the assumed continuous-time nature of most physical and biological processes, and makes it hard to accommodate data arriving at irregular temporal intervals. Neural ODEs are one solution, but they do not reach the DSR performance of PLRNNs and often lack their tractability. Here we develop theory for continuous-time PLRNNs (cPLRNNs): We present a novel algorithm for training and simulating such models, bypassing numerical integration by efficiently exploiting their PL structure. We further demonstrate how important topological objects like equilibria or limit cycles can be determined semi-analytically in trained models. We compare cPLRNNs to both their discrete-time cousins as well as Neural ODEs on DSR benchmarks, including systems with discontinuities which come with hard thresholds.