Continuous-Time Piecewise-Linear Recurrent Neural Networks
Not on the map. Why?
Abstract (arXiv)
In dynamical systems reconstruction (DSR) we aim to recover the dynamical system (DS) underlying observed time series. Specifically, we aim to learn a generative surrogate model which approximates the underlying, data-generating DS, and recreates its long-term properties (`climate statistics'). In scientific and medical areas, in particular, these models need to be mechanistically tractable -- through their mathematical analysis we would like to obtain insight into the recovered system's workings. Piecewise-linear (PL), ReLU-based RNNs (PLRNNs) have a strong track-record in this regard, representing SOTA DSR models while allowing mathematical insight by virtue of their PL design. However, all current PLRNN variants are discrete-time maps. This is in disaccord with the assumed continuous-time nature of most physical and biological processes, and makes it hard to accommodate data arriving at irregular temporal intervals. Neural ODEs are one solution, but they do not reach the DSR performance of PLRNNs and often lack their tractability. Here we develop theory for continuous-time PLRNNs (cPLRNNs): We present a novel algorithm for training and simulating such models, bypassing numerical integration by efficiently exploiting their PL structure. We further demonstrate how important topological objects like equilibria or limit cycles can be determined semi-analytically in trained models. We compare cPLRNNs to both their discrete-time cousins as well as Neural ODEs on DSR benchmarks, including systems with discontinuities which come with hard thresholds.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 36 of its 94 references that are in the map. First posted on arXiv 17 Feb 2026.
Builds on
36 of 94 references.
References · 36
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2019Hamiltonian Neural Networkscited by 702
- 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
- 2019Augmented Neural ODEscited by 396
- 2021Modern Koopman Theory for Dynamical Systemscited by 269
- 2020Scalable Gradients for Stochastic Differential Equationscited by 264
- 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
- 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 183
- 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
- 2018Data-Driven Forecasting of High-Dimensional Chaotic Systems with Long Short-Term Memory Networkscited by 155
- 2020On Second Order Behaviour in Augmented Neural ODEscited by 68
- 2021On the difficulty of learning chaotic dynamics with RNNscited by 64
24 more
- 2020STEER: Simple Temporal Regularization For Neural ODEscited by 58
- 2023Generalized Teacher Forcing for Learning Chaotic Dynamicscited by 55
- 2021Neural Controlled Differential Equations for Online Prediction Taskscited by 46
- 2021Learning Dissipative Dynamics in Chaotic Systemscited by 41
- 2019Identifying nonlinear dynamical systems via generative recurrent neural networks with applications to fMRIcited by 39
- 2016Recurrent switching linear dynamical systemscited by 34
- 2022Tractable Dendritic RNNs for Reconstructing Nonlinear Dynamical Systemscited by 34
- 2024Out-of-Domain Generalization in Dynamical Systems Reconstructioncited by 33
- 2022A Systematic Exploration of Reservoir Computing for Forecasting Complex Spatiotemporal Dynamicscited by 30
- 2016A State Space Approach for Piecewise-Linear Recurrent Neural Networks for Reconstructing Nonlinear Dynamics from Neural Measurementscited by 24
- 2023Generative learning for nonlinear dynamicscited by 22
- 2025True Zero-Shot Inference of Dynamical Systems Preserving Long-Term Statisticscited by 20
- 2024Almost-Linear RNNs Yield Highly Interpretable Symbolic Codes in Dynamical Systems Reconstructioncited by 18
- 2020DyNODE: Neural Ordinary Differential Equations for Dynamics Modeling in Continuous Controlcited by 17
- 2021Reverse engineering recurrent neural networks with Jacobian switching linear dynamical systemscited by 17
- 2023Bifurcations and loss jumps in RNN trainingcited by 17
- 2024Inferring stochastic low-rank recurrent neural networks from neural datacited by 15
- 2022Integrating Multimodal Data for Joint Generative Modeling of Complex Dynamicscited by 13
- 2024Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methodscited by 11
- 2020Variational Inference and Learning of Piecewise-linear Dynamical Systemscited by 8
- 2023Oscillatory networks: Insights from piecewise-linear modelingcited by 4
- 2025Uncovering the Computational Roles of Nonlinearity in Sequence Modeling Using Almost-Linear RNNscited by 4
- 2025Detecting Invariant Manifolds in ReLU-Based RNNscited by 3
- 2026Universal Approximation Theorems for Dynamical Systems with Infinite-Time Horizon Guaranteescited by 2
Built on by
Citing papers per year: 2026: 3.
3 papers.