Realization Theory Of Recurrent Neural ODEs Using Polynomial System Embeddings
Not on the map. Why?
Abstract (arXiv)
In this paper we show that neural ODE analogs of recurrent (ODE-RNN) and Long Short-Term Memory (ODE-LSTM) networks can be algorithmically embeddeded into the class of polynomial systems. This embedding preserves input-output behavior and can suitably be extended to other neural DE architectures. We then use realization theory of polynomial systems to provide necessary conditions for an input-output map to be realizable by an ODE-LSTM and sufficient conditions for minimality of such systems. These results represent the first steps towards realization theory of recurrent neural ODE architectures, which is is expected be useful for model reduction and learning algorithm analysis of recurrent neural ODEs.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 13 of its 55 references that are in the map. First posted on arXiv 24 May 2022.
Builds on
13 of 55 references.
References · 13
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
- 2019Deep Equilibrium Modelscited by 603
- 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
- 2022It's Raw! Audio Generation with State-Space Modelscited by 171
- 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
- 2021Long Expressive Memory for Sequence Modelingcited by 41
- 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
- 2021Framing RNN as a kernel method: A neural ODE approachcited by 23
- 2020Learning Recurrent Neural Net Models of Nonlinear Systemscited by 11
1 more
Built on by
Citing papers per year: 2022: 1.
1 paper.