# Enhancing Continuous Time Series Modelling with a Latent ODE-LSTM Approach

arXiv: https://arxiv.org/abs/2307.05126 · HTML: https://mlsanity.com/paper/2307.05126/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 11 Jul 2023.

## Builds on (most cited first, 4 of 4)

- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Learning Long-Term Dependencies in Irregularly-Sampled Time Series](https://mlsanity.com/paper/2006.04418/) (2020)

## Built on by (most cited first, 9 of 9)

- [The Seismic Wavefield Common Task Framework](https://mlsanity.com/paper/2512.19927/) (2025)
- [CTF4Nuclear: Common Task Framework for Nuclear Fission and Fusion Models](https://mlsanity.com/paper/2605.15549/) (2026)
- [Introducing Interval Neural Networks for Uncertainty-Aware System Identification](https://mlsanity.com/paper/2504.18845/) (2025)
- [Stable-by-Design Neural Network-Based LPV State-Space Models for System Identification](https://mlsanity.com/paper/2510.24757/) (2025)
- [A deep learning framework for jointly solving transient Fokker-Planck equations with arbitrary parameters and initial distributions](https://mlsanity.com/paper/2604.06001/) (2026)
- [Physics-informed reduced-order modelling with equivariant spectral submanifolds](https://mlsanity.com/paper/2608.04239/) (2026)
- [Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatment](https://mlsanity.com/paper/2506.15019/) (2025)
- [Data-Driven Temperature Modelling of Machine Tools by Neural Networks: A Benchmark](https://mlsanity.com/paper/2510.03261/) (2025)
- [Beyond Prediction: Interval Neural Networks for Uncertainty-Aware System Identification](https://mlsanity.com/paper/2605.11460/) (2026)

## Abstract (arXiv)

Due to their dynamic properties such as irregular sampling rate and high-frequency sampling, Continuous Time Series (CTS) are found in many applications. Since CTS with irregular sampling rate are difficult to model with standard Recurrent Neural Networks (RNNs), RNNs have been generalised to have continuous-time hidden dynamics defined by a Neural Ordinary Differential Equation (Neural ODE), leading to the ODE-RNN model. Another approach that provides a better modelling is that of the Latent ODE model, which constructs a continuous-time model where a latent state is defined at all times. The Latent ODE model uses a standard RNN as the encoder and a Neural ODE as the decoder. However, since the RNN encoder leads to difficulties with missing data and ill-defined latent variables, a Latent ODE-RNN model has recently been proposed that uses a ODE-RNN model as the encoder instead. Both the Latent ODE and Latent ODE-RNN models are difficult to train due to the vanishing and exploding gradients problem. To overcome this problem, the main contribution of this paper is to propose and illustrate a new model based on a new Latent ODE using an ODE-LSTM (Long Short-Term Memory) network as an encoder -- the Latent ODE-LSTM model. To limit the growth of the gradients the Norm Gradient Clipping strategy was embedded on the Latent ODE-LSTM model. The performance evaluation of the new Latent ODE-LSTM (with and without Norm Gradient Clipping) for modelling CTS with regular and irregular sampling rates is then demonstrated. Numerical experiments show that the new Latent ODE-LSTM performs better than Latent ODE-RNNs and can avoid the vanishing and exploding gradients during training.