# Recent Trends in Modelling the Continuous Time Series using Deep Learning: A Survey

arXiv: https://arxiv.org/abs/2409.09106 · HTML: https://mlsanity.com/paper/2409.09106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 174 references that are in the map.
- First posted on arXiv 13 Sep 2024.

## Builds on (most cited first, 30 of 32)

- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Neural Controlled Differential Equations for Irregular Time Series](https://mlsanity.com/paper/2005.08926/) (2020)
- [Time Series Forecasting With Deep Learning: A Survey](https://mlsanity.com/paper/2004.13408/) (2020)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Time Series Data Augmentation for Deep Learning: A Survey](https://mlsanity.com/paper/2002.12478/) (2020)
- [Symplectic ODE-Net: Learning Hamiltonian Dynamics with Control](https://mlsanity.com/paper/1909.12077/) (2019)
- [Physics Informed Deep Learning (Part II): Data-driven Discovery of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10566/) (2017)
- [Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations](https://mlsanity.com/paper/1811.02033/) (2018)
- [Learning to Control PDEs with Differentiable Physics](https://mlsanity.com/paper/2001.07457/) (2020)
- [Neural SDE: Stabilizing Neural ODE Networks with Stochastic Noise](https://mlsanity.com/paper/1906.02355/) (2019)
- [Quasi-Recurrent Neural Networks](https://mlsanity.com/paper/1611.01576/) (2016)
- [Reversible Architectures for Arbitrarily Deep Residual Neural Networks](https://mlsanity.com/paper/1709.03698/) (2017)
- [Physics-Guided Machine Learning for Scientific Discovery: An Application in Simulating Lake Temperature Profiles](https://mlsanity.com/paper/2001.11086/) (2020)
- [Deep Learning for Time-Series Analysis](https://mlsanity.com/paper/1701.01887/) (2017)
- [Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks](https://mlsanity.com/paper/1708.06834/) (2017)
- [NeuPDE: Neural Network Based Ordinary and Partial Differential Equations for Modeling Time-Dependent Data](https://mlsanity.com/paper/1908.03190/) (2019)
- [nPINNs: nonlocal Physics-Informed Neural Networks for a parametrized nonlocal universal Laplacian operator. Algorithms and Applications](https://mlsanity.com/paper/2004.04276/) (2020)
- [Convolutional Neural Networks combined with Runge-Kutta Methods](https://mlsanity.com/paper/1802.08831/) (2018)
- [An overview and comparative analysis of Recurrent Neural Networks for Short Term Load Forecasting](https://mlsanity.com/paper/1705.04378/) (2017)
- [Neural Ordinary Differential Equation based Recurrent Neural Network Model](https://mlsanity.com/paper/2005.09807/) (2020)
- [Temporal-Clustering Invariance in Irregular Healthcare Time Series](https://mlsanity.com/paper/1904.12206/) (2019)
- [Automatic Differentiation of Algorithms for Machine Learning](https://mlsanity.com/paper/1404.7456/) (2014)
- [Multi-resolution Networks For Flexible Irregular Time Series Modeling (Multi-FIT)](https://mlsanity.com/paper/1905.00125/) (2019)
- [Neural ODEs for Informative Missingness in Multivariate Time Series](https://mlsanity.com/paper/2005.10693/) (2020)
- [Physics Informed Data Driven model for Flood Prediction: Application of Deep Learning in prediction of urban flood development](https://mlsanity.com/paper/1908.10312/) (2019)
- [SciANN: A Keras/Tensorflow wrapper for scientific computations and physics-informed deep learning using artificial neural networks](https://mlsanity.com/paper/2005.08803/) (2020)
- [Synthetic Event Time Series Health Data Generation](https://mlsanity.com/paper/1911.06411/) (2019)

## Abstract (arXiv)

Continuous-time series is essential for different modern application areas, e.g. healthcare, automobile, energy, finance, Internet of things (IoT) and other related areas. Different application needs to process as well as analyse a massive amount of data in time series structure in order to determine the data-driven result, for example, financial trend prediction, potential probability of the occurrence of a particular event occurrence identification, patient health record processing and so many more. However, modeling real-time data using a continuous-time series is challenging since the dynamical systems behind the data could be a differential equation. Several research works have tried to solve the challenges of modelling the continuous-time series using different neural network models and approaches for data processing and learning. The existing deep learning models are not free from challenges and limitations due to diversity among different attributes, behaviour, duration of steps, energy, and data sampling rate. This paper has described the general problem domain of time series and reviewed the challenges of modelling the continuous time series. We have presented a comparative analysis of recent developments in deep learning models and their contribution to solving different difficulties of modelling the continuous time series. We have also identified the limitations of the existing neural network model and open issues. The main goal of this review is to understand the recent trend of neural network models used in a different real-world application with continuous-time data.