mlsanityData to 7 Oct 2026

Modeling Irregular Time Series with Continuous Recurrent Units

arXiv 2111.11344 ↗First posted 22 Nov 2021cs.LG stat.MLcited by 101

Not on the map. Why?

Abstract (arXiv)

Recurrent neural networks (RNNs) are a popular choice for modeling sequential data. Modern RNN architectures assume constant time-intervals between observations. However, in many datasets (e.g. medical records) observation times are irregular and can carry important information. To address this challenge, we propose continuous recurrent units (CRUs) -- a neural architecture that can naturally handle irregular intervals between observations. The CRU assumes a hidden state, which evolves according to a linear stochastic differential equation and is integrated into an encoder-decoder framework. The recursive computations of the CRU can be derived using the continuous-discrete Kalman filter and are in closed form. The resulting recurrent architecture has temporal continuity between hidden states and a gating mechanism that can optimally integrate noisy observations. We derive an efficient parameterization scheme for the CRU that leads to a fast implementation f-CRU. We empirically study the CRU on a number of challenging datasets and find that it can interpolate irregular time series better than methods based on neural ordinary differential equations.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 22 Nov 2021.

Builds on

35 of 61 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  7. 2016Recurrent Neural Networks for Multivariate Time Series with Missing Valuescited by 548
  8. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  9. 2015A Recurrent Latent Variable Model for Sequential Datacited by 362
  10. 2018BRITS: Bidirectional Recurrent Imputation for Time Seriescited by 288
  11. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  12. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 234
23 more
  1. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 230
  2. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 182
  3. 2015Deep Kalman Filterscited by 163
  4. 2021Neural SDEs as Infinite-Dimensional GANscited by 159
  5. 2016Deep Variational Bayes Filters: Unsupervised Learning of State Space Models from Raw Datacited by 154
  6. 2019Neural Jump Stochastic Differential Equationscited by 147
  7. 2020Neural Rough Differential Equations for Long Time Seriescited by 140
  8. 2019Set Functions for Time Seriescited by 137
  9. 2016Sequential Neural Models with Stochastic Layerscited by 115
  10. 2017A Disentangled Recognition and Nonlinear Dynamics Model for Unsupervised Learningcited by 107
  11. 2020Learning Long-Term Dependencies in Irregularly-Sampled Time Seriescited by 82
  12. 2016Modeling Missing Data in Clinical Time Series with RNNscited by 80
  13. 2019Trivializations for Gradient-Based Optimization on Manifoldscited by 78
  14. 2018MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcarecited by 72
  15. 2019Recurrent Kalman Networks: Factorized Inference in High-Dimensional Deep Feature Spacescited by 64
  16. 2014Learning Stochastic Recurrent Networkscited by 60
  17. 2017Z-Forcing: Training Stochastic Recurrent Networkscited by 51
  18. 2020A Survey on Principles, Models and Methods for Learning from Irregularly Sampled Time Seriescited by 42
  19. 2020Modeling Continuous Stochastic Processes with Dynamic Normalizing Flowscited by 37
  20. 2020Neural Jump Ordinary Differential Equations: Consistent Continuous-Time Prediction and Filteringcited by 33
  21. 2017Discrete Event, Continuous Time RNNscited by 31
  22. 2021Continuous Latent Process Flowscited by 10
  23. 2018Deep State Space Models for Unconditional Word Generationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 15; 2024: 27; 2025: 25; 2026: 28.

101 papers.

Sort:

By area:

  1. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  2. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 97
  3. 2024ContiFormer: Continuous-Time Transformer for Irregular Time Series Modelingcited by 61
  4. 2025Rethinking Irregular Time Series Forecasting: A Simple yet Effective Baselinecited by 54
  5. 2024Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Seriescited by 42
  6. 2022Deep Latent State Space Models for Time-Series Generationcited by 37
  7. 2024Scalable Event-by-event Processing of Neuromorphic Sensory Signals With Deep State-Space Modelscited by 31
  8. 2023Neural Continuous-Discrete State Space Models for Irregularly-Sampled Time Seriescited by 28
  9. 2024Universal Time-Series Representation Learning: A Surveycited by 28
  10. 2025MIRA: Medical Time Series Foundation Model for Real-World Health Datacited by 21
  11. 2023Irregular Traffic Time Series Forecasting Based on Asynchronous Spatio-Temporal Graph Convolutional Networkcited by 19
  12. 2024S7: Selective and Simplified State Space Layers for Sequence Modelingcited by 17
38 more
  1. 2022Markovian Gaussian Process Variational Autoencoderscited by 16
  2. 2023Latent SDEs on Homogeneous Spacescited by 16
  3. 2024Rough Transformers: Lightweight and Continuous Time Series Modelling through Signature Patchingcited by 16
  4. 2025Time-IMM: A Dataset and Benchmark for Irregular Multimodal Multivariate Time Seriescited by 16
  5. 2024Continuous Temporal Domain Generalizationcited by 14
  6. 2024Graph Neural Flows for Unveiling Systemic Interactions Among Irregularly Sampled Time Seriescited by 14
  7. 2025HyperIMTS: Hypergraph Neural Network for Irregular Multivariate Time Series Forecastingcited by 14
  8. 2023Revisiting Structured Variational Autoencoderscited by 11
  9. 2024Unleashing The Power of Pre-Trained Language Models for Irregularly Sampled Time Seriescited by 10
  10. 2024TimeCHEAT: A Channel Harmony Strategy for Irregularly Sampled Multivariate Time Series Analysiscited by 10
  11. 2025Physiome-ODE: A Benchmark for Irregularly Sampled Multivariate Time Series Forecasting Based on Biological ODEscited by 10
  12. 2024Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Seriescited by 9
  13. 2025LSCD: Lomb-Scargle Conditioned Diffusion for Time series Imputationcited by 8
  14. 2026Bridging Time and Frequency: A Joint Modeling Framework for Irregular Multivariate Time Series Forecastingcited by 8
  15. 2023Temporal Graph Neural Networks for Irregular Datacited by 7
  16. 2023PGODE: Towards High-quality System Dynamics Modelingcited by 7
  17. 2025ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecastingcited by 7
  18. 2025A Diffusion Model for Regular Time Series Generation from Irregular Data with Completion and Maskingcited by 7
  19. 2024SMR: State Memory Replay for Long Sequence Modelingcited by 6
  20. 2024Temporal Test-Time Adaptation with State-Space Modelscited by 6
  21. 2025S4M: S4 for multivariate time series forecasting with Missing valuescited by 6
  22. 2023IVP-VAE: Modeling EHR Time Series with Initial Value Problem Solverscited by 5
  23. 2024How Deep is your Guess? A Fresh Perspective on Deep Learning for Medical Time-Series Imputationcited by 5
  24. 2025LabTOP: A Unified Model for Lab Test Outcome Prediction on Electronic Health Recordscited by 5
  25. 2025Revitalizing Canonical Pre-Alignment for Irregular Multivariate Time Series Forecastingcited by 5
  26. 2022SeqLink: A Robust Neural-ODE Architecture for Modelling Partially Observed Time Seriescited by 4
  27. 2024Irregularity-Informed Time Series Analysis: Adaptive Modelling of Spatial and Temporal Dynamicscited by 4
  28. 2025LAST SToP For Modeling Asynchronous Time Seriescited by 4
  29. 2025Rotary Masked Autoencoders are Versatile Learnerscited by 4
  30. 2023Learning PDE Solution Operator for Continuous Modeling of Time-Seriescited by 3
  31. 2024System-Aware Neural ODE Processes for Few-Shot Bayesian Optimizationcited by 3
  32. 2025Neural Predictive Control to Coordinate Discrete- and Continuous-Time Models for Time-Series Analysis with Control-Theoretical Improvementscited by 3
  33. 2023Finding Short Signals in Long Irregular Time Series with Continuous-Time Attention Policy Networkscited by 2
  34. 2023Hybrid Modeling Design Patternscited by 2
  35. 2025IMTS is Worth Time × Channel Patches: Visual Masked Autoencoders for Irregular Multivariate Time Series Predictioncited by 2
  36. 2025MTM: A Multi-Scale Token Mixing Transformer for Irregular Multivariate Time Series Classificationcited by 2
  37. 2026MILM: Large Language Models for Multimodal Irregular Time Series with Informative Samplingcited by 2
  38. 2022Continuous Forecasting via Neural Eigen Decompositioncited by 1

50 most cited of 101 All 101 (JSON)