mlsanityData to 14 Oct 2026

Continuous-Time Machine Learning: A Unified Mathematical Perspective

arXiv 2609.16710 ↗First posted 15 Sep 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Continuous-time (CT) machine learning has emerged as a principled framework for modeling temporal dynamics as a continuous process, particularly when observations are sampled at arbitrary time points or span long-range horizons. However, major branches of CT machine learning have matured in separate research communities, leaving their mathematical relationships and design trade-offs insufficiently characterized. In this survey, we develop a unified, concept-driven view of major CT machine learning branches through a taxonomy that organizes families according to their underlying base mathematical formulations. We present a canonical mathematical formulation that relates these families through different architectural choices of vector-field parameterization, stochasticity, memory mechanisms, and discretization. We compare training algorithms, optimization strategies, and failure modes, highlighting the trade-offs across families. We further provide a comparative analysis of theoretical computational complexity alongside an illustrative architecture-controlled benchmark analysis on representative architectures from each family. We also review software ecosystems supporting their implementation. Finally, we identify open challenges in approximation theory, training stability, hardware-efficient implementations, benchmarking, foundation models, and scientific machine learning, and discuss an agenda for future research.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 90 of its 128 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

90 of 128 references.

References · 90

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  4. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  5. 2018Neural Ordinary Differential Equationscited by 3,779
  6. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,908
  7. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  8. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  9. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  10. 2024Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Dualitycited by 1,336
  11. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  12. 2024VMamba: Visual State Space Modelcited by 969
38 more
  1. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  2. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  3. 2019Hamiltonian Neural Networkscited by 712
  4. 2023A decoder-only foundation model for time-series forecastingcited by 692
  5. 2023Retentive Network: A Successor to Transformer for Large Language Modelscited by 613
  6. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 608
  7. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 605
  8. 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
  9. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 506
  10. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  11. 2022Transformers in Time Series: A Surveycited by 497
  12. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 424
  13. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 408
  14. 2024Jamba: A Hybrid Transformer-Mamba Language Modelcited by 401
  15. 2020Lagrangian Neural Networkscited by 398
  16. 2019Augmented Neural ODEscited by 397
  17. 2020Universal Differential Equations for Scientific Machine Learningcited by 353
  18. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  19. 2020Scalable Gradients for Stochastic Differential Equationscited by 271
  20. 2022On Neural Differential Equationscited by 268
  21. 2022Long Range Language Modeling via Gated State Spacescited by 265
  22. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 242
  23. 2021Multi-Time Attention Networks for Irregularly Sampled Time Seriescited by 238
  24. 2019GRU-ODE-Bayes: Continuous modeling of sporadically-observed time seriescited by 231
  25. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  26. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 176
  27. 2024Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modelingcited by 174
  28. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  29. 2020Liquid Time-constant Networkscited by 161
  30. 2019Graph Neural Ordinary Differential Equationscited by 154
  31. 2019Neural Jump Stochastic Differential Equationscited by 150
  32. 2020Neural Rough Differential Equations for Long Time Seriescited by 143
  33. 2022Liquid Structural State-Space Modelscited by 136
  34. 2023Lag-Llama: Towards Foundation Models for Probabilistic Time Series Forecastingcited by 113
  35. 2021Neural Flows: Efficient Alternative to Neural ODEscited by 101
  36. 2016Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequencescited by 99
  37. 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 84
  38. 2024Hyper-Connectionscited by 84

50 most cited of 90 All 90 (JSON)

Built on by

Not cited yet.