mlsanityData to 7 Oct 2026

Continuous-Time Model-Based Reinforcement Learning

arXiv 2102.04764 ↗First posted 9 Feb 2021cs.LG stat.MLcited by 39

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the underlying process, we propose a continuous-time MBRL framework based on a novel actor-critic method. Our approach also infers the unknown state evolution differentials with Bayesian neural ordinary differential equations (ODE) to account for epistemic uncertainty. We implement and test our method on a new ODE-RL suite that explicitly solves continuous-time control systems. Our experiments illustrate that the model is robust against irregular and noisy data, is sample-efficient, and can solve control problems which pose challenges to discrete-time MBRL methods.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 38 of its 68 references that are in the map. First posted on arXiv 9 Feb 2021.

Builds on

38 of 68 references.

References · 38

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2017Proximal Policy Optimization Algorithmscited by 14,441
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2018Neural Ordinary Differential Equationscited by 3,733
  7. 2015Continuous control with deep reinforcement learningcited by 3,279
  8. 2015Trust Region Policy Optimizationcited by 2,503
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  10. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  11. 2016Variational Inference: A Review for Statisticianscited by 1,466
  12. 2018World Modelscited by 1,444
26 more
  1. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  2. 2020Mastering Atari with Discrete World Modelscited by 990
  3. 2018DeepMind Control Suitecited by 935
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  6. 2019Hamiltonian Neural Networkscited by 702
  7. 2019Augmented Neural ODEscited by 396
  8. 2020Lagrangian Neural Networkscited by 388
  9. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  10. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 288
  11. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  12. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 214
  13. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 205
  14. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 183
  15. 2019Neural Jump Stochastic Differential Equationscited by 147
  16. 2019Functional Variational Bayesian Neural Networkscited by 127
  17. 2020Adaptive Checkpoint Adjoint Method for Gradient Estimation in Neural ODEcited by 76
  18. 2018Learning unknown ODE models with Gaussian processescited by 54
  19. 2019Making Deep Q-learning methods robust to time discretizationcited by 51
  20. 2017Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Controlcited by 50
  21. 2016Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networkscited by 49
  22. 2020Unsupervised Learning of Lagrangian Dynamics from Images for Prediction and Controlcited by 38
  23. 2019ODE2VAE: Deep generative second order ODEs with Bayesian neural networkscited by 36
  24. 2020Trust the Model When It Is Confident: Masked Model-based Actor-Criticcited by 36
  25. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  26. 2020Scalable Bayesian neural networks by layer-wise input augmentationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 10; 2023: 8; 2024: 7; 2025: 8; 2026: 4.

39 papers.

Sort:

By area:

  1. 2021Policy Gradient and Actor-Critic Learning in Continuous Time and Space: Theory and Algorithmscited by 76
  2. 2022Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processescited by 23
  3. 2023Efficient Exploration in Continuous-time Model-based Reinforcement Learningcited by 21
  4. 2023Robustifying State-space Models for Long Sequences via Approximate Diagonalizationcited by 18
  5. 2024Tuning Frequency Bias of State Space Modelscited by 16
  6. 2022Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rulescited by 15
  7. 2022Dynamic Decision Frequency with Continuous Optionscited by 11
  8. 2023Neural Laplace Control for Continuous-time Delayed Systemscited by 10
  9. 2024When to Sense and Control? A Time-adaptive Approach for Continuous-Time RLcited by 9
  10. 2025Accuracy of Discretely Sampled Stochastic Policies in Continuous-time Reinforcement Learningcited by 9
  11. 2024CL-DiffPhyCon: Closed-loop Diffusion Control of Complex Physical Systemscited by 8
  12. 2023Learning Energy Conserving Dynamics Efficiently with Hamiltonian Gaussian Processescited by 6
27 more
  1. 2023ODE-based Recurrent Model-free Reinforcement Learning for POMDPscited by 6
  2. 2022Learning Interacting Dynamical Systems with Latent Gaussian Process ODEscited by 5
  3. 2022Temporal Difference Learning with Continuous Time and State in the Stochastic Settingcited by 4
  4. 2022Neural ODEs as Feedback Policies for Nonlinear Optimal Controlcited by 4
  5. 2023Data-driven optimal stopping: A pure exploration analysiscited by 3
  6. 2025Optimal Control of Probabilistic Dynamics Models via Mean Hamiltonian Minimizationcited by 3
  7. 2025A Temporal Difference Method for Stochastic Continuous Dynamicscited by 3
  8. 2022Two-Timescale Stochastic Approximation for Bilevel Optimisation Problems in Continuous-Time Modelscited by 2
  9. 2022Adaptive Asynchronous Control Using Meta-learned Neural Ordinary Differential Equationscited by 2
  10. 2024Physics-Informed Model and Hybrid Planning for Efficient Dyna-Style Reinforcement Learningcited by 2
  11. 2024Model-based Policy Optimization using Symbolic World Modelcited by 2
  12. 2025Sample and Computationally Efficient Continuous-Time Reinforcement Learning with General Function Approximationcited by 2
  13. 2025Continuous-Time Value Iteration for Multi-Agent Reinforcement Learningcited by 2
  14. 2021Characteristic Neural Ordinary Differential Equationscited by 1
  15. 2022Optimisation of Structured Neural Controller Based on Continuous-Time Policy Gradientcited by 1
  16. 2022Managing Temporal Resolution in Continuous Value Estimation: A Fundamental Trade-offcited by 1
  17. 2023CERiL: Continuous Event-based Reinforcement Learningcited by 1
  18. 2024Impact of Computation in Integral Reinforcement Learning for Continuous-Time Controlcited by 1
  19. 2024A Differential and Pointwise Control Approach to Reinforcement Learningcited by 1
  20. 2025Instance-Dependent Continuous-Time Reinforcement Learning via Maximum Likelihood Estimationcited by 1
  21. 2026Action Shapley: A Training Data Selection Metric for World Model in Reinforcement Learningcited by 1
  22. 2023Continuous-Time Reinforcement Learning: New Design Algorithms with Theoretical Insights and Performance Guaranteescited by 0
  23. 2025Learning non-Markovian Dynamical Systems with Signature-based Encoderscited by 0
  24. 2025AD-NODE: Adaptive Dynamics Learning with Neural ODEs for Mobile Robots Controlcited by 0
  25. 2026Learning to Bid in Repeated Second-Price Auctions with Dynamic Values and Aggregated Feedbackcited by 0
  26. 2026Interaction-Limited Safe Continuous-Time RL for Dynamical Medical Treatmentcited by 0
  27. 2026Temporal Self-Imitation Learningcited by 0