# Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs

arXiv: https://arxiv.org/abs/2006.16210 · HTML: https://mlsanity.com/paper/2006.16210/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jun 2020.

## Built on by (most cited first, 30 of 32)

- [Planning with Diffusion for Flexible Behavior Synthesis](https://mlsanity.com/paper/2205.09991/) (2022)
- [SafeDiffuser: Safe Planning with Diffusion Probabilistic Models](https://mlsanity.com/paper/2306.00148/) (2023)
- [Multivariate Time Series Forecasting with Dynamic Graph Neural ODEs](https://mlsanity.com/paper/2202.08408/) (2022)
- [Heavy Ball Neural Ordinary Differential Equations](https://mlsanity.com/paper/2110.04840/) (2021)
- [Continuous-Time Model-Based Reinforcement Learning](https://mlsanity.com/paper/2102.04764/) (2021)
- [Policy Optimization for Continuous Reinforcement Learning](https://mlsanity.com/paper/2305.18901/) (2023)
- [Learning continuous models for continuous physics](https://mlsanity.com/paper/2202.08494/) (2022)
- [Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods](https://mlsanity.com/paper/2111.03941/) (2021)
- [Bellman Meets Hawkes: Model-Based Reinforcement Learning via Temporal Point Processes](https://mlsanity.com/paper/2201.12569/) (2022)
- [Efficient Exploration in Continuous-time Model-based Reinforcement Learning](https://mlsanity.com/paper/2310.19848/) (2023)
- [Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules](https://mlsanity.com/paper/2206.01649/) (2022)
- [The Medkit-Learn(ing) Environment: Medical Decision Modelling through Simulation](https://mlsanity.com/paper/2106.04240/) (2021)
- [How Does Momentum Benefit Deep Neural Networks Architecture Design? A Few Case Studies](https://mlsanity.com/paper/2110.07034/) (2021)
- [Neural Laplace Control for Continuous-time Delayed Systems](https://mlsanity.com/paper/2302.12604/) (2023)
- [TO-FLOW: Efficient Continuous Normalizing Flows with Temporal Optimization adjoint with Moving Speed](https://mlsanity.com/paper/2203.10335/) (2022)
- [Continuous-Depth Neural Models for Dynamic Graph Prediction](https://mlsanity.com/paper/2106.11581/) (2021)
- [ODE-based Recurrent Model-free Reinforcement Learning for POMDPs](https://mlsanity.com/paper/2309.14078/) (2023)
- [Identifying Decision Points for Safe and Interpretable Reinforcement Learning in Hypotension Treatment](https://mlsanity.com/paper/2101.03309/) (2021)
- [Decoding complexity: how machine learning is redefining scientific discovery](https://mlsanity.com/paper/2405.04161/) (2024)
- [Adaptive Asynchronous Control Using Meta-learned Neural Ordinary Differential Equations](https://mlsanity.com/paper/2207.12062/) (2022)
- [Time-Aware Q-Networks: Resolving Temporal Irregularity for Deep Reinforcement Learning](https://mlsanity.com/paper/2105.02580/) (2021)
- [STR-GODEs: Spatial-Temporal-Ridership Graph ODEs for Metro Ridership Prediction](https://mlsanity.com/paper/2107.04980/) (2021)
- [Deep Generative Models for Decision-Making and Control](https://mlsanity.com/paper/2306.08810/) (2023)
- [Neural ODE and SDE Models for Adaptation and Planning in Model-Based Reinforcement Learning](https://mlsanity.com/paper/2603.23245/) (2026)
- [Forward-Backward Latent State Inference for Hidden Continuous-Time semi-Markov Chains](https://mlsanity.com/paper/2210.09058/) (2022)
- [Approximate Control for Continuous-Time POMDPs](https://mlsanity.com/paper/2402.01431/) (2024)
- [Discovering Continuous-Time Memory-Based Symbolic Policies using Genetic Programming](https://mlsanity.com/paper/2406.02765/) (2024)
- [Increasing Information for Model Predictive Control with Semi-Markov Decision Processes](https://mlsanity.com/paper/2501.17256/) (2025)
- [Meta-reinforcement learning with minimum attention](https://mlsanity.com/paper/2505.16741/) (2025)
- [SoC-DT: Standard-of-Care Aligned Digital Twins for Patient-Specific Tumor Dynamics](https://mlsanity.com/paper/2510.03287/) (2025)

## Abstract (arXiv)

We present two elegant solutions for modeling continuous-time dynamics, in a novel model-based reinforcement learning (RL) framework for semi-Markov decision processes (SMDPs), using neural ordinary differential equations (ODEs). Our models accurately characterize continuous-time dynamics and enable us to develop high-performing policies using a small amount of data. We also develop a model-based approach for optimizing time schedules to reduce interaction rates with the environment while maintaining the near-optimal performance, which is not possible for model-free methods. We experimentally demonstrate the efficacy of our methods across various continuous-time domains.