mlsanityData to 7 Oct 2026

Neural Laplace Control for Continuous-time Delayed Systems

arXiv 2302.12604 ↗First posted 24 Feb 2023cs.LG cs.AI cs.ROcited by 10

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Many real-world offline reinforcement learning (RL) problems involve continuous-time environments with delays. Such environments are characterized by two distinctive features: firstly, the state x(t) is observed at irregular time intervals, and secondly, the current action a(t) only affects the future state x(t + g) with an unknown delay g > 0. A prime example of such an environment is satellite control where the communication link between earth and a satellite causes irregular observations and delays. Existing offline RL algorithms have achieved success in environments with irregularly observed states in time or known delays. However, environments involving both irregular observations in time and unknown delays remains an open and challenging problem. To this end, we propose Neural Laplace Control, a continuous-time model-based offline RL method that combines a Neural Laplace dynamics model with a model predictive control (MPC) planner--and is able to learn from an offline dataset sampled with irregular time intervals from an environment that has a inherent unknown constant delay. We show experimentally on continuous-time delayed environments it is able to achieve near expert policy performance.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 38 of its 86 references that are in the map. First posted on arXiv 24 Feb 2023.

Builds on

38 of 86 references.

Closest ancestors · 7

Same direction, newest first

  1. 2021Offline Reinforcement Learning with Reverse Model-based Imagination1 Oct
  2. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  3. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  4. 2020MOPO: Model-based Offline Policy Optimization27 May
  5. 2020MOReL : Model-Based Offline Reinforcement Learning12 May
  6. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr
  7. 2019Behavior Regularized Offline Reinforcement Learning26 Nov

Other references · 31

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  8. 2018World Modelscited by 1,444
  9. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  10. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  11. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  12. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
19 more
  1. 2020Neural Controlled Differential Equations for Irregular Time Seriescited by 501
  2. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  3. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  4. 2017DART: Noise Injection for Robust Imitation Learningcited by 187
  5. 2019Symplectic ODE-Net: Learning Hamiltonian Dynamics with Controlcited by 183
  6. 2019Latent ODEs for Irregularly-Sampled Time Seriescited by 182
  7. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  8. 2020Model-Based Offline Planningcited by 126
  9. 2022Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equationscited by 59
  10. 2020Reinforcement Learning with Random Delayscited by 56
  11. 2021Continuous-Time Model-Based Reinforcement Learningcited by 39
  12. 2020Delay-Aware Model-Based Reinforcement Learning for Continuous Controlcited by 33
  13. 2020Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEscited by 32
  14. 2021Acting in Delayed Environments with Non-Stationary Markov Policiescited by 28
  15. 2018At Human Speed: Deep Reinforcement Learning with Action Delaycited by 24
  16. 2022Neural Laplace: Learning diverse classes of differential equations in the Laplace domaincited by 24
  17. 2021Learning Dynamics Models for Model Predictive Agentscited by 21
  18. 2022Delayed Reinforcement Learning by Imitationcited by 15
  19. 2020Blind Decision Making: Reinforcement Learning with Delayed Observationscited by 14

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 6; 2025: 3.

10 papers.

  1. 2024Dense Reward for Free in Reinforcement Learning from Human Feedbackcited by 77
  2. 2024Discovering Preference Optimization Algorithms with and for Large Language Modelscited by 38
  3. 2024Automatically Learning Hybrid Digital Twins of Dynamical Systemscited by 13
  4. 2024ODE Discovery for Longitudinal Heterogeneous Treatment Effects Inferencecited by 12
  5. 2023Accountability in Offline Reinforcement Learning: Explaining Decisions with a Corpus of Examplescited by 9
  6. 2024Defining Expertise: Applications to Treatment Effect Estimationcited by 2
  7. 2024Impact of Computation in Integral Reinforcement Learning for Continuous-Time Controlcited by 1
  8. 2025Increasing Information for Model Predictive Control with Semi-Markov Decision Processescited by 0
  9. 2025Breaking Free: Decoupling Forced Systems with Laplace Neural Networkscited by 0
  10. 2025Learning non-Markovian Dynamical Systems with Signature-based Encoderscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Belief-Based Offline Reinforcement Learning for Delay-Robust Policy Optimization
  2. 2025Model-Based Reinforcement Learning under Random Observation Delays
  3. 2020Offline Reinforcement Learning from Images with Latent Space Models
  4. 2024Reinforcement Learning from Delayed Observations via World Models
  5. 2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
  6. 2025Forecasting in Offline Reinforcement Learning for Non-stationary Environments
  7. 2022Comparing Model-free and Model-based Algorithms for Offline Reinforcement Learning
  8. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learning