mlsanityData to 7 Oct 2026

Model-based Reinforcement Learning with a Hamiltonian Canonical ODE Network

arXiv 2211.00942 ↗First posted 2 Nov 2022cs.LGcited by 0

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Model-based reinforcement learning usually suffers from a high sample complexity in training the world model, especially for the environments with complex dynamics. To make the training for general physical environments more efficient, we introduce Hamiltonian canonical ordinary differential equations into the learning process, which inspires a novel model of neural ordinary differential auto-encoder (NODA). NODA can model the physical world by nature and is flexible to impose Hamiltonian mechanics (e.g., the dimension of the physical equations) which can further accelerate training of the environment models. It can consequentially empower an RL agent with the robust extrapolation using a small amount of samples as well as the guarantee on the physical plausibility. Theoretically, we prove that NODA has uniform bounds for multi-step transition errors and value errors under certain conditions. Extensive experiments show that NODA can learn the environment dynamics effectively with a high sample efficiency, making it possible to facilitate reinforcement learning agents at the early stage.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 23 of its 41 references that are in the map. First posted on arXiv 2 Nov 2022.

Builds on

23 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Bridging Imagination and Reality for Model-Based Deep Reinforcement Learning23 Oct
  2. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec

Other references · 21

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  6. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  7. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  8. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  9. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  10. 2018DeepMind Control Suitecited by 935
  11. 2019Hamiltonian Neural Networkscited by 702
  12. 2020Lagrangian Neural Networkscited by 388
9 more
  1. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  2. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  3. 2019Deep Lagrangian Networks: Using Physics as Model Prior for Deep Learningcited by 205
  4. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  5. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  6. 2019When to use parametric models in reinforcement learning?cited by 118
  7. 2018Lipschitz Continuity in Model-based Reinforcement Learningcited by 90
  8. 2020Model-based Policy Optimization with Unsupervised Model Adaptationcited by 16
  9. 2020Deep Model-Based Reinforcement Learning for High-Dimensional Problems, a Surveycited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026World models of environment, agent and joint agent-environment systems
  2. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  3. 2026Physically Native World Models: A Hamiltonian Perspective on Generative World Modeling
  4. 2026Flowing Through States: Neural ODE Regularization for Reinforcement Learning
  5. 2020Smaller World Models for Reinforcement Learning
  6. 2025Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
  7. 2026Inverting the Bellman Equation: From Q-Values to World Models
  8. 2026CausalNav: Reliability-Certified Causal World Models for Control under Physical-Parameter Shift