Physics-Informed Model and Hybrid Planning for Efficient Dyna-Style Reinforcement Learning
Not on the map. Why?
Abstract (arXiv)
Applying reinforcement learning (RL) to real-world applications requires addressing a trade-off between asymptotic performance, sample efficiency, and inference time. In this work, we demonstrate how to address this triple challenge by leveraging partial physical knowledge about the system dynamics. Our approach involves learning a physics-informed model to boost sample efficiency and generating imaginary trajectories from this model to learn a model-free policy and Q-function. Furthermore, we propose a hybrid planning strategy, combining the learned policy and Q-function with the learned model to enhance time efficiency in planning. Through practical demonstrations, we illustrate that our method improves the compromise between sample efficiency, time efficiency, and performance over state-of-the-art methods.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 31 of its 39 references that are in the map. First posted on arXiv 2 Jul 2024.
Builds on
31 of 39 references.
References · 31
- 2015Continuous control with deep reinforcement learningcited by 3,279
- 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
- 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
- 2018World Modelscited by 1,444
- 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
- 2020Mastering Atari with Discrete World Modelscited by 990
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
- 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
- 2020Lagrangian Neural Networkscited by 388
- 2022Temporal Difference Learning for Model Predictive Controlcited by 375
19 more
- 2019Challenges of Real-World Reinforcement Learningcited by 282
- 2018Residual Reinforcement Learning for Robot Controlcited by 242
- 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
- 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 156
- 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 131
- 2020Augmenting Physical Models with Deep Networks for Complex Dynamics Forecastingcited by 93
- 2019Exploring Model-based Planning with Policy Networkscited by 90
- 2020Model-Augmented Actor-Critic: Backpropagating through Pathscited by 61
- 2020Learning Off-Policy with Online Planningcited by 46
- 2021Continuous-Time Model-Based Reinforcement Learningcited by 39
- 2018Augmenting Physical Simulators with Stochastic Neural Networks: Case Study of Planar Pushing and Bouncingcited by 36
- 2020Blending MPC & Value Function Approximation for Efficient Reinforcement Learningcited by 27
- 2021Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learningcited by 25
- 2022The Benefits of Model-Based Generalization in Reinforcement Learningcited by 17
- 2022Physics-Informed Model-Based Reinforcement Learningcited by 17
- 2021Evaluating model-based planning and planner amortization for continuous controlcited by 15
- 2017Combining Learned and Analytical Models for Predicting Action Effects from Sensory Datacited by 10
- 2015Model-based Reinforcement Learning with Parametrized Physical Models and Optimism-Driven Explorationcited by 8
- 2019Modelling Generalized Forces with Reinforcement Learning for Sim-to-Real Transfercited by 5
Built on by
Citing papers per year: 2025: 2.
2 papers.