mlsanityData to 14 Oct 2026

Guiding Reinforcement Learning with Incomplete System Dynamics

arXiv 2410.16821 ↗First posted 22 Oct 2024cs.RO cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Model-free reinforcement learning (RL) is inherently a reactive method, operating under the assumption that it starts with no prior knowledge of the system and entirely depends on trial-and-error for learning. This approach faces several challenges, such as poor sample efficiency, generalization, and the need for well-designed reward functions to guide learning effectively. On the other hand, controllers based on complete system dynamics do not require data. This paper addresses the intermediate situation where there is not enough model information for complete controller design, but there is enough to suggest that a model-free approach is not the best approach either. By carefully decoupling known and unknown information about the system dynamics, we obtain an embedded controller guided by our partial model and thus improve the learning efficiency of an RL-enhanced approach. A modular design allows us to deploy mainstream RL algorithms to refine the policy. Simulation results show that our method significantly improves sample efficiency compared with standard RL methods on continuous control tasks, and also offers enhanced performance over traditional control approaches. Experiments on a real ground vehicle also validate the performance of our method, including generalization and robustness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 24 of its 53 references that are in the map. First posted on arXiv 22 Oct 2024.

Builds on

24 of 53 references.

References · 24

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  5. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  6. 2016OpenAI Gymcited by 1,736
  7. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  8. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  9. 2018Quantifying Generalization in Reinforcement Learningcited by 352
  10. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
  11. 2018Differentiable MPC for End-to-end Planning and Controlcited by 189
  12. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
12 more
  1. 2019Benchmarking Model-Based Reinforcement Learningcited by 162
  2. 2016Value Iteration Networkscited by 160
  3. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 140
  4. 2018Model-Based Reinforcement Learning via Meta-Policy Optimizationcited by 101
  5. 2021Sample Efficient Reinforcement Learning via Model-Ensemble Exploration and Exploitationcited by 18
  6. 2020Infinite-Horizon Differentiable Model Predictive Controlcited by 17
  7. 2020EGO-Swarm: A Fully Autonomous and Decentralized Quadrotor Swarm System in Cluttered Environmentscited by 15
  8. 2018Provably Robust Learning-Based Approach for High-Accuracy Tracking Control of Lagrangian Systemscited by 10
  9. 2018Learn Fast, Forget Slow: Safe Predictive Learning Control for Systems with Unknown and Changing Dynamics Performing Repetitive Taskscited by 8
  10. 2018Trajectory Generation for Multiagent Point-To-Point Transitions via Distributed Model Predictive Controlcited by 6
  11. 2018Parallel Optimal Control for Cooperative Automation of Large-scale Connected Vehicles via ADMMcited by 2
  12. 2020Gaussian Processes Model-based Control of Underactuated Balance Robotscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents
  2. 2026Model-Based Data-Efficient and Robust Reinforcement Learning
  3. 2025First Order Model-Based RL through Decoupled Backpropagation
  4. 2024Overcoming Slow Decision Frequencies in Continuous Control: Model-Based Sequence Reinforcement Learning for Model-Free Control
  5. 2025Accelerating Model-Based Reinforcement Learning with State-Space World Models
  6. 2025Towards General-Purpose Model-Free Reinforcement Learning
  7. 2021Two-step reinforcement learning for model-free redesign of nonlinear optimal regulator
  8. 2024Reinforcement Learning for Motor Control: A Comprehensive Review