Accelerating Model-Based Reinforcement Learning using Non-Linear Trajectory Optimization
On the map: Deep reinforcement learning methods → RL theory and sample complexity
Abstract (arXiv)
This paper addresses the slow policy optimization convergence of Monte Carlo Probabilistic Inference for Learning Control (MC-PILCO), a state-of-the-art model-based reinforcement learning (MBRL) algorithm, by integrating it with iterative Linear Quadratic Regulator (iLQR), a fast trajectory optimization method suitable for nonlinear systems. The proposed method, Exploration-Boosted MC-PILCO (EB-MC-PILCO), leverages iLQR to generate informative, exploratory trajectories and initialize the policy, significantly reducing the number of required optimization steps. Experiments on the cart-pole task demonstrate that EB-MC-PILCO accelerates convergence compared to standard MC-PILCO, achieving up to bm45.9% reduction in execution time when both methods solve the task in four trials. EB-MC-PILCO also maintains a bm100% success rate across trials while solving the task faster, even in cases where MC-PILCO converges in fewer iterations.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 4 of its 13 references that are in the map. First posted on arXiv 3 Jun 2025.
Builds on
4 of 13 references.
References · 4
- 2019PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaoscited by 56
- 2017Black-Box Data-efficient Policy Search for Roboticscited by 14
- 2021Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Applicationcited by 10
- 2023Learning Control from Raw Position Measurementscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Learning global control of underactuated systems with Model-Based Reinforcement Learning
- 2024Towards Global Optimality for Practical Average Reward Reinforcement Learning without Mixing Time Oracles
- 2025First Order Model-Based RL through Decoupled Backpropagation
- 2024Adaptive Horizon Actor-Critic for Policy Learning in Contact-Rich Differentiable Simulation
- 2021Learning Robust Controllers Via Probabilistic Model-Based Policy Search
- 2024SPO: Sequential Monte Carlo Policy Optimisation
- 2020Ready Policy One: World Building Through Active Learning
- 2024A General Control-Theoretic Approach for Reinforcement Learning: Theory and Algorithms