# Accelerating Model-Based Reinforcement Learning using Non-Linear Trajectory Optimization

arXiv: https://arxiv.org/abs/2506.02767 · HTML: https://mlsanity.com/paper/2506.02767/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 4 of its 13 references that are in the map.
- First posted on arXiv 3 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 4 of 4)

- [PIPPS: Flexible Model-Based Policy Search Robust to the Curse of Chaos](https://mlsanity.com/paper/1902.01240/) (2019)
- [Black-Box Data-efficient Policy Search for Robotics](https://mlsanity.com/paper/1703.07261/) (2017)
- [Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application](https://mlsanity.com/paper/2101.12115/) (2021)
- [Learning Control from Raw Position Measurements](https://mlsanity.com/paper/2301.13183/) (2023)

## Abstract (arXiv)

This paper addresses the slow policy optimization convergence of Monte Carlo Probabilistic Inference for Learning Control (MC-PILCO), a state-of-the-art model-based reinforcement learning (MBRL) algorithm, by integrating it with iterative Linear Quadratic Regulator (iLQR), a fast trajectory optimization method suitable for nonlinear systems. The proposed method, Exploration-Boosted MC-PILCO (EB-MC-PILCO), leverages iLQR to generate informative, exploratory trajectories and initialize the policy, significantly reducing the number of required optimization steps. Experiments on the cart-pole task demonstrate that EB-MC-PILCO accelerates convergence compared to standard MC-PILCO, achieving up to bm45.9% reduction in execution time when both methods solve the task in four trials. EB-MC-PILCO also maintains a bm100% success rate across trials while solving the task faster, even in cases where MC-PILCO converges in fewer iterations.