# Learning global control of underactuated systems with Model-Based Reinforcement Learning

arXiv: https://arxiv.org/abs/2504.06721 · HTML: https://mlsanity.com/paper/2504.06721/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 5 of its 18 references that are in the map.
- First posted on arXiv 9 Apr 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 5 of 5)

- [Model-Based Policy Search Using Monte Carlo Gradient Estimation with Real Systems Application](https://mlsanity.com/paper/2101.12115/) (2021)
- [Learning Control from Raw Position Measurements](https://mlsanity.com/paper/2301.13183/) (2023)
- [Learning control of underactuated double pendulum with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2409.05811/) (2024)
- [Reinforcement Learning for Robust Athletic Intelligence: Lessons from the 2nd 'AI Olympics with RealAIGym' Competition](https://mlsanity.com/paper/2503.15290/) (2025)
- [Data efficient Robotic Object Throwing with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2502.05595/) (2025)

## Abstract (arXiv)

This short paper describes our proposed solution for the third edition of the "AI Olympics with RealAIGym" competition, held at ICRA 2025. We employed Monte-Carlo Probabilistic Inference for Learning Control (MC-PILCO), an MBRL algorithm recognized for its exceptional data efficiency across various low-dimensional robotic tasks, including cart-pole, ball & plate, and Furuta pendulum systems. MC-PILCO optimizes a system dynamics model using interaction data, enabling policy refinement through simulation rather than direct system data optimization. This approach has proven highly effective in physical systems, offering greater data efficiency than Model-Free (MF) alternatives. Notably, MC-PILCO has previously won the first two editions of this competition, demonstrating its robustness in both simulated and real-world environments. Besides briefly reviewing the algorithm, we discuss the most critical aspects of the MC-PILCO implementation in the tasks at hand: learning a global policy for the pendubot and acrobot systems.