# Reinforcement Learning for Ballbot Navigation in Uneven Terrain

arXiv: https://arxiv.org/abs/2505.18417 · HTML: https://mlsanity.com/paper/2505.18417/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 23 May 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [MuJoCo Playground](https://mlsanity.com/paper/2502.08844/) (2025)
- [A Review of Nine Physics Engines for Reinforcement Learning Research](https://mlsanity.com/paper/2407.08590/) (2024)
- [Learning Ball-balancing Robot Through Deep Reinforcement Learning](https://mlsanity.com/paper/2208.10142/) (2022)
- [Design and Control of a Ballbot Drivetrain with High Agility, Minimal Footprint, and High Payload](https://mlsanity.com/paper/2304.02887/) (2023)
- [Parameter Refinement of a Ballbot and Predictive Control for Reference Tracking with Linear Parameter-Varying Embedding](https://mlsanity.com/paper/2402.12092/) (2024)
- [Closed-Loop Identification and Tracking Control of a Ballbot](https://mlsanity.com/paper/2404.14845/) (2024)

## Built on by (most cited first, 1 of 1)

- [asRoBallet: Closing the Sim2Real Gap via Friction-Aware Reinforcement Learning for Underactuated Spherical Dynamics](https://mlsanity.com/paper/2604.24916/) (2026)

## Abstract (arXiv)

Ballbot (i.e. Ball balancing robot) navigation usually relies on methods rooted in control theory (CT), and works that apply Reinforcement learning (RL) to the problem remain rare while generally being limited to specific subtasks (e.g. balance recovery). Unlike CT based methods, RL does not require (simplifying) assumptions about environment dynamics (e.g. the absence of slippage between the ball and the floor). In addition to this increased accuracy in modeling, RL agents can easily be conditioned on additional observations such as depth-maps without the need for explicit formulations from first principles, leading to increased adaptivity. Despite those advantages, there has been little to no investigation into the capabilities, data-efficiency and limitations of RL based methods for ballbot control and navigation. Furthermore, there is a notable absence of an open-source, RL-friendly simulator for this task. In this paper, we present an open-source ballbot simulation based on MuJoCo, and show that with appropriate conditioning on exteroceptive observations as well as reward shaping, policies learned by classical model-free RL methods are capable of effectively navigating through randomly generated uneven terrain, using a reasonable amount of data (four to five hours on a system operating at 500hz). Our code is made publicly available.