# CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control

arXiv: https://arxiv.org/abs/2603.15013 · HTML: https://mlsanity.com/paper/2603.15013/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 16 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 10 of 10)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Ultra Fast Structure-aware Deep Lane Detection](https://mlsanity.com/paper/2004.11757/) (2020)
- [A Survey of Deep Learning Applications to Autonomous Vehicle Control](https://mlsanity.com/paper/1912.10773/) (2019)
- [Bi-directional Domain Adaptation for Sim2Real Transfer of Embodied Navigation Agents](https://mlsanity.com/paper/2011.12421/) (2020)
- [Reinforcement Learning and Deep Learning based Lateral Control for Autonomous Driving](https://mlsanity.com/paper/1810.12778/) (2018)
- [Controlling an Autonomous Vehicle with Deep Reinforcement Learning](https://mlsanity.com/paper/1909.12153/) (2019)
- [Control of rough terrain vehicles using deep reinforcement learning](https://mlsanity.com/paper/2107.01867/) (2021)
- [Design and implementation of an adaptive critic-based neuro-fuzzy controller on an unmanned bicycle](https://mlsanity.com/paper/1702.03304/) (2017)

## Abstract (arXiv)

Autonomous bicycles offer a promising agile solution for urban mobility and last-mile logistics. However, conventional control strategies often struggle with underactuated nonlinear dynamics, suffering from sensitivity to model mismatches and limited adaptability to real-world uncertainties. To address this, we develop CycleRL, a comprehensive sim-to-real framework for robust autonomous bicycle control. Our approach establishes a direct perception-to-action mapping within the high-fidelity NVIDIA Isaac Sim environment, leveraging Proximal Policy Optimization (PPO) to optimize the control policy. The framework features a composite reward function tailored for concurrent balance maintenance, velocity tracking, and steering control. Crucially, systematic domain randomization is employed to reduce the reliance on precise system modeling, bridge the simulation-to-reality gap and facilitate direct transfer. In simulation, CycleRL achieves promising performance, including a 99.90% balance success rate, a heading tracking error of 1.15deg, and a velocity tracking error of 0.18 m/s. These quantitative results, coupled with successful hardware deployment, validate DRL as an effective paradigm for autonomous bicycle control, offering superior adaptability over traditional methods. Video demonstrations are available at https://cpnt-lab.github.io/CycleRL/.