# Reinforcement Learning for Wheeled Mobility on Vertically Challenging Terrain

arXiv: https://arxiv.org/abs/2409.02383 · HTML: https://mlsanity.com/paper/2409.02383/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 4 Sep 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 30 of 37)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Socially Compliant Navigation Dataset (SCAND): A Large-Scale Dataset of Demonstrations for Social Navigation](https://mlsanity.com/paper/2203.15041/) (2022)
- [Motion Planning and Control for Mobile Robot Navigation Using Machine Learning: a Survey](https://mlsanity.com/paper/2011.13112/) (2020)
- [Elevation Mapping for Locomotion and Navigation using GPU](https://mlsanity.com/paper/2204.12876/) (2022)
- [Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model](https://mlsanity.com/paper/1804.01947/) (2018)
- [V-STRONG: Visual Self-Supervised Traversability Learning for Off-road Navigation](https://mlsanity.com/paper/2312.16016/) (2023)
- [VOILA: Visual-Observation-Only Imitation Learning for Autonomous Navigation](https://mlsanity.com/paper/2105.09371/) (2021)
- [Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset](https://mlsanity.com/paper/2303.14880/) (2023)
- [TerrainNet: Visual Modeling of Complex Terrain for High-speed, Off-road Navigation](https://mlsanity.com/paper/2303.15771/) (2023)
- [Learning Model Predictive Controllers with Real-Time Attention for Real-World Navigation](https://mlsanity.com/paper/2209.10780/) (2022)
- [How Does It Feel? Self-Supervised Costmap Learning for Off-Road Vehicle Traversability](https://mlsanity.com/paper/2209.10788/) (2022)
- [Learning Off-Road Terrain Traversability with Self-Supervisions Only](https://mlsanity.com/paper/2305.18896/) (2023)
- [Learning Inverse Kinodynamics for Accurate High-Speed Off-Road Navigation on Unstructured Terrain](https://mlsanity.com/paper/2102.12667/) (2021)
- [VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics](https://mlsanity.com/paper/2203.15983/) (2022)
- [EVORA: Deep Evidential Traversability Learning for Risk-Aware Off-Road Autonomy](https://mlsanity.com/paper/2311.06234/) (2023)
- [APPLD: Adaptive Planner Parameter Learning from Demonstration](https://mlsanity.com/paper/2004.00116/) (2020)
- [OFFSEG: A Semantic Segmentation Framework For Off-Road Driving](https://mlsanity.com/paper/2103.12417/) (2021)
- [APPL: Adaptive Planner Parameter Learning](https://mlsanity.com/paper/2105.07620/) (2021)
- [Risk-Aware Off-Road Navigation via a Learned Speed Distribution Map](https://mlsanity.com/paper/2203.13429/) (2022)
- [APPLI: Adaptive Planner Parameter Learning From Interventions](https://mlsanity.com/paper/2011.00400/) (2020)
- [Benchmarking Reinforcement Learning Techniques for Autonomous Navigation](https://mlsanity.com/paper/2210.04839/) (2022)
- [Toward Wheeled Mobility on Vertically Challenging Terrain: Platforms, Datasets, and Algorithms](https://mlsanity.com/paper/2303.00998/) (2023)
- [Visual Representation Learning for Preference-Aware Path Planning](https://mlsanity.com/paper/2109.08968/) (2021)
- [APPLR: Adaptive Planner Parameter Learning from Reinforcement](https://mlsanity.com/paper/2011.00397/) (2020)
- [APPLE: Adaptive Planner Parameter Learning from Evaluative Feedback](https://mlsanity.com/paper/2108.09801/) (2021)
- [Toward Agile Maneuvers in Highly Constrained Spaces: Learning from Hallucination](https://mlsanity.com/paper/2007.14479/) (2020)
- [High-Speed Accurate Robot Control using Learned Forward Kinodynamics and Non-linear Least Squares Optimization](https://mlsanity.com/paper/2206.08487/) (2022)
- [Learning to Model and Plan for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2306.11611/) (2023)
- [From Agile Ground to Aerial Navigation: Learning from Learned Hallucination](https://mlsanity.com/paper/2108.09793/) (2021)

## Built on by (most cited first, 7 of 7)

- [Grounded Curriculum Learning](https://mlsanity.com/paper/2409.19816/) (2024)
- [Reward Training Wheels: Adaptive Auxiliary Rewards for Robotics Reinforcement Learning](https://mlsanity.com/paper/2503.15724/) (2025)
- [VertiSelector: Automatic Curriculum Learning for Wheeled Mobility on Vertically Challenging Terrain](https://mlsanity.com/paper/2409.17469/) (2024)
- [Wheeled Lab: Modern Sim2Real for Low-cost, Open-source Wheeled Robotics](https://mlsanity.com/paper/2502.07380/) (2025)
- [GACL: Grounded Adaptive Curriculum Learning with Active Task and Performance Monitoring](https://mlsanity.com/paper/2508.02988/) (2025)
- [Sim2Dust: Mastering Dynamic Waypoint Tracking on Granular Media](https://mlsanity.com/paper/2508.11503/) (2025)
- [APPLV: Adaptive Planner Parameter Learning from Vision-Language-Action Model](https://mlsanity.com/paper/2603.08862/) (2026)

## Abstract (arXiv)

Off-road navigation on vertically challenging terrain, involving steep slopes and rugged boulders, presents significant challenges for wheeled robots both at the planning level to achieve smooth collision-free trajectories and at the control level to avoid rolling over or getting stuck. Considering the complex model of wheel-terrain interactions, we develop an end-to-end Reinforcement Learning (RL) system for an autonomous vehicle to learn wheeled mobility through simulated trial-and-error experiences. Using a custom-designed simulator built on the Chrono multi-physics engine, our approach leverages Proximal Policy Optimization (PPO) and a terrain difficulty curriculum to refine a policy based on a reward function to encourage progress towards the goal and penalize excessive roll and pitch angles, which circumvents the need of complex and expensive kinodynamic modeling, planning, and control. Additionally, we present experimental results in the simulator and deploy our approach on a physical Verti-4-Wheeler (V4W) platform, demonstrating that RL can equip conventional wheeled robots with previously unrealized potential of navigating vertically challenging terrain.