# Flying on Point Clouds with Reinforcement Learning

arXiv: https://arxiv.org/abs/2503.00496 · HTML: https://mlsanity.com/paper/2503.00496/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 33 of its 46 references that are in the map.
- First posted on arXiv 1 Mar 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 30 of 33)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Learning Dexterous In-Hand Manipulation](https://mlsanity.com/paper/1808.00177/) (2018)
- [Learning agile and dynamic motor skills for legged robots](https://mlsanity.com/paper/1901.08652/) (2019)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Decoupling Representation Learning from Reinforcement Learning](https://mlsanity.com/paper/2009.08319/) (2020)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Deep Drone Acrobatics](https://mlsanity.com/paper/2006.05768/) (2020)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [EGO-Planner: An ESDF-free Gradient-based Local Planner for Quadrotors](https://mlsanity.com/paper/2008.08835/) (2020)
- [A Benchmark Comparison of Learned Control Policies for Agile Quadrotor Flight](https://mlsanity.com/paper/2202.10796/) (2022)
- [Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physics](https://mlsanity.com/paper/2407.10648/) (2024)
- [Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight](https://mlsanity.com/paper/1907.01531/) (2019)
- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge](https://mlsanity.com/paper/1811.03307/) (2018)
- [NavRL: Learning Safe Flight in Dynamic Environments](https://mlsanity.com/paper/2409.15634/) (2024)
- [Learning Perception-Aware Agile Flight in Cluttered Environments](https://mlsanity.com/paper/2210.01841/) (2022)
- [Learning to walk in confined spaces using 3D representation](https://mlsanity.com/paper/2403.00187/) (2024)
- [Geometrically Constrained Trajectory Optimization for Multicopters](https://mlsanity.com/paper/2103.00190/) (2021)
- [Learning Quadrotor Control From Visual Features Using Differentiable Simulation](https://mlsanity.com/paper/2410.15979/) (2024)
- [FASTER: Fast and Safe Trajectory Planner for Navigation in Unknown Environments](https://mlsanity.com/paper/2001.04420/) (2020)
- [Learning Speed Adaptation for Flight in Clutter](https://mlsanity.com/paper/2403.04586/) (2024)
- [Whole-Body Control Through Narrow Gaps From Pixels To Action](https://mlsanity.com/paper/2409.00895/) (2024)
- [Reinforcement Learning for Collision-free Flight Exploiting Deep Collision Encoding](https://mlsanity.com/paper/2402.03947/) (2024)
- [MAVRL: Learn to Fly in Cluttered Environments with Varying Speed](https://mlsanity.com/paper/2402.08381/) (2024)
- [Seeing Through Pixel Motion: Learning Obstacle Avoidance from Optical Flow with One Camera](https://mlsanity.com/paper/2411.04413/) (2024)
- [Bubble Planner: Planning High-speed Smooth Quadrotor Trajectories using Receding Corridors](https://mlsanity.com/paper/2202.12177/) (2022)
- [ROG-Map: An Efficient Robocentric Occupancy Grid Map for Large-scene and High-resolution LiDAR-based Motion Planning](https://mlsanity.com/paper/2302.14819/) (2023)

## Built on by (most cited first, 7 of 7)

- [Towards Task-Oriented Flying: Framework, Infrastructure, and Principles](https://mlsanity.com/paper/2504.15129/) (2025)
- [High-Speed Vision-Based Flight in Clutter with Safety-Shielded Reinforcement Learning](https://mlsanity.com/paper/2602.08653/) (2026)
- [Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2602.07101/) (2026)
- [CMRL: Collision-Aware and Memory-Enhanced Reinforcement Learning for UAV Navigation in Multi-Scale Obstacle Environments](https://mlsanity.com/paper/2605.14810/) (2026)
- [Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning](https://mlsanity.com/paper/2606.16513/) (2026)
- [Anticipatory Risk-Guided Reinforcement Learning for Safe Flight Through Dynamic Clutter](https://mlsanity.com/paper/2607.23565/) (2026)
- [Learning Multi-Agent Task Assignment and Navigation in the Factory: from Simulation to Real Robots](https://mlsanity.com/paper/2609.14567/) (2026)

## Abstract (arXiv)

A long-cherished vision of drones is to autonomously traverse through clutter to reach every corner of the world using onboard sensing and computation. In this paper, we combine onboard 3D lidar sensing and sim-to-real reinforcement learning (RL) to enable autonomous flight in cluttered environments. Compared to vision sensors, lidars appear to be more straightforward and accurate for geometric modeling of surroundings, which is one of the most important cues for successful obstacle avoidance. On the other hand, sim-to-real RL approach facilitates the realization of low-latency control, without the hierarchy of trajectory generation and tracking. We demonstrate that, with design choices of practical significance, we can effectively combine the advantages of 3D lidar sensing and RL to control a quadrotor through a low-level control interface at 50Hz. The key to successfully learn the policy in a lightweight way lies in a specialized surrogate of the lidar's raw point clouds, which simplifies learning while retaining a fine-grained perception to detect narrow free space and thin obstacles. Simulation statistics demonstrate the advantages of the proposed system over alternatives, such as performing easier maneuvers and higher success rates at different speed constraints. With lightweight simulation techniques, the policy trained in the simulator can control a physical quadrotor, where the system can dodge thin obstacles and safely traverse randomly distributed obstacles.