# Learning Perception-Aware Agile Flight in Cluttered Environments

arXiv: https://arxiv.org/abs/2210.01841 · HTML: https://mlsanity.com/paper/2210.01841/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning by Cheating](https://mlsanity.com/paper/1912.12294/) (2019)
- [CAD2RL: Real Single-Image Flight without a Single Real Image](https://mlsanity.com/paper/1611.04201/) (2016)
- [Vision-Only Robot Navigation in a Neural Radiance World](https://mlsanity.com/paper/2110.00168/) (2021)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [Flightmare: A Flexible Quadrotor Simulator](https://mlsanity.com/paper/2009.00563/) (2020)
- [Autonomous Drone Racing with Deep Reinforcement Learning](https://mlsanity.com/paper/2103.08624/) (2021)
- [Learning Deep Control Policies for Autonomous Aerial Vehicles with MPC-Guided Policy Search](https://mlsanity.com/paper/1509.06791/) (2015)
- [Robust and Efficient Quadrotor Trajectory Generation for Fast Autonomous Flight](https://mlsanity.com/paper/1907.01531/) (2019)
- [Differential Flatness of Quadrotor Dynamics Subject to Rotor Drag for Accurate Tracking of High-Speed Trajectories](https://mlsanity.com/paper/1712.02402/) (2017)
- [PAMPC: Perception-Aware Model Predictive Control for Quadrotors](https://mlsanity.com/paper/1804.04811/) (2018)
- [Memory-based Deep Reinforcement Learning for Obstacle Avoidance in UAV with Limited Environment Knowledge](https://mlsanity.com/paper/1811.03307/) (2018)
- [A Comparative Study of Nonlinear MPC and Differential-Flatness-Based Control for Quadrotor Agile Flight](https://mlsanity.com/paper/2109.01365/) (2021)
- [Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2108.04537/) (2021)
- [Policy Search for Model Predictive Control with Application to Agile Drone Flight](https://mlsanity.com/paper/2112.03850/) (2021)
- [Learning Minimum-Time Flight in Cluttered Environments](https://mlsanity.com/paper/2203.15052/) (2022)
- [Search-based Motion Planning for Aggressive Flight in SE(3)](https://mlsanity.com/paper/1710.02748/) (2017)
- [Aggressive Quadrotor Flight through Narrow Gaps with Onboard Sensing and Computing using Active Vision](https://mlsanity.com/paper/1612.00291/) (2016)
- [Minimum-Time Quadrotor Waypoint Flight in Cluttered Environments](https://mlsanity.com/paper/2202.03947/) (2022)

## Built on by (most cited first, 23 of 23)

- [Extreme Parkour with Legged Robots](https://mlsanity.com/paper/2309.14341/) (2023)
- [NavRL: Learning Safe Flight in Dynamic Environments](https://mlsanity.com/paper/2409.15634/) (2024)
- [Vision Transformers for End-to-End Vision-Based Quadrotor Obstacle Avoidance](https://mlsanity.com/paper/2405.10391/) (2024)
- [MAVRL: Learn to Fly in Cluttered Environments with Varying Speed](https://mlsanity.com/paper/2402.08381/) (2024)
- [Flying on Point Clouds with Reinforcement Learning](https://mlsanity.com/paper/2503.00496/) (2025)
- [RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigation](https://mlsanity.com/paper/2502.02054/) (2025)
- [Depth Transfer: Learning to See Like a Simulator for Real-World Drone Navigation](https://mlsanity.com/paper/2505.12428/) (2025)
- [To Distill or Decide? Understanding the Algorithmic Trade-off in Partially Observable Reinforcement Learning](https://mlsanity.com/paper/2510.03207/) (2025)
- [High-Speed Vision-Based Flight in Clutter with Safety-Shielded Reinforcement Learning](https://mlsanity.com/paper/2602.08653/) (2026)
- [Curriculum Reinforcement Learning for Quadrotor Racing with Random Obstacles](https://mlsanity.com/paper/2602.24030/) (2026)
- [NavRL++: A System-Level Framework for Improving Sim-to-Real Transfer in Reinforcement Learning-Based Robot Navigation](https://mlsanity.com/paper/2605.15559/) (2026)
- [Learning to Initialize Trajectory Optimization for Vision-Based Autonomous Flight in Unknown Environments](https://mlsanity.com/paper/2309.10683/) (2023)
- [RUMOR: Reinforcement learning for Understanding a Model of the Real World for Navigation in Dynamic Environments](https://mlsanity.com/paper/2404.16672/) (2024)
- [PRIMER: Perception-Aware Robust Learning-based Multiagent Trajectory Planner](https://mlsanity.com/paper/2406.10060/) (2024)
- [RESC: A Reinforcement Learning Based Search-to-Control Framework for Quadrotor Local Planning in Dense Environments](https://mlsanity.com/paper/2408.00275/) (2024)
- [VizFlyt: Perception-centric Pedagogical Framework For Autonomous Aerial Robots](https://mlsanity.com/paper/2503.22876/) (2025)
- [Unified Linear Parametric Map Modeling and Perception-aware Trajectory Planning for Mobile Robotics](https://mlsanity.com/paper/2507.09340/) (2025)
- [Zero-Shot UAV Navigation in Forests via Relightable 3D Gaussian Splatting](https://mlsanity.com/paper/2602.07101/) (2026)
- [Cross-Modal Reinforcement Learning for Navigation with Degraded Depth Measurements](https://mlsanity.com/paper/2603.22182/) (2026)
- [Vision-Guided Outdoor Flight and Obstacle Evasion via Reinforcement Learning](https://mlsanity.com/paper/2605.24449/) (2026)
- [Teacher-Student Representational Alignment for Reinforcement Learning-Driven Imitation Learning](https://mlsanity.com/paper/2605.28372/) (2026)
- [AeroVerse-SatAgent: UAV-Satellite Collaborative Spatial Reasoning Inspired by the Dual Visual Pathway Theory of Cognitive Neuroscience](https://mlsanity.com/paper/2606.31467/) (2026)
- [PILOT: Privileged Imitation Learning for End-to-End Motion Planning of Autonomous UAVs under Partial Observability](https://mlsanity.com/paper/2608.14082/) (2026)

## Abstract (arXiv)

Recently, neural control policies have outperformed existing model-based planning-and-control methods for autonomously navigating quadrotors through cluttered environments in minimum time. However, they are not perception aware, a crucial requirement in vision-based navigation due to the camera's limited field of view and the underactuated nature of a quadrotor. We propose a learning-based system that achieves perception-aware, agile flight in cluttered environments. Our method combines imitation learning with reinforcement learning (RL) by leveraging a privileged learning-by-cheating framework. Using RL, we first train a perception-aware teacher policy with full-state information to fly in minimum time through cluttered environments. Then, we use imitation learning to distill its knowledge into a vision-based student policy that only perceives the environment via a camera. Our approach tightly couples perception and control, showing a significant advantage in computation speed (10 times faster) and success rate. We demonstrate the closed-loop control performance using hardware-in-the-loop simulation.