Learning to Fly by Crashing
Not on the map. Why?
Abstract (arXiv)
How do you learn to navigate an Unmanned Aerial Vehicle (UAV) and avoid obstacles? One approach is to use a small dataset collected by human experts: however, high capacity learning algorithms tend to overfit when trained with little data. An alternative is to use simulation. But the gap between simulation and real world remains large especially for perception problems. The reason most research avoids using large-scale real data is the fear of crashes! In this paper, we propose to bite the bullet and collect a dataset of crashes itself! We build a drone whose sole purpose is to crash into objects: it samples naive trajectories and crashes into random objects. We crash our drone 11,500 times to create one of the biggest UAV crash dataset. This dataset captures the different ways in which a UAV can crash. We use all this negative flying data in conjunction with positive data sampled from the same trajectories to learn a simple yet powerful policy for UAV navigation. We show that this simple self-supervised model is quite effective in navigating the UAV even in extremely cluttered environments with dynamic obstacles including humans. For supplementary video see: https://youtu.be/u151hJaGKUo
Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Apr 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 14; 2022: 5; 2023: 5; 2024: 4; 2025: 3; 2026: 2.
33 papers.
Sort:
By area:
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarioscited by 124
- 2021Learning High-Speed Flight in the Wildcited by 98
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
- 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 50
- 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 47
- 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 36
- 2021Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approachcited by 18
- 2021Learned Visual Navigation for Under-Canopy Agricultural Robotscited by 18
- 2021Structure from Silence: Learning Scene Structure from Ambient Soundcited by 17
21 more
- 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuumcited by 13
- 2021Deep reinforcement learning for guidewire navigation in coronary artery phantomcited by 9
- 2022New Challenges in Reinforcement Learning: A Survey of Security and Privacycited by 7
- 2021Self-Supervised Motion Retargeting with Safety Guaranteecited by 6
- 2025RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigationcited by 5
- 2021Robust Navigation for Racing Drones based on Imitation Learning and Modularizationcited by 4
- 2023Mutation Testing of Deep Reinforcement Learning Based on Real Faultscited by 4
- 2023BatMobility: Towards Flying Without Seeing for Autonomous Dronescited by 4
- 2024On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotorscited by 4
- 2024Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mWcited by 4
- 2022SAFER: Safe Collision Avoidance using Focused and Efficient Trajectory Search with Reinforcement Learningcited by 2
- 2023A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Dronescited by 2
- 2026Fly360: Omnidirectional Obstacle Avoidance within Drone Viewcited by 2
- 2021Deep Reinforcement Learning for Adaptive Exploration of Unknown Environmentscited by 1
- 2023Channel-Aware Distillation Transformer for Depth Estimation on Nano Dronescited by 1
- 2024OpenBot-Fleet: A System for Collective Learning with Real Robotscited by 1
- 2025Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approachcited by 1
- 2021Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?cited by 0
- 2023Grasping Student: semi-supervised learning for robotic manipulationcited by 0
- 2025Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policiescited by 0
- 2026Learning to Fly: Stable Vision-Guided UAV Servoing with Compact Target-Centric Cues and Reinforcement Learningcited by 0
- 2026Learning to Fly: Stable Vision-Guided UAV Servoing with Compact Target-Centric Cues and Reinforcement Learning7 Sep 2026
- 2026Fly360: Omnidirectional Obstacle Avoidance within Drone View6 Mar 2026
- 2025Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policies7 Oct 2025
- 2025Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach22 Aug 2025
- 2025RAPID: Robust and Agile Planner Using Inverse Reinforcement Learning for Vision-Based Drone Navigation4 Feb 2025
- 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuum20 Dec 2024
- 2024Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW6 Aug 2024
- 2024OpenBot-Fleet: A System for Collective Learning with Real Robots13 May 2024
- 2024On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors6 Mar 2024
- 2023BatMobility: Towards Flying Without Seeing for Autonomous Drones21 Jul 2023
- 2023A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones19 Jul 2023
- 2023Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones18 Mar 2023
21 more
- 2023Grasping Student: semi-supervised learning for robotic manipulation8 Mar 2023
- 2023Mutation Testing of Deep Reinforcement Learning Based on Real Faults13 Jan 2023
- 2022New Challenges in Reinforcement Learning: A Survey of Security and Privacy31 Dec 2022
- 2022Imitation Is Not Enough: Robustifying Imitation with Reinforcement Learning for Challenging Driving Scenarios21 Dec 2022
- 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Reality12 Oct 2022
- 2022SAFER: Safe Collision Avoidance using Focused and Efficient Trajectory Search with Reinforcement Learning23 Sep 2022
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action10 Jul 2022
- 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency2 Dec 2021
- 2021Structure from Silence: Learning Scene Structure from Ambient Sound10 Nov 2021
- 2021Learning High-Speed Flight in the Wild11 Oct 2021
- 2021Deep reinforcement learning for guidewire navigation in coronary artery phantom5 Oct 2021
- 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review26 Jul 2021
- 2021Learned Visual Navigation for Under-Canopy Agricultural Robots6 Jul 2021
- 2021Learning to See before Learning to Act: Visual Pre-training for Manipulation1 Jul 2021
- 2021Robust Navigation for Racing Drones based on Imitation Learning and Modularization27 May 2021
- 2021Deep Reinforcement Learning for Adaptive Exploration of Unknown Environments4 May 2021
- 2021Collision Replay: What Does Bumping Into Things Tell You About Scene Geometry?3 May 2021
- 2021Self-Supervised Motion Retargeting with Safety Guarantee11 Mar 2021
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned4 Feb 2021
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Survey2 Jan 2021
- 2021Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approach1 Jan 2021
- 2022LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Actioncited by 435
- 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
- 2021Learning High-Speed Flight in the Wildcited by 98
- 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
- 2022Holo-Dex: Teaching Dexterity with Immersive Mixed Realitycited by 52
- 2021Learning to See before Learning to Act: Visual Pre-training for Manipulationcited by 50
- 2021SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistencycited by 47
- 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 36
- 2021Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approachcited by 18
- 2021Learned Visual Navigation for Under-Canopy Agricultural Robotscited by 18
- 2021Structure from Silence: Learning Scene Structure from Ambient Soundcited by 17
- 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuumcited by 13
12 most cited of 31 All 31 (JSON)
- 2026Learning to Fly: Stable Vision-Guided UAV Servoing with Compact Target-Centric Cues and Reinforcement Learning7 Sep 2026
- 2026Fly360: Omnidirectional Obstacle Avoidance within Drone View6 Mar 2026
- 2025Oracle-Guided Masked Contrastive Reinforcement Learning for Visuomotor Policies7 Oct 2025
- 2025Autonomous UAV Flight Navigation in Confined Spaces: A Reinforcement Learning Approach22 Aug 2025
- 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuum20 Dec 2024
- 2024Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW6 Aug 2024
- 2024OpenBot-Fleet: A System for Collective Learning with Real Robots13 May 2024
- 2024On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors6 Mar 2024
- 2023BatMobility: Towards Flying Without Seeing for Autonomous Drones21 Jul 2023
- 2023A3D: Adaptive, Accurate, and Autonomous Navigation for Edge-Assisted Drones19 Jul 2023
- 2023Channel-Aware Distillation Transformer for Depth Estimation on Nano Drones18 Mar 2023
- 2023Grasping Student: semi-supervised learning for robotic manipulation8 Mar 2023
12 newest of 31 All 31 (JSON)