# Human-Piloted Drone Racing: Visual Processing and Control

arXiv: https://arxiv.org/abs/2103.04672 · HTML: https://mlsanity.com/paper/2103.04672/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Mar 2021.

## Built on by (most cited first, 9 of 9)

- [Demonstrating Agile Flight from Pixels without State Estimation](https://mlsanity.com/paper/2406.12505/) (2024)
- [Time-Optimal Planning for Quadrotor Waypoint Flight](https://mlsanity.com/paper/2108.04537/) (2021)
- [Learning Deep Sensorimotor Policies for Vision-based Autonomous Drone Racing](https://mlsanity.com/paper/2210.14985/) (2022)
- [Dream to Fly: Model-Based Reinforcement Learning for Vision-Based Drone Flight](https://mlsanity.com/paper/2501.14377/) (2025)
- [SkyDreamer: Interpretable End-to-End Vision-Based Drone Racing with Model-Based Reinforcement Learning](https://mlsanity.com/paper/2510.14783/) (2025)
- [Visual Attention Prediction Improves Performance of Autonomous Drone Racing Agents](https://mlsanity.com/paper/2201.02569/) (2022)
- [Quantum-Inspired Neural Network Model of Optical Illusions](https://mlsanity.com/paper/2312.03447/) (2023)
- [DiffusionCinema: Text-to-Aerial Cinematography](https://mlsanity.com/paper/2601.17412/) (2026)
- [RAR: Rate-Adaptive Rollout for Time-Consistent Interactive World Models](https://mlsanity.com/paper/2610.16302/) (2026)

## Abstract (arXiv)

Humans race drones faster than algorithms, despite being limited to a fixed camera angle, body rate control, and response latencies in the order of hundreds of milliseconds. A better understanding of the ability of human pilots of selecting appropriate motor commands from highly dynamic visual information may provide key insights for solving current challenges in vision-based autonomous navigation. This paper investigates the relationship between human eye movements, control behavior, and flight performance in a drone racing task. We collected a multimodal dataset from 21 experienced drone pilots using a highly realistic drone racing simulator, also used to recruit professional pilots. Our results show task-specific improvements in drone racing performance over time. In particular, we found that eye gaze tracks future waypoints (i.e., gates), with first fixations occurring on average 1.5 seconds and 16 meters before reaching the gate. Moreover, human pilots consistently looked at the inside of the future flight path for lateral (i.e., left and right turns) and vertical maneuvers (i.e., ascending and descending). Finally, we found a strong correlation between pilots eye movements and the commanded direction of quadrotor flight, with an average visual-motor response latency of 220 ms. These results highlight the importance of coordinated eye movements in human-piloted drone racing. We make our dataset publicly available.