# The eyes and hearts of UAV pilots: observations of physiological responses in real-life scenarios

arXiv: https://arxiv.org/abs/2210.14910 · HTML: https://mlsanity.com/paper/2210.14910/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 44 references that are in the map.
- First posted on arXiv 26 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors](https://mlsanity.com/paper/2207.02696/) (2022)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Object Detection in 20 Years: A Survey](https://mlsanity.com/paper/1905.05055/) (2019)
- [DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution](https://mlsanity.com/paper/2006.02334/) (2020)
- [A Survey of Modern Deep Learning based Object Detection Models](https://mlsanity.com/paper/2104.11892/) (2021)
- [A Survey on Deep Learning Methods for Robot Vision](https://mlsanity.com/paper/1803.10862/) (2018)
- [ROS for Human-Robot Interaction](https://mlsanity.com/paper/2012.13944/) (2020)
- [Neural Networks for Semantic Gaze Analysis in XR Settings](https://mlsanity.com/paper/2103.10451/) (2021)
- [Automated analysis of eye-tracker-based human-human interaction studies](https://mlsanity.com/paper/2007.04671/) (2020)
- [When Being Soft Makes You Tough: A Collision-Resilient Quadcopter Inspired by Arthropods' Exoskeletons](https://mlsanity.com/paper/2103.04423/) (2021)

## Abstract (arXiv)

The drone industry is diversifying and the number of pilots increases rapidly. In this context, flight schools need adapted tools to train pilots, most importantly with regard to their own awareness of their physiological and cognitive limits. In civil and military aviation, pilots can train themselves on realistic simulators to tune their reaction and reflexes, but also to gather data on their piloting behavior and physiological states. It helps them to improve their performances. Opposed to cockpit scenarios, drone teleoperation is conducted outdoor in the field, thus with only limited potential from desktop simulation training. This work aims to provide a solution to gather pilots behavior out in the field and help them increase their performance. We combined advance object detection from a frontal camera to gaze and heart-rate variability measurements. We observed pilots and analyze their behavior over three flight challenges. We believe this tool can support pilots both in their training and in their regular flight tasks. A demonstration video is available on https://www.youtube.com/watch?v=eePhjd2qNiI