# Teaching Drones on the Fly: Can Emotional Feedback Serve as Learning Signal for Training Artificial Agents?

arXiv: https://arxiv.org/abs/2202.09634 · HTML: https://mlsanity.com/paper/2202.09634/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 8 of its 28 references that are in the map.
- First posted on arXiv 19 Feb 2022.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 8 of 8)

- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Challenges in Representation Learning: A report on three machine learning contests](https://mlsanity.com/paper/1307.0414/) (2013)
- [Interactive Learning from Policy-Dependent Human Feedback](https://mlsanity.com/paper/1701.06049/) (2017)
- [SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild](https://mlsanity.com/paper/1901.02839/) (2019)
- [Learning Rewards from Linguistic Feedback](https://mlsanity.com/paper/2009.14715/) (2020)
- [Real-time Convolutional Neural Networks for Emotion and Gender Classification](https://mlsanity.com/paper/1710.07557/) (2017)
- [AI Challenges in Human-Robot Cognitive Teaming](https://mlsanity.com/paper/1707.04775/) (2017)
- [Perception for Autonomous Systems (PAZ)](https://mlsanity.com/paper/2010.14541/) (2020)

## Abstract (arXiv)

We investigate whether naturalistic emotional human feedback can be directly exploited as a reward signal for training artificial agents via interactive human-in-the-loop reinforcement learning. To answer this question, we devise an experimental setting inspired by animal training, in which human test subjects interactively teach an emulated drone agent their desired command-action-mapping by providing emotional feedback on the drone's action selections. We present a first empirical proof-of-concept study and analysis confirming that human facial emotion expression can be directly exploited as reward signal in such interactive learning settings. Thereby, we contribute empirical findings towards more naturalistic and intuitive forms of reinforcement learning especially designed for non-expert users.