# Face valuing: Training user interfaces with facial expressions and reinforcement learning

arXiv: https://arxiv.org/abs/1606.02807 · HTML: https://mlsanity.com/paper/1606.02807/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jun 2016.

## Built on by (most cited first, 6 of 6)

- [The EMPATHIC Framework for Task Learning from Implicit Human Feedback](https://mlsanity.com/paper/2009.13649/) (2020)
- [Maximizing BCI Human Feedback using Active Learning](https://mlsanity.com/paper/2008.04873/) (2020)
- [Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework](https://mlsanity.com/paper/2411.11761/) (2024)
- [Task Success Prediction for Open-Vocabulary Manipulation Based on Multi-Level Aligned Representations](https://mlsanity.com/paper/2410.00436/) (2024)
- [Training Robots to Evaluate Robots: Example-Based Interactive Reward Functions for Policy Learning](https://mlsanity.com/paper/2212.08961/) (2022)
- [Towards Reinforcement Learning from Neural Feedback: Mapping fNIRS Signals to Agent Performance](https://mlsanity.com/paper/2511.12844/) (2025)

## Abstract (arXiv)

An important application of interactive machine learning is extending or amplifying the cognitive and physical capabilities of a human. To accomplish this, machines need to learn about their human users' intentions and adapt to their preferences. In most current research, a user has conveyed preferences to a machine using explicit corrective or instructive feedback; explicit feedback imposes a cognitive load on the user and is expensive in terms of human effort. The primary objective of the current work is to demonstrate that a learning agent can reduce the amount of explicit feedback required for adapting to the user's preferences pertaining to a task by learning to perceive a value of its behavior from the human user, particularly from the user's facial expressions---we call this face valuing. We empirically evaluate face valuing on a grip selection task. Our preliminary results suggest that an agent can quickly adapt to a user's changing preferences with minimal explicit feedback by learning a value function that maps facial features extracted from a camera image to expected future reward. We believe that an agent learning to perceive a value from the body language of its human user is complementary to existing interactive machine learning approaches and will help in creating successful human-machine interactive applications.