# RAF-AU Database: In-the-Wild Facial Expressions with Subjective Emotion Judgement and Objective AU Annotations

arXiv: https://arxiv.org/abs/2008.05196 · HTML: https://mlsanity.com/paper/2008.05196/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Aug 2020.

## Built on by (most cited first, 11 of 11)

- [FaceBench: A Multi-View Multi-Level Facial Attribute VQA Dataset for Benchmarking Face Perception MLLMs](https://mlsanity.com/paper/2503.21457/) (2025)
- [AgentAvatar: Disentangling Planning, Driving and Rendering for Photorealistic Avatar Agents](https://mlsanity.com/paper/2311.17465/) (2023)
- [FEAFA+: An Extended Well-Annotated Dataset for Facial Expression Analysis and 3D Facial Animation](https://mlsanity.com/paper/2111.02751/) (2021)
- [Rethinking Affect Analysis: A Protocol for Ensuring Fairness and Consistency](https://mlsanity.com/paper/2408.02164/) (2024)
- [Meta Auxiliary Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2105.06620/) (2021)
- [Contrastive Learning of Person-independent Representations for Facial Action Unit Detection](https://mlsanity.com/paper/2403.03400/) (2024)
- [AU-Aware Vision Transformers for Biased Facial Expression Recognition](https://mlsanity.com/paper/2211.06609/) (2022)
- [Multimodal Representation Learning Techniques for Comprehensive Facial State Analysis](https://mlsanity.com/paper/2504.10351/) (2025)
- [Focal-RegionFace: Generating Fine-Grained Multi-attribute Descriptions for Arbitrarily Selected Face Focal Regions](https://mlsanity.com/paper/2601.00156/) (2026)
- [AU-Supervised Convolutional Vision Transformers for Synthetic Facial Expression Recognition](https://mlsanity.com/paper/2207.09777/) (2022)
- [SeetaPsych v1.0: An Open-source Computer Vision Toolkit for Behavior-based Psychological Measurement](https://mlsanity.com/paper/2609.19719/) (2026)

## Abstract (arXiv)

Much of the work on automatic facial expression recognition relies on databases containing a certain number of emotion classes and their exaggerated facial configurations (generally six prototypical facial expressions), based on Ekman's Basic Emotion Theory. However, recent studies have revealed that facial expressions in our human life can be blended with multiple basic emotions. And the emotion labels for these in-the-wild facial expressions cannot easily be annotated solely on pre-defined AU patterns. How to analyze the action units for such complex expressions is still an open question. To address this issue, we develop a RAF-AU database that employs a sign-based (i.e., AUs) and judgement-based (i.e., perceived emotion) approach to annotating blended facial expressions in the wild. We first reviewed the annotation methods in existing databases and identified crowdsourcing as a promising strategy for labeling in-the-wild facial expressions. Then, RAF-AU was finely annotated by experienced coders, on which we also conducted a preliminary investigation of which key AUs contribute most to a perceived emotion, and the relationship between AUs and facial expressions. Finally, we provided a baseline for AU recognition in RAF-AU using popular features and multi-label learning methods.