# AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkit

arXiv: https://arxiv.org/abs/2202.12059 · HTML: https://mlsanity.com/paper/2202.12059/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 49 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 11 of 11)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Deep Facial Expression Recognition: A Survey](https://mlsanity.com/paper/1804.08348/) (2018)
- [ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challenges](https://mlsanity.com/paper/2202.10659/) (2022)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Semantic Relationships Guided Representation Learning for Facial Action Unit Recognition](https://mlsanity.com/paper/1904.09939/) (2019)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [Deep Structure Inference Network for Facial Action Unit Recognition](https://mlsanity.com/paper/1803.05873/) (2018)
- [SchiNet: Automatic Estimation of Symptoms of Schizophrenia from Facial Behaviour Analysis](https://mlsanity.com/paper/1808.02531/) (2018)
- [Automatic Detection of ADHD and ASD from Expressive Behaviour in RGBD Data](https://mlsanity.com/paper/1612.02374/) (2016)
- [Which CNNs and Training Settings to Choose for Action Unit Detection? A Study Based on a Large-Scale Dataset](https://mlsanity.com/paper/2111.08320/) (2021)
- [Choose Settings Carefully: Comparing Action Unit detection at Different Settings Using a Large-Scale Dataset](https://mlsanity.com/paper/2111.08324/) (2021)

## Built on by (most cited first, 5 of 5)

- [LibreFace: An Open-Source Toolkit for Deep Facial Expression Analysis](https://mlsanity.com/paper/2308.10713/) (2023)
- [Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkit](https://mlsanity.com/paper/2409.17717/) (2024)
- [OpenFace 3.0: A Lightweight Multitask System for Comprehensive Facial Behavior Analysis](https://mlsanity.com/paper/2506.02891/) (2025)
- [Monitoring Viewer Attention During Online Ads](https://mlsanity.com/paper/2504.06237/) (2025)
- [PainNet: Statistical Relation Network with Episode-Based Training for Pain Estimation](https://mlsanity.com/paper/2504.06257/) (2025)

## Abstract (arXiv)

In this paper we introduce AFFDEX 2.0 - a toolkit for analyzing facial expressions in the wild, that is, it is intended for users aiming to; a) estimate the 3D head pose, b) detect facial Action Units (AUs), c) recognize basic emotions and 2 new emotional states (sentimentality and confusion), and d) detect high-level expressive metrics like blink and attention. AFFDEX 2.0 models are mainly based on Deep Learning, and are trained using a large-scale naturalistic dataset consisting of thousands of participants from different demographic groups. AFFDEX 2.0 is an enhanced version of our previous toolkit [1], that is capable of tracking efficiently faces at more challenging conditions, detecting more accurately facial expressions, and recognizing new emotional states (sentimentality and confusion). AFFDEX 2.0 can process multiple faces in real time, and is working across the Windows and Linux platforms.