# One-Frame Calibration with Siamese Network in Facial Action Unit Recognition

arXiv: https://arxiv.org/abs/2409.00240 · HTML: https://mlsanity.com/paper/2409.00240/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 58 references that are in the map.
- First posted on arXiv 30 Aug 2024.

## Builds on (most cited first, 20 of 20)

- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FC](https://mlsanity.com/paper/2203.15565/) (2022)
- [JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention](https://mlsanity.com/paper/2003.08834/) (2020)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [Neural Aggregation Network for Video Face Recognition](https://mlsanity.com/paper/1603.05474/) (2016)
- [AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors](https://mlsanity.com/paper/2403.04697/) (2024)
- [Facial Action Unit Intensity Estimation via Semantic Correspondence Learning with Dynamic Graph Convolution](https://mlsanity.com/paper/2004.09681/) (2020)
- [Responsible AI: Gender bias assessment in emotion recognition](https://mlsanity.com/paper/2103.11436/) (2021)
- [Domain-Incremental Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition](https://mlsanity.com/paper/2103.08637/) (2021)
- [Deep Structured Learning for Facial Action Unit Intensity Estimation](https://mlsanity.com/paper/1704.04481/) (2017)
- [Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding](https://mlsanity.com/paper/2304.00058/) (2023)
- [Multi-scale Promoted Self-adjusting Correlation Learning for Facial Action Unit Detection](https://mlsanity.com/paper/2308.07770/) (2023)
- [Multi-scale Dynamic and Hierarchical Relationship Modeling for Facial Action Units Recognition](https://mlsanity.com/paper/2404.06443/) (2024)
- [DeepCoder: Semi-parametric Variational Autoencoders for Automatic Facial Action Coding](https://mlsanity.com/paper/1704.02206/) (2017)
- [DeepFN: Towards Generalizable Facial Action Unit Recognition with Deep Face Normalization](https://mlsanity.com/paper/2103.02484/) (2021)
- [Using Positive Matching Contrastive Loss with Facial Action Units to mitigate bias in Facial Expression Recognition](https://mlsanity.com/paper/2303.04896/) (2023)
- [Towards Fair Affective Robotics: Continual Learning for Mitigating Bias in Facial Expression and Action Unit Recognition](https://mlsanity.com/paper/2103.09233/) (2021)

## Built on by (most cited first, 1 of 1)

- [Facial-Expression-Aware Prompting for Empathetic LLM Tutoring](https://mlsanity.com/paper/2604.15336/) (2026)

## Abstract (arXiv)

Automatic facial action unit (AU) recognition is used widely in facial expression analysis. Most existing AU recognition systems aim for cross-participant non-calibrated generalization (NCG) to unseen faces without further calibration. However, due to the diversity of facial attributes across different identities, accurately inferring AU activation from single images of an unseen face is sometimes infeasible, even for human experts -- it is crucial to first understand how the face appears in its neutral expression, or significant bias may be incurred. Therefore, we propose to perform one-frame calibration (OFC) in AU recognition: for each face, a single image of its neutral expression is used as the reference image for calibration. With this strategy, we develop a Calibrating Siamese Network (CSN) for AU recognition and demonstrate its remarkable effectiveness with a simple iResNet-50 (IR50) backbone. On the DISFA, DISFA+, and UNBC-McMaster datasets, we show that our OFC CSN-IR50 model (a) substantially improves the performance of IR50 by mitigating facial attribute biases (including biases due to wrinkles, eyebrow positions, facial hair, etc.), (b) substantially outperforms the naive OFC method of baseline subtraction as well as (c) a fine-tuned version of this naive OFC method, and (d) also outperforms state-of-the-art NCG models for both AU intensity estimation and AU detection. The code is available at https://github.com/ShuangquanFeng/CSN.