# CausalAffect: Causal Discovery for Facial Affective Understanding

arXiv: https://arxiv.org/abs/2512.00456 · HTML: https://mlsanity.com/paper/2512.00456/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 79 references that are in the map.
- First posted on arXiv 29 Nov 2025.

## Builds on (most cited first, 30 of 37)

- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [How Attentive are Graph Attention Networks?](https://mlsanity.com/paper/2105.14491/) (2021)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study](https://mlsanity.com/paper/2105.03790/) (2021)
- [Face Behavior a la carte: Expressions, Affect and Action Units in a Single Network](https://mlsanity.com/paper/1910.11111/) (2019)
- [Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition](https://mlsanity.com/paper/2205.01782/) (2022)
- [JhatAA-Net: Joint Facial Action Unit Detection and Face Alignment via Adaptive Attention](https://mlsanity.com/paper/2003.08834/) (2020)
- [Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyond](https://mlsanity.com/paper/2401.01219/) (2024)
- [Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignment](https://mlsanity.com/paper/1803.05588/) (2018)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [Facial Expression Recognition with Visual Transformers and Attentional Selective Fusion](https://mlsanity.com/paper/2103.16854/) (2021)
- [Facial Action Unit Detection Using Attention and Relation Learning](https://mlsanity.com/paper/1808.03457/) (2018)
- [A Survey on Explainability of Graph Neural Networks](https://mlsanity.com/paper/2306.01958/) (2023)
- [AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors](https://mlsanity.com/paper/2403.04697/) (2024)
- [MVT: Mask Vision Transformer for Facial Expression Recognition in the wild](https://mlsanity.com/paper/2106.04520/) (2021)
- [Facial Affective Behavior Analysis with Instruction Tuning](https://mlsanity.com/paper/2404.05052/) (2024)
- [Relation Modeling with Graph Convolutional Networks for Facial Action Unit Detection](https://mlsanity.com/paper/1910.10334/) (2019)
- [Multi-modal Facial Affective Analysis based on Masked Autoencoder](https://mlsanity.com/paper/2303.10849/) (2023)
- [Benchmarking Counterfactual Image Generation](https://mlsanity.com/paper/2403.20287/) (2024)
- [Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognition](https://mlsanity.com/paper/2303.08356/) (2023)
- [Counterfactual Image Editing](https://mlsanity.com/paper/2403.09683/) (2024)
- [Learning to Amend Facial Expression Representation via De-albino and Affinity](https://mlsanity.com/paper/2103.10189/) (2021)
- [Self-Supervised Facial Representation Learning with Facial Region Awareness](https://mlsanity.com/paper/2403.02138/) (2024)
- [Weakly-Supervised Text-driven Contrastive Learning for Facial Behavior Understanding](https://mlsanity.com/paper/2304.00058/) (2023)
- [ExpLLM: Towards Chain of Thought for Facial Expression Recognition](https://mlsanity.com/paper/2409.02828/) (2024)
- [Metrics for Dataset Demographic Bias: A Case Study on Facial Expression Recognition](https://mlsanity.com/paper/2303.15889/) (2023)
- [FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Features](https://mlsanity.com/paper/2308.12380/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Closed-Form Solution for Debiasing Vision-Language Models with Utility Guarantees Across Modalities and Tasks](https://mlsanity.com/paper/2603.12998/) (2026)
- [Bidirectional Learning of Facial Action Units and Expressions via Structured Semantic Mapping across Heterogeneous Datasets](https://mlsanity.com/paper/2604.10541/) (2026)
- [Faithful Action-unit Causal Reasoning for Counterfactually Faithful Emotion Explanations](https://mlsanity.com/paper/2606.15779/) (2026)

## Abstract (arXiv)

Understanding human affect from facial behavior requires not only accurate recognition but also structured reasoning over the latent dependencies that drive muscle activations and their expressive outcomes. Although Action Units (AUs) have long served as the foundation of affective computing, existing approaches rarely address how to infer psychologically plausible causal relations between AUs and expressions directly from data. We propose CausalAffect, the first framework for causal graph discovery in facial affect analysis. CausalAffect models AU-AU and AU-Expression dependencies through a two-level polarity and direction aware causal hierarchy that integrates population-level regularities with sample-adaptive structures. A feature-level counterfactual intervention mechanism further enforces true causal effects while suppressing spurious correlations. Crucially, our approach requires neither jointly annotated datasets nor handcrafted causal priors, yet it recovers causal structures consistent with established psychological theories while revealing novel inhibitory and previously uncharacterized dependencies. Extensive experiments across six benchmarks demonstrate that CausalAffect advances the state of the art in both AU detection and expression recognition, establishing a principled connection between causal discovery and interpretable facial behavior. All trained models and source code will be released upon acceptance.