AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flow
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
We present AffectFlow-DINO, a multi-task learning system for the 11th ABAW challenge that extends a standard deterministic architecture with a conditional rectified-flow head to model the inherent ambiguity of in-the-wild facial behavior. Instead of predicting a single affect estimate, the model learns a conditional generative distribution, enabling uncertainty-aware one-to-many predictions through Monte Carlo sampling. The system jointly estimates continuous valence-arousal, classifies eight facial expressions, and detects twelve Action Units from static face images. Built on a frozen DINOv3 ViT-S/16 backbone, extensive ablation studies show that rectified-flow decoding consistently improves deterministic prediction, particularly for valence-arousal estimation (CCC-V +0.058). We further show that post-hoc threshold calibration effectively recovers performance on severely imbalanced rare classes (e.g., Fear: 3.8% rightarrow 33.1%) without retraining. Combined with backbone fine-tuning and flow retuning, the final model achieves mathbfPMTL=1.177, substantially outperforming the official challenge baseline of PMTL=0.45.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 27 of its 38 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
27 of 38 references.
References · 27
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2018End-to-End Multi-Task Learning with Attentioncited by 533
- 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
- 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
- 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
- 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
- 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
- 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
- 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
- 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
15 more
- 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
- 2020DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the Wildcited by 128
- 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
- 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
- 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
- 2022FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videoscited by 69
- 2019SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wildcited by 62
- 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
- 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
- 2018The OMG-Emotion Behavior Datasetcited by 25
- 2024Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkitcited by 16
- 2026From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competitioncited by 12
- 2026Conflict-Aware Multimodal Fusion for Ambivalence and Hesitancy Recognitioncited by 11
- 2024Affective Behavior Analysis using Task-adaptive and AU-assisted Graph Networkcited by 8
- 2024Enhancing Facial Expression Recognition through Dual-Direction Attention Mixed Feature Networks: Application to 7th ABAW Challengecited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysis
- 2025Reading Smiles: Proxy Bias in Foundation Models for Facial Emotion Recognition
- 2026Multimodal Emotion Recognition via Causal-Diffusion Bridge (Affect-Diff)
- 2025MELLM: A Flow-Guided Large Language Model for Micro-Expression Understanding
- 2026Emotion as a Distribution: Joint Valence-Arousal Probability Learning for Speaker-Independent Multimodal Emotion Recognition
- 2026ARGen: Affect-Reinforced Generative Augmentation towards Vision-based Dynamic Emotion Perception
- 2026Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approach
- 2026Distance-aware Soft Prompt Guidance for Multimodal Valence-Arousal Estimation