mlsanityData to 14 Oct 2026

Behaviour4All: in-the-wild Facial Behaviour Analysis Toolkit

arXiv 2409.17717 ↗First posted 26 Sep 2024cs.CVcited by 16

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce Behavior4All, a comprehensive, open-source toolkit for in-the-wild facial behavior analysis, integrating Face Localization, Valence-Arousal Estimation, Basic Expression Recognition and Action Unit Detection, all within a single framework. Available in both CPU-only and GPU-accelerated versions, Behavior4All leverages 12 large-scale, in-the-wild datasets consisting of over 5 million images from diverse demographic groups. It introduces a novel framework that leverages distribution matching and label co-annotation to address tasks with non-overlapping annotations, encoding prior knowledge of their relatedness. In the largest study of its kind, Behavior4All outperforms both state-of-the-art and toolkits in overall performance as well as fairness across all databases and tasks. It also demonstrates superior generalizability on unseen databases and on compound expression recognition. Finally, Behavior4All is way times faster than other toolkits.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 36 of its 63 references that are in the map. First posted on arXiv 26 Sep 2024.

Builds on

36 of 63 references.

References · 36

  1. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 492
  2. 2018Characterizing and Avoiding Negative Transfercited by 229
  3. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  4. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  5. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
  6. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  7. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  8. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  9. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  10. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  11. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  12. 2022Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognitioncited by 82
24 more
  1. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  2. 2020Deep Transparent Prediction through Latent Representation Analysiscited by 72
  3. 2022Learn From All: Erasing Attention Consistency for Noisy Label Facial Expression Recognitioncited by 67
  4. 2024Distribution Matching for Multi-Task Learning of Classification Tasks: a Large-Scale Study on Faces & Beyondcited by 60
  5. 2018Deep Adaptive Attention for Joint Facial Action Unit Detection and Face Alignmentcited by 55
  6. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  7. 2023A Deep Neural Architecture for Harmonizing 3-D Input Data Analysis and Decision Making in Medical Imagingcited by 39
  8. 2021Py-Feat: Python Facial Expression Analysis Toolboxcited by 33
  9. 2022MixAugment & Mixup: Augmentation Methods for Facial Expression Recognitioncited by 31
  10. 2024Domain adaptation, Explainability & Fairness in AI for Medical Image Analysis: Diagnosis of COVID-19 based on 3-D Chest CT-scanscited by 30
  11. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  12. 2024COVID-19 Computer-aided Diagnosis through AI-assisted CT Imaging Analysis: Deploying a Medical AI Systemcited by 23
  13. 2023Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognitioncited by 20
  14. 2023Identification of Negative Transfers in Multitask Learning Using Surrogate Modelscited by 20
  15. 2023BTDNet: a Multi-Modal Approach for Brain Tumor Radiogenomic Classificationcited by 20
  16. 2024Uncertainty-guided Contrastive Learning for Single Source Domain Generalisationcited by 14
  17. 2024SAM2CLIP2SAM: Vision Language Model for Segmentation of 3D CT Scans for Covid-19 Detectioncited by 14
  18. 2021Exploiting Emotional Dependencies with Graph Convolutional Networks for Facial Expression Recognitioncited by 12
  19. 2024Ensuring UAV Safety: A Vision-only and Real-time Framework for Collision Avoidance Through Object Detection, Tracking, and Distance Estimationcited by 11
  20. 2017FERA 2017 - Addressing Head Pose in the Third Facial Expression Recognition and Analysis Challengecited by 9
  21. 2024Can Machine Learning Assist in Diagnosis of Primary Immune Thrombocytopenia? A feasibility studycited by 8
  22. 2017Multi-View Dynamic Facial Action Unit Detectioncited by 7
  23. 2024Rethinking Affect Analysis: A Protocol for Ensuring Fairness and Consistencycited by 7
  24. 2022AFFDEX 2.0: A Real-Time Facial Expression Analysis Toolkitcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 14.

16 papers.

Sort:

By area:

  1. 2025DVD: A Comprehensive Dataset for Advancing Violence Detection in Real-World Scenarioscited by 13
  2. 2026From Affect to Complex Behavior: Advancing Multimodal Human-Centered AI at the 10th ABAW Workshop & Competitioncited by 12
  3. 2024Using Prompts to Guide Large Language Models in Imitating a Real Person's Language Stylecited by 6
  4. 2026Multimodal Emotion Regression with Multi-Objective Optimization and VAD-Aware Audio Modeling for the 10th ABAW EMI Trackcited by 2
  5. 2026Solution to the 10th ABAW Expression Recognition Challenge: A Robust Multimodal Framework with Safe Cross-Attention and Modality Dropoutcited by 1
  6. 2026Stage-Adaptive Reliability Modeling for Continuous Valence-Arousal Estimationcited by 1
  7. 2026Multimodal Emotion Recognition via Bi-directional Cross-Attention and Temporal Modelingcited by 1
  8. 2026HSEmotion Team at ABAW-10 Competition: Facial Expression Recognition, Valence-Arousal Estimation, Action Unit Detection and Fine-Grained Violence Classificationcited by 1
  9. 2026Team RAS in 10th ABAW Competition: Multimodal Valence and Arousal Estimation Approachcited by 1
  10. 2026Anchoring Emotions in Text: Robust Multimodal Fusion for Mimicry Intensity Estimationcited by 1
  11. 2026The Circumplex Degeneracy Behind the Rare-Class Limit in Affect Recognitioncited by 1
  12. 2026A Shared Latent for Partially-Labeled Multi-Task Facial Affect Recognitioncited by 1
4 more
  1. 2026Causal Supervision of Attention for Affective Behaviour Analysiscited by 0
  2. 2026AffectFlow-DINO: Uncertainty-Aware Multi-Task Affect Estimation via Conditional Rectified Flowcited by 0
  3. 2026Strength-Parity Ensembling with Parameter-Isolated Experts for Multi-Task Affect Recognitioncited by 0
  4. 2026AffectFuse: Cross-Task Feature Fusion with Temporal Modeling for Multi-Task Affective Behavior Analysiscited by 0