mlsanityData to 14 Oct 2026

Two-Stream Aural-Visual Affect Analysis in the Wild

arXiv 2002.03399 ↗First posted 9 Feb 2020cs.CV cs.LG stat.MLcited by 51

Not on the map. Why?

Abstract (arXiv)

Human affect recognition is an essential part of natural human-computer interaction. However, current methods are still in their infancy, especially for in-the-wild data. In this work, we introduce our submission to the Affective Behavior Analysis in-the-wild (ABAW) 2020 competition. We propose a two-stream aural-visual analysis model to recognize affective behavior from videos. Audio and image streams are first processed separately and fed into a convolutional neural network. Instead of applying recurrent architectures for temporal analysis we only use temporal convolutions. Furthermore, the model is given access to additional features extracted during face-alignment. At training time, we exploit correlations between different emotion representations to improve performance. Our model achieves promising results on the challenging Aff-Wild2 database.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). First posted on arXiv 9 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 12; 2022: 12; 2023: 9; 2024: 14; 2025: 4.

51 papers.

Sort:

By area:

  1. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  2. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  3. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  4. 2022ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Multi-Task Learning Challengescited by 156
  5. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  6. 2023ABAW: Valence-Arousal Estimation, Expression Recognition, Action Unit Detection & Emotional Reaction Intensity Estimation Challengescited by 110
  7. 2024The 6th Affective Behavior Analysis in-the-wild (ABAW) Competitioncited by 78
  8. 20247th ABAW Competition: Multi-Task Learning and Compound Expression Recognitioncited by 53
  9. 2022A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognitioncited by 34
  10. 2022NR-DFERNet: Noise-Robust Network for Dynamic Facial Expression Recognitioncited by 27
  11. 2021A Multi-modal and Multi-task Learning Method for Action Unit and Expression Recognitioncited by 24
  12. 2023Leveraging TCN and Transformer for effective visual-audio fusion in continuous emotion recognitioncited by 20
38 more
  1. 2021A Multi-task Mean Teacher for Semi-supervised Facial Affective Behavior Analysiscited by 19
  2. 2021Prior Aided Streaming Network for Multi-task Affective Recognitionat the 2nd ABAW2 Competitioncited by 18
  3. 2022Audio-Visual Fusion for Emotion Recognition in the Valence-Arousal Space Using Joint Cross-Attentioncited by 18
  4. 2022Coarse-to-Fine Cascaded Networks with Smooth Predicting for Video Facial Expression Recognitioncited by 17
  5. 2021Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognitioncited by 16
  6. 2024Recursive Joint Cross-Modal Attention for Multimodal Fusion in Dimensional Emotion Recognitioncited by 15
  7. 2024Joint Multimodal Transformer for Emotion Recognition in the Wildcited by 14
  8. 2021An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wildcited by 12
  9. 2021Multitask Multi-database Emotion Recognitioncited by 12
  10. 2022Multi-modal Multi-label Facial Action Unit Detection with Transformercited by 10
  11. 2022An Ensemble Approach for Facial Expression Analysis in Videocited by 9
  12. 2024Textualized and Feature-based Models for Compound Multimodal Emotion Recognition in the Wildcited by 9
  13. 2021Causal affect prediction model using a facial image sequencecited by 8
  14. 2025MAVEN: Multi-modal Attention for Valence-Arousal Emotion Networkcited by 8
  15. 2022Continuous-Time Audiovisual Fusion with Recurrence vs. Attention for In-The-Wild Affect Recognitioncited by 7
  16. 2023Recursive Joint Attention for Audio-Visual Fusion in Regression based Emotion Recognitioncited by 6
  17. 2024Boosting Continuous Emotion Recognition with Self-Pretraining using Masked Autoencoders, Temporal Convolutional Networks, and Transformerscited by 6
  18. 2025Emotion Recognition with CLIP and Sequential Learningcited by 6
  19. 2024Multimodal Fusion Method with Spatiotemporal Sequences and Relationship Learning for Valence-Arousal Estimationcited by 5
  20. 2021Multi-modal Affect Analysis using standardized data within subjects in the Wildcited by 4
  21. 2022Multitask Emotion Recognition Model with Knowledge Distillation and Task Discriminatorcited by 4
  22. 2022An Ensemble Approach for Multiple Emotion Descriptors Estimation Using Multi-task Learningcited by 3
  23. 2022A Study on the Ambiguity in Human Annotation of German Oral History Interviews for Perceived Emotion Recognition and Sentiment Analysiscited by 2
  24. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  25. 2024Inconsistency-Aware Cross-Attention for Audio-Visual Fusion in Dimensional Emotion Recognitioncited by 1
  26. 2025ARPGNet: Appearance- and Relation-aware Parallel Graph Attention Fusion Network for Facial Expression Recognitioncited by 1
  27. 2021Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognitioncited by 0
  28. 2023Facial Expression Recognition using Squeeze and Excitation-powered Swin Transformerscited by 0
  29. 2023A Unified Transformer-based Network for multimodal Emotion Recognitioncited by 0
  30. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
  31. 2023GMTR: Graph Matching Transformerscited by 0
  32. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
  33. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  34. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  35. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  36. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  37. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  38. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0

50 most cited of 51 All 51 (JSON)