mlsanityData to 7 Oct 2026

Multi-modal Expression Recognition with Ensemble Method

arXiv 2303.10033 ↗First posted 17 Mar 2023cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper presents our submission to the Expression Classification Challenge of the fifth Affective Behavior Analysis in-the-wild (ABAW) Competition. In our method, multimodal feature combinations extracted by several different pre-trained models are applied to capture more effective emotional information. For these combinations of visual and audio modal features, we utilize two temporal encoders to explore the temporal contextual information in the data. In addition, we employ several ensemble strategies for different experimental settings to obtain the most accurate expression recognition results. Our system achieves the average F1 Score of 0.45774 on the validation set.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 30 references that are in the map. First posted on arXiv 17 Mar 2023.

Builds on

22 of 30 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2019Searching for MobileNetV3cited by 1,984
  7. 2016CNN Architectures for Large-Scale Audio Classificationcited by 701
  8. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  9. 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
  10. 2019Expression, Affect, Action Unit Recognition: Aff-Wild2, Multi-Task Learning and ArcFacecited by 222
  11. 2018Deep Affect Prediction in-the-wild: Aff-Wild Database and Challenge, Deep Architectures, and Beyondcited by 216
  12. 2020Analysing Affective Behavior in the First ABAW 2020 Competitioncited by 178
10 more
  1. 2021Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Frameworkcited by 174
  2. 2021Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Studycited by 168
  3. 2021Analysing Affective Behavior in the second ABAW2 Competitioncited by 163
  4. 2016Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distributioncited by 158
  5. 2019Face Behavior a la carte: Expressions, Affect and Action Units in a Single Networkcited by 155
  6. 2020DFEW: A Large-Scale Database for Recognizing Dynamic Facial Expressions in the Wildcited by 128
  7. 2014Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognitioncited by 111
  8. 2022ABAW: Learning from Synthetic Data & Multi-Task Learning Challengescited by 110
  9. 2022FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videoscited by 69
  10. 2020Improved Residual Networks for Image and Video Recognitioncited by 52

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 1; 2025: 1.

3 papers.

  1. 2023Multi-modal Facial Affective Analysis based on Masked Autoencodercited by 23
  2. 2024Boosting Continuous Emotion Recognition with Self-Pretraining using Masked Autoencoders, Temporal Convolutional Networks, and Transformerscited by 6
  3. 2025Emotion Recognition with CLIP and Sequential Learningcited by 6