mlsanityData to 7 Oct 2026

Exploring Emotion Features and Fusion Strategies for Audio-Video Emotion Recognition

arXiv 2012.13912 ↗First posted 27 Dec 2020cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

The audio-video based emotion recognition aims to classify a given video into basic emotions. In this paper, we describe our approaches in EmotiW 2019, which mainly explores emotion features and feature fusion strategies for audio and visual modality. For emotion features, we explore audio feature with both speech-spectrogram and Log Mel-spectrogram and evaluate several facial features with different CNN models and different emotion pretrained strategies. For fusion strategies, we explore intra-modal and cross-modal fusion methods, such as designing attention mechanisms to highlights important emotion feature, exploring feature concatenation and factorized bilinear pooling (FBP) for cross-modal feature fusion. With careful evaluation, we obtain 65.5% on the AFEW validation set and 62.48% on the test set and rank third in the challenge.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 3; 2023: 7; 2024: 10; 2025: 1.

24 papers.

Sort:

By area:

  1. 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 117
  2. 2022The Effect of Model Compression on Fairness in Facial Expression Recognitioncited by 13
  3. 2021A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognitioncited by 11
  4. 2022Learning from Synthetic Data: Facial Expression Classification based on Ensemble of Multi-task Networkscited by 5
  5. 2023Fuzzy Approach for Audio-Video Emotion Recognition in Computer Games for Childrencited by 5
  6. 2023Hierarchical Audio-Visual Information Fusion with Multi-label Joint Decoding for MER 2023cited by 3
  7. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  8. 2024Facing Asymmetry -- Uncovering the Causal Link between Facial Symmetry and Expression Classifiers using Synthetic Interventionscited by 2
  9. 2023A Weakly Supervised Approach to Emotion-change Prediction and Improved Mood Inferencecited by 1
  10. 2024Detail-Enhanced Intra- and Inter-modal Interaction for Audio-Visual Emotion Recognitioncited by 1
  11. 2021Signal Transformer: Complex-valued Attention and Meta-Learning for Signal Recognitioncited by 0
  12. 2021Label quality in AffectNet: results of crowd-based re-annotationcited by 0
12 more
  1. 2022Event and Entity Extraction from Generated Video Captionscited by 0
  2. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
  3. 2023GMTR: Graph Matching Transformerscited by 0
  4. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
  5. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  6. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  7. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  8. 2024SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognitioncited by 0
  9. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  10. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  11. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0
  12. 2025Mask6D: Masked Pose Priors For 6D Object Pose Estimationcited by 0