Emotion Understanding in Videos Through Body, Context, and Visual-Semantic Embedding Loss
Not on the map. Why?
Abstract (arXiv)
We present our winning submission to the First International Workshop on Bodily Expressed Emotion Understanding (BEEU) challenge. Based on recent literature on the effect of context/environment on emotion, as well as visual representations with semantic meaning using word embeddings, we extend the framework of Temporal Segment Network to accommodate these. Our method is verified on the validation set of the Body Language Dataset (BoLD) and achieves 0.26235 Emotion Recognition Score on the test set, surpassing the previous best result of 0.2530.
Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Oct 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 3; 2022: 0; 2023: 6; 2024: 1; 2025: 2; 2026: 2.
14 papers.
Sort:
By area:
- 2023Learning Emotion Representations from Verbal and Nonverbal Communicationcited by 31
- 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classificationcited by 25
- 2024VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoningcited by 16
- 2021SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Losscited by 13
- 2021Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wildcited by 12
- 2021An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wildcited by 12
- 2023Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotioncited by 12
- 2025Towards Fine-Grained Emotion Understanding via Skeleton-Based Micro-Gesture Recognitioncited by 4
- 2023Bodily expressed emotion understanding through integrating Laban movement analysiscited by 3
- 2023Using Scene and Semantic Features for Multi-modal Emotion Recognitioncited by 1
- 2025A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networkscited by 1
- 2023Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspectivecited by 0
- 2026Context-Aware Mixture of Domain Experts for Bodily Expression of Emotion in the Wild3 Aug 2026
- 2026A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognition11 Jun 2026
- 2025Towards Fine-Grained Emotion Understanding via Skeleton-Based Micro-Gesture Recognition15 Jun 2025
- 2025A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks13 Jan 2025
- 2024VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning10 Apr 2024
- 2023Using Scene and Semantic Features for Multi-modal Emotion Recognition1 Aug 2023
- 2023Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion25 Jul 2023
- 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification20 Jul 2023
- 2023Learning Emotion Representations from Verbal and Nonverbal Communication22 May 2023
- 2023Bodily expressed emotion understanding through integrating Laban movement analysis5 Apr 2023
- 2023Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective20 Feb 2023
- 2021SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss30 Sep 2021
2 more
- 2023Learning Emotion Representations from Verbal and Nonverbal Communicationcited by 31
- 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classificationcited by 25
- 2021SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Losscited by 13
- 2021Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wildcited by 12
- 2021An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wildcited by 12
- 2023Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotioncited by 12
- 2025Towards Fine-Grained Emotion Understanding via Skeleton-Based Micro-Gesture Recognitioncited by 4
- 2023Bodily expressed emotion understanding through integrating Laban movement analysiscited by 3
- 2023Using Scene and Semantic Features for Multi-modal Emotion Recognitioncited by 1
- 2025A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networkscited by 1
- 2023Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspectivecited by 0
- 2026A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognitioncited by 0
12 most cited of 13 All 13 (JSON)
- 2026Context-Aware Mixture of Domain Experts for Bodily Expression of Emotion in the Wild3 Aug 2026
- 2026A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognition11 Jun 2026
- 2025Towards Fine-Grained Emotion Understanding via Skeleton-Based Micro-Gesture Recognition15 Jun 2025
- 2025A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks13 Jan 2025
- 2023Using Scene and Semantic Features for Multi-modal Emotion Recognition1 Aug 2023
- 2023Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion25 Jul 2023
- 2023Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification20 Jul 2023
- 2023Learning Emotion Representations from Verbal and Nonverbal Communication22 May 2023
- 2023Bodily expressed emotion understanding through integrating Laban movement analysis5 Apr 2023
- 2023Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective20 Feb 2023
- 2021SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss30 Sep 2021
- 2021An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wild7 Jul 2021
12 newest of 13 All 13 (JSON)