# Emotion Understanding in Videos Through Body, Context, and Visual-Semantic Embedding Loss

arXiv: https://arxiv.org/abs/2010.16396 · HTML: https://mlsanity.com/paper/2010.16396/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Oct 2020.

## Built on by (most cited first, 14 of 14)

- [Learning Emotion Representations from Verbal and Nonverbal Communication](https://mlsanity.com/paper/2305.13500/) (2023)
- [Joint Skeletal and Semantic Embedding Loss for Micro-gesture Classification](https://mlsanity.com/paper/2307.10624/) (2023)
- [VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning](https://mlsanity.com/paper/2404.07078/) (2024)
- [SPATE-GAN: Improved Generative Modeling of Dynamic Spatio-Temporal Patterns with an Autoregressive Embedding Loss](https://mlsanity.com/paper/2109.15044/) (2021)
- [Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wild](https://mlsanity.com/paper/2105.07484/) (2021)
- [An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wild](https://mlsanity.com/paper/2107.03465/) (2021)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [Towards Fine-Grained Emotion Understanding via Skeleton-Based Micro-Gesture Recognition](https://mlsanity.com/paper/2506.12848/) (2025)
- [Bodily expressed emotion understanding through integrating Laban movement analysis](https://mlsanity.com/paper/2304.02187/) (2023)
- [Using Scene and Semantic Features for Multi-modal Emotion Recognition](https://mlsanity.com/paper/2308.00228/) (2023)
- [A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks](https://mlsanity.com/paper/2501.07746/) (2025)
- [Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective](https://mlsanity.com/paper/2302.10021/) (2023)
- [A Multi-Modal Framework with Cross-Subject Pseudo-Labeling and Semantic Alignment for Micro-Gesture Recognition](https://mlsanity.com/paper/2606.13030/) (2026)
- [Context-Aware Mixture of Domain Experts for Bodily Expression of Emotion in the Wild](https://mlsanity.com/paper/2608.02331/) (2026)

## Abstract (arXiv)

We present our winning submission to the First International Workshop on Bodily Expressed Emotion Understanding (BEEU) challenge. Based on recent literature on the effect of context/environment on emotion, as well as visual representations with semantic meaning using word embeddings, we extend the framework of Temporal Segment Network to accommodate these. Our method is verified on the validation set of the Body Language Dataset (BoLD) and achieves 0.26235 Emotion Recognition Score on the test set, surpassing the previous best result of 0.2530.