# ARBEE: Towards Automated Recognition of Bodily Expression of Emotion In the Wild

arXiv: https://arxiv.org/abs/1808.09568 · HTML: https://mlsanity.com/paper/1808.09568/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Aug 2018.

## Built on by (most cited first, 30 of 34)

- [iMiGUE: An Identity-free Video Dataset for Micro-Gesture Understanding and Emotion Analysis](https://mlsanity.com/paper/2107.00285/) (2021)
- [Benchmarking Micro-action Recognition: Dataset, Methods, and Applications](https://mlsanity.com/paper/2403.05234/) (2024)
- [Learning Emotion Representations from Verbal and Nonverbal Communication](https://mlsanity.com/paper/2305.13500/) (2023)
- [A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insights](https://mlsanity.com/paper/2407.08428/) (2024)
- [MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models](https://mlsanity.com/paper/2508.09210/) (2025)
- [VLLMs Provide Better Context for Emotion Understanding Through Common Sense Reasoning](https://mlsanity.com/paper/2404.07078/) (2024)
- [JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groups](https://mlsanity.com/paper/2404.04458/) (2024)
- [Leveraging Semantic Scene Characteristics and Multi-Stream Convolutional Architectures in a Contextual Approach for Video-Based Visual Emotion Recognition in the Wild](https://mlsanity.com/paper/2105.07484/) (2021)
- [An audiovisual and contextual approach for categorical and continuous emotion recognition in-the-wild](https://mlsanity.com/paper/2107.03465/) (2021)
- [Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion](https://mlsanity.com/paper/2307.13463/) (2023)
- [PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition](https://mlsanity.com/paper/2207.14807/) (2022)
- [Identity-free Artificial Emotional Intelligence via Micro-Gesture Understanding](https://mlsanity.com/paper/2405.13206/) (2024)
- [R^3-VQA: "Read the Room" by Video Social Reasoning](https://mlsanity.com/paper/2505.04147/) (2025)
- [TalkCuts: A Large-Scale Dataset for Multi-Shot Human Speech Video Generation](https://mlsanity.com/paper/2510.07249/) (2025)
- [OmniOPSD: Rationale-Privileged On-Policy Self-Distillation for Affective Computing](https://mlsanity.com/paper/2606.15920/) (2026)
- [HICEM: A High-Coverage Emotion Model for Artificial Emotional Intelligence](https://mlsanity.com/paper/2206.07593/) (2022)
- [Affective Video Content Analysis: Decade Review and New Perspectives](https://mlsanity.com/paper/2310.17212/) (2023)
- [Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding](https://mlsanity.com/paper/2601.16449/) (2026)
- [MUG: Multi-human Graph Network for 3D Mesh Reconstruction from 2D Pose](https://mlsanity.com/paper/2205.12583/) (2022)
- [Bodily expressed emotion understanding through integrating Laban movement analysis](https://mlsanity.com/paper/2304.02187/) (2023)
- [A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation](https://mlsanity.com/paper/2308.08849/) (2023)
- [BQA: Body Language Question Answering Dataset for Video Large Language Models](https://mlsanity.com/paper/2410.13206/) (2024)
- [iMiGUE-3K: A Large-Scale Benchmark for Micro-Gesture Analysis with Self-Supervised Learning](https://mlsanity.com/paper/2605.17179/) (2026)
- [Emotion Recognition from Skeleton Data: A Comprehensive Survey](https://mlsanity.com/paper/2507.18026/) (2025)
- [Incorporating simulated spatial context information improves the effectiveness of contrastive learning models](https://mlsanity.com/paper/2401.15120/) (2024)
- [BEE-NET: A deep neural network to identify in-the-wild Bodily Expression of Emotions](https://mlsanity.com/paper/2402.13955/) (2024)
- [A Heterogeneous Multimodal Graph Learning Framework for Recognizing User Emotions in Social Networks](https://mlsanity.com/paper/2501.07746/) (2025)
- [Mind the Motions: Benchmarking Theory-of-Mind in Everyday Body Language](https://mlsanity.com/paper/2511.15887/) (2025)
- [SayNext-Bench: Why Do LLMs Struggle with Next-Utterance Anticipation?](https://mlsanity.com/paper/2602.00327/) (2026)
- [Medical Face Masks and Emotion Recognition from the Body: Insights from a Deep Learning Perspective](https://mlsanity.com/paper/2302.10021/) (2023)

## Abstract (arXiv)

Humans are arguably innately prepared to comprehend others' emotional expressions from subtle body movements. If robots or computers can be empowered with this capability, a number of robotic applications become possible. Automatically recognizing human bodily expression in unconstrained situations, however, is daunting given the incomplete understanding of the relationship between emotional expressions and body movements. The current research, as a multidisciplinary effort among computer and information sciences, psychology, and statistics, proposes a scalable and reliable crowdsourcing approach for collecting in-the-wild perceived emotion data for computers to learn to recognize body languages of humans. To accomplish this task, a large and growing annotated dataset with 9,876 video clips of body movements and 13,239 human characters, named BoLD (Body Language Dataset), has been created. Comprehensive statistical analysis of the dataset revealed many interesting insights. A system to model the emotional expressions based on bodily movements, named ARBEE (Automated Recognition of Bodily Expression of Emotion), has also been developed and evaluated. Our analysis shows the effectiveness of Laban Movement Analysis (LMA) features in characterizing arousal, and our experiments using LMA features further demonstrate computability of bodily expression. We report and compare results of several other baseline methods which were developed for action recognition based on two different modalities, body skeleton, and raw image. The dataset and findings presented in this work will likely serve as a launchpad for future discoveries in body language understanding that will enable future robots to interact and collaborate more effectively with humans.