# SEWA DB: A Rich Database for Audio-Visual Emotion and Sentiment Research in the Wild

arXiv: https://arxiv.org/abs/1901.02839 · HTML: https://mlsanity.com/paper/1901.02839/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2019.

## Built on by (most cited first, 30 of 62)

- [EMOCA: Emotion Driven Monocular Face Capture and Animation](https://mlsanity.com/paper/2204.11312/) (2022)
- [Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework](https://mlsanity.com/paper/2103.15792/) (2021)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [Efficient Emotional Adaptation for Audio-Driven Talking-Head Generation](https://mlsanity.com/paper/2309.04946/) (2023)
- [The Multimodal Sentiment Analysis in Car Reviews (MuSe-CaR) Dataset: Collection, Insights and Improvements](https://mlsanity.com/paper/2101.06053/) (2021)
- [COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition](https://mlsanity.com/paper/2206.05833/) (2022)
- [Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions](https://mlsanity.com/paper/2404.11023/) (2024)
- [Affect2MM: Affective Analysis of Multimedia Content Using Emotion Causality](https://mlsanity.com/paper/2103.06541/) (2021)
- [Affective Processes: stochastic modelling of temporal context for emotion and facial expression recognition](https://mlsanity.com/paper/2103.13372/) (2021)
- [Refashioning Emotion Recognition Modelling: The Advent of Generalised Large Models](https://mlsanity.com/paper/2308.11578/) (2023)
- [Continuous Emotion Recognition with Audio-visual Leader-follower Attentive Fusion](https://mlsanity.com/paper/2107.01175/) (2021)
- [BAH Dataset for Ambivalence/Hesitancy Recognition in Videos for Digital Behavioural Change](https://mlsanity.com/paper/2505.19328/) (2025)
- [Speech Emotion Diarization: Which Emotion Appears When?](https://mlsanity.com/paper/2306.12991/) (2023)
- [Didn't see that coming: a survey on non-verbal social human behavior forecasting](https://mlsanity.com/paper/2203.02480/) (2022)
- [From the Lab to the Wild: Affect Modeling via Privileged Information](https://mlsanity.com/paper/2305.10919/) (2023)
- [Estimating the Uncertainty in Emotion Attributes using Deep Evidential Regression](https://mlsanity.com/paper/2306.06760/) (2023)
- [Affective Game Computing: A Survey](https://mlsanity.com/paper/2309.14104/) (2023)
- [Towards Context-Aware Emotion Recognition Debiasing from a Causal Demystification Perspective via De-confounded Training](https://mlsanity.com/paper/2407.04963/) (2024)
- [MuSe-Toolbox: The Multimodal Sentiment Analysis Continuous Annotation Fusion and Discrete Class Transformation Toolbox](https://mlsanity.com/paper/2107.11757/) (2021)
- [End-to-End Speech Emotion Recognition: Challenges of Real-Life Emergency Call Centers Data Recordings](https://mlsanity.com/paper/2110.14957/) (2021)
- [EALD-MLLM: Emotion Analysis in Long-sequential and De-identity videos with Multi-modal Large Language Model](https://mlsanity.com/paper/2405.00574/) (2024)
- [End-to-End Label Uncertainty Modeling in Speech Emotion Recognition using Bayesian Neural Networks and Label Distribution Learning](https://mlsanity.com/paper/2209.15449/) (2022)
- [Affective Computing Has Changed: The Foundation Model Disruption](https://mlsanity.com/paper/2409.08907/) (2024)
- [End-to-End Continuous Speech Emotion Recognition in Real-life Customer Service Call Center Conversations](https://mlsanity.com/paper/2310.02281/) (2023)
- [AVCAffe: A Large Scale Audio-Visual Dataset of Cognitive Load and Affect for Remote Work](https://mlsanity.com/paper/2205.06887/) (2022)
- [Affective Video Content Analysis: Decade Review and New Perspectives](https://mlsanity.com/paper/2310.17212/) (2023)
- [GameVibe: A Multimodal Affective Game Corpus](https://mlsanity.com/paper/2407.12787/) (2024)
- [Multilingual Dyadic Interaction Corpus NoXi+J: Toward Understanding Asian-European Non-verbal Cultural Characteristics and their Influences on Engagement](https://mlsanity.com/paper/2409.13726/) (2024)
- [Emotions in the Loop: A Survey of Affective Computing for Emotional Support](https://mlsanity.com/paper/2505.01542/) (2025)
- [SocialInteractionGAN: Multi-person Interaction Sequence Generation](https://mlsanity.com/paper/2103.05916/) (2021)

## Abstract (arXiv)

Natural human-computer interaction and audio-visual human behaviour sensing systems, which would achieve robust performance in-the-wild are more needed than ever as digital devices are increasingly becoming an indispensable part of our life. Accurately annotated real-world data are the crux in devising such systems. However, existing databases usually consider controlled settings, low demographic variability, and a single task. In this paper, we introduce the SEWA database of more than 2000 minutes of audio-visual data of 398 people coming from six cultures, 50% female, and uniformly spanning the age range of 18 to 65 years old. Subjects were recorded in two different contexts: while watching adverts and while discussing adverts in a video chat. The database includes rich annotations of the recordings in terms of facial landmarks, facial action units (FAU), various vocalisations, mirroring, and continuously valued valence, arousal, liking, agreement, and prototypic examples of (dis)liking. This database aims to be an extremely valuable resource for researchers in affective computing and automatic human sensing and is expected to push forward the research in human behaviour analysis, including cultural studies. Along with the database, we provide extensive baseline experiments for automatic FAU detection and automatic valence, arousal and (dis)liking intensity estimation.