# Affective Computing for Large-Scale Heterogeneous Multimedia Data: A Survey

arXiv: https://arxiv.org/abs/1911.05609 · HTML: https://mlsanity.com/paper/1911.05609/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2019.

## Built on by (most cited first, 18 of 18)

- [Affective Image Content Analysis: Two Decades Review and New Perspectives](https://mlsanity.com/paper/2106.16125/) (2021)
- [Multimodal Engagement Analysis from Facial Videos in the Classroom](https://mlsanity.com/paper/2101.04215/) (2021)
- [SOLVER: Scene-Object Interrelated Visual Emotion Reasoning Network](https://mlsanity.com/paper/2110.12334/) (2021)
- [A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directions](https://mlsanity.com/paper/2305.07665/) (2023)
- [Computational Emotion Analysis From Images: Recent Advances and Future Directions](https://mlsanity.com/paper/2103.10798/) (2021)
- [Addressing Racial Bias in Facial Emotion Recognition](https://mlsanity.com/paper/2308.04674/) (2023)
- [A Multimodal Symphony: Integrating Taste and Sound through Generative AI](https://mlsanity.com/paper/2503.02823/) (2025)
- [A Review of Affective Generation Models](https://mlsanity.com/paper/2202.10763/) (2022)
- [Quantifying the Suicidal Tendency on Social Media: A Survey](https://mlsanity.com/paper/2110.03663/) (2021)
- [Behind the Screens: Uncovering Bias in AI-Driven Video Interview Assessments Using Counterfactuals](https://mlsanity.com/paper/2505.12114/) (2025)
- [Documenting use cases in the affective computing domain using Unified Modeling Language](https://mlsanity.com/paper/2209.09666/) (2022)
- [Interpretable Concept-based Deep Learning Framework for Multimodal Human Behavior Modeling](https://mlsanity.com/paper/2502.10145/) (2025)
- [Non-verbal Real-time Human-AI Interaction in Constrained Robotic Environments](https://mlsanity.com/paper/2603.01804/) (2026)
- [Analysis of the use of color and its emotional relationship in visual creations based on experiences during the context of the COVID-19 pandemic](https://mlsanity.com/paper/2203.13770/) (2022)
- [Emotion Embeddings unicodex2014 Learning Stable and Homogeneous Abstractions from Heterogeneous Affective Datasets](https://mlsanity.com/paper/2308.07871/) (2023)
- [Maia: A Real-time Non-Verbal Chat for Human-AI Interaction](https://mlsanity.com/paper/2402.06385/) (2024)
- [Introducing WARM-VR: Benchmark Dataset for Multimodal Wearable Affect Recognition in Virtual Reality](https://mlsanity.com/paper/2605.00184/) (2026)
- [Multi-Source and Multi-Dimensional Information Fusion in Resource-Constrained Edge Networks](https://mlsanity.com/paper/2610.13298/) (2026)

## Abstract (arXiv)

The wide popularity of digital photography and social networks has generated a rapidly growing volume of multimedia data (i.e., image, music, and video), resulting in a great demand for managing, retrieving, and understanding these data. Affective computing (AC) of these data can help to understand human behaviors and enable wide applications. In this article, we survey the state-of-the-art AC technologies comprehensively for large-scale heterogeneous multimedia data. We begin this survey by introducing the typical emotion representation models from psychology that are widely employed in AC. We briefly describe the available datasets for evaluating AC algorithms. We then summarize and compare the representative methods on AC of different multimedia types, i.e., images, music, videos, and multimodal data, with the focus on both handcrafted features-based methods and deep learning methods. Finally, we discuss some challenges and future directions for multimedia affective computing.