# Modeling Induced Pleasure through Cognitive Appraisal Prediction via Multimodal Fusion

arXiv: https://arxiv.org/abs/2604.23753 · HTML: https://mlsanity.com/paper/2604.23753/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 87 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [Dawn of the transformer era in speech emotion recognition: closing the valence gap](https://mlsanity.com/paper/2203.07378/) (2022)
- [EmoVIT: Revolutionizing Emotion Insights with Visual Instruction Tuning](https://mlsanity.com/paper/2404.16670/) (2024)
- [MER 2025: When Affective Computing Meets Large Language Models](https://mlsanity.com/paper/2504.19423/) (2025)
- [StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2409.00304/) (2024)
- [MAVEN: Multi-modal Attention for Valence-Arousal Emotion Network](https://mlsanity.com/paper/2503.12623/) (2025)
- [Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection](https://mlsanity.com/paper/2409.07901/) (2024)
- [VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models](https://mlsanity.com/paper/2511.02712/) (2025)
- [Transfer-based adaptive tree for multimodal sentiment analysis based on user latent aspects](https://mlsanity.com/paper/2106.14174/) (2021)
- [EmoDNN: Understanding emotions from short texts through a deep neural network ensemble](https://mlsanity.com/paper/2106.01706/) (2021)
- [Interpretable Multimodal Emotion Recognition using Hybrid Fusion of Speech and Image Data](https://mlsanity.com/paper/2208.11868/) (2022)
- [EmoStim: A Database of Emotional Film Clips with Discrete and Componential Assessment](https://mlsanity.com/paper/2306.08196/) (2023)
- [Infer Induced Sentiment of Comment Response to Video: A New Task, Dataset and Baseline](https://mlsanity.com/paper/2407.06115/) (2024)
- [Intelligent Agents with Emotional Intelligence: Current Trends, Challenges, and Future Prospects](https://mlsanity.com/paper/2511.20657/) (2025)
- [Detecting Cognitive Appraisals from Facial Expressions for Interest Recognition](https://mlsanity.com/paper/1609.09761/) (2016)
- [A Blast From the Past: Personalizing Predictions of Video-Induced Emotions using Personal Memories as Context](https://mlsanity.com/paper/2008.12096/) (2020)
- [A Proxy-Based Method for Mapping Discrete Emotions onto VAD model](https://mlsanity.com/paper/2511.12521/) (2025)

## Abstract (arXiv)

Multimodal affective computing analyzes user-generated social media content to predict emotional states. However, a critical gap remains in understanding how visual content shapes cognitive interpretations and elicits specific affective experiences such as pleasure. This study introduces a novel computational model to infer video-induced pleasure via cognitive appraisal variables. The proposed model addresses four challenges: (1) noisy and inconsistent human labels, (2) the semantic gap between "positive emotions" and "pleasure," (3) the scarcity of pleasure-specific datasets, and (4) the limited interpretability of existing black-box fusion methods. Our approach integrates data-driven and cognitive theory-driven methods, using cognitive appraisal theory and a fuzzy model within an innovative framework. The model employs transformer-based architectures and attention mechanisms for fine-grained multimodal feature extraction and interpretable fusion to capture both inter- and intra-modal dynamics associated with pleasure. This enables the prediction of underlying appraisal variables, thereby bridging the semantic gap and enhancing model explainability beyond conventional statistical associations. Experimental results validate the efficacy of the proposed method in detecting video-induced pleasure, achieving a peak accuracy of 0.6624 in predicting pleasure levels. These findings highlight promising implications for affective content recommendation, intelligent media creation, and advancing our understanding of how digital media influences human emotions.