EmoScene: A Dual-space Dataset for Controllable Affective Image Generation
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
Text-to-image diffusion models achieve high visual fidelity, yet fine-grained affective control remains difficult because textual emotion cues often fail to specify the visual perceptual factors underlying affective expression. Existing visual-affect datasets are likewise often limited to discrete labels, specific domains, or limited supervision of perceptual attributes. We introduce EmoScene, a large-scale dual-space dataset for controllable affective image generation, containing 1.2M images across more than 300 scene categories. Its affective space jointly represents discrete emotions and continuous valence--arousal--dominance (VAD), its perceptual space records measurable appearance attributes, and contextual descriptions ground both in scene semantics. EmoScene combines multi-model annotation with human-in-the-loop quality control. A random audit of 30,519 images yields 91.22% agreement on discrete emotion labels, while an independent multi-rater evaluation yields Fleiss' kappa=0.85. After controlling for source and scene composition, affective dimensions and perceptual attributes exhibit stable associations across data sources, reflecting statistical tendencies rather than deterministic visual rules. To demonstrate the dataset's utility, we further develop AffectCtrl, which learns residuals in the conditioning space of frozen diffusion models to support categorical emotion generation and continuous control over VAD, brightness, and saturation. AffectCtrl achieves 85.75% categorical emotion accuracy, outperforms EmotiCrafter in valence and arousal control under a shared evaluation protocol, and obtains Pearson correlations of 0.673--0.765 across all five continuous axes. These results demonstrate that EmoScene provides a scalable data foundation for analyzing and controlling affective expression in visual generation.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 38 of its 52 references that are in the map. First posted on arXiv 1 Apr 2026.
Builds on
38 of 52 references.
Closest ancestors · 1
Same direction, newest first
Other references · 37
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2017AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wildcited by 490
- 2024Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuningcited by 117
- 2021ArtEmis: Affective Language for Visual Artcited by 103
- 2023EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributescited by 81
- 2016Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmarkcited by 71
- 2022FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videoscited by 69
25 more
- 2022A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advancescited by 59
- 2023GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognitioncited by 38
- 2024MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognitioncited by 27
- 2024EmoEdit: Evoking Emotions through Image Manipulationcited by 27
- 2024Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspectivecited by 24
- 2025EmotiCrafter: Text-to-Emotional-Image Generation based on Valence-Arousal Modelcited by 23
- 2021Emotion Recognition from Multiple Modalities: Fundamentals and Methodologiescited by 22
- 2024FindingEmo: An Image Dataset for Emotion Recognition in the Wildcited by 20
- 2024A Survey on Facial Expression Recognition of Static and Dynamic Emotionscited by 14
- 2024JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groupscited by 13
- 2025Affective Image Editing: Shaping Emotional Factors via Text Descriptionscited by 11
- 2021AffectGAN: Affect-Based Generative Art Driven by Semanticscited by 10
- 2024A Unified and Interpretable Emotion Representation and Expression Generationcited by 10
- 2025CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generationcited by 9
- 2025Modeling Thousands of Human Annotators for Generalizable Text-to-Image Person Re-identificationcited by 8
- 2023A Comprehensive Survey on Affective Computing; Challenges, Trends, Applications, and Future Directionscited by 7
- 2024A Review of Human Emotion Synthesis Based on Generative Technologycited by 7
- 2025Emotions in the Loop: A Survey of Affective Computing for Emotional Supportcited by 5
- 2025EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?cited by 5
- 2025AttriCtrl: Fine-Grained Control of Aesthetic Attribute Intensity in Diffusion Modelscited by 5
- 2025Towards Robust Multimodal Emotion Recognition under Missing Modalities and Distribution Shiftscited by 4
- 2025EmoCtrl: Controllable Emotional Image Content Generationcited by 4
- 2024Knowledge-based Emotion Recognition using Large Language Modelscited by 3
- 2025Multimodal Emotion Recognition and Sentiment Analysis in Multi-Party Conversation Contextscited by 2
- 2026EmoSpace: Immersive Affective Image Generation Guided by Fine-Grained Emotion Prototypescited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026EmoWorld: A Decoupled Affective Field for Controllable Emotional Video Generation
- 2026Visual Affect Analysis: Predicting Emotions of Image Viewers with Vision-Language Models
- 2026EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation
- 2026Balancing Emotional Alignment and Semantic Consistency in Image Generation via Reinforcement Learning with Valence-Arousal Anchoring
- 2025MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization
- 2025EmoVerse: A MLLMs-Driven Emotion Representation Dataset for Interpretable Visual Emotion Analysis
- 2026Do Emotion Concepts Generalize Across Sources, Modalities, and Architectures in Vision-Language Models?
- 2025Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach