mlsanityData to 14 Oct 2026

EmoStyle: Affective Conditioning of Style-Specialist Experts for Emotional Image Generation

arXiv 2607.10165 ↗First posted 11 Jul 2026cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Emotion-aware artistic image generation requires an image to match the input prompt, follow the specified artistic style, and convey the target emotion. In this challenge, the main difficulty is that the visual and affective attributes available in the training data are not explicitly provided at test time. Without these attributes, the generator has to decide not only what to depict, but also how the target emotion should be expressed through color, lighting, brushwork, composition, line, and layout. This creates a control gap between the available test prompt and the fine-grained conditions needed for emotion-aware artistic generation. To bridge this gap, we propose EmoStyle, a Z-Image-based framework that converts the input prompt into a structured generation state. An LLM reasoner first predicts affective cues (valence-arousal, dominant emotion, and therapeutic-effect labels) and an aspect-ratio decision. Instead of using these predictions only as additional prompt text, we encode the affective fields into an affective condition vector and inject it into the denoising blocks through AdaLN-style modulation. This allows the inferred control variables to directly guide the generation of intermediate features. Since emotional expression is also style-dependent, we further train a dedicated LoRA adapter for each artistic style bucket and select the corresponding expert during inference, enabling the same affective cues to be rendered with bucket-specific priors for color, texture, brushwork, and composition. Finally, a lightweight VLM-guided candidate selection step ranks the generated images based on prompt alignment, style consistency, emotional expression, and visual quality. In Track 1 of the AffectiveArt Challenge 2026, our USTCPILABTEAM submission achieved first place.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 39 of its 84 references that are in the map. First posted on arXiv 11 Jul 2026.

Builds on

39 of 84 references.

References · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  8. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  9. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  10. 2021Scaling Vision with Sparse Mixture of Expertscited by 747
  11. 2022Mixture-of-Experts with Expert Choice Routingcited by 623
  12. 2023LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Compositioncited by 364
27 more
  1. 2022Inversion-Based Style Transfer with Diffusion Modelscited by 291
  2. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  3. 2023Style Aligned Image Generation via Shared Attentioncited by 216
  4. 2022A Review of Sparse Expert Models in Deep Learningcited by 148
  5. 2023EmoSet: A Large-scale Visual Emotion Dataset with Rich Attributescited by 82
  6. 2024Multi-LoRA Composition for Image Generationcited by 80
  7. 2020Context Based Emotion Recognition using EMOTIC Datasetcited by 73
  8. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 60
  9. 2023RePrompt: Automatic Prompt Editing to Refine AI-Generative Art Towards Precise Expressionscited by 43
  10. 2023ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptationcited by 40
  11. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  12. 2024Graph Mixture of Experts and Memory-augmented Routers for Multivariate Time Series Anomaly Detectioncited by 20
  13. 2022End-to-End Compressed Video Representation Learning for Generic Event Boundary Detectioncited by 16
  14. 2025DreamPoster: A Unified Framework for Image-Conditioned Generative Poster Designcited by 16
  15. 2024Dual-path Collaborative Generation Network for Emotional Video Captioningcited by 14
  16. 2021Generic Event Boundary Detection Challenge at CVPR 2021 Technical Report: Cascaded Temporal Attention Network (CASTANET)cited by 13
  17. 2023Improving Image Captioning via Predicting Structured Conceptscited by 13
  18. 2025EmoArt: A Multidimensional Dataset for Emotion-Aware Artistic Generationcited by 11
  19. 2022Structured Context Transformer for Generic Event Boundary Detectioncited by 10
  20. 2025CoEmoGen: Towards Semantically-Coherent and Scalable Emotional Image Content Generationcited by 9
  21. 2026CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layerscited by 9
  22. 2024Sentiment-oriented Transformer-based Variational Autoencoder Network for Live Video Commentingcited by 8
  23. 2026FACE-net: Factual Calibration and Emotion Augmentation for Retrieval-enhanced Emotional Video Captioningcited by 7
  24. 2026A Multi-Agent Framework with Structured Reasoning and Reflective Refinement for Multimodal Empathetic Response Generationcited by 5
  25. 2026Towards Accurate Emotion-Attributed Video Captioning via Fine-grained Emotion-Cause Pair Extractioncited by 4
  26. 2026Audio-Visual Exchange-Aware Token Pruning for Efficient Audio-Visual Captioningcited by 3
  27. 2025Structured Context Learning for Generic Event Boundary Detectioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025EmoStyle: Emotion-Driven Image Stylization
  2. 2025EmoCtrl: Controllable Emotional Image Content Generation
  3. 2026EmoScene: A Dual-space Dataset for Controllable Affective Image Generation
  4. 2024EmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models
  5. 2026ProFocus: Interpreting Affective Experience in Artistic Images with Progressive Visual Focusing
  6. 2025MUSE: Manipulating Unified Framework for Synthesizing Emotions in Images via Test-Time Optimization
  7. 2026EmoKGEdit: Training-free Affective Injection via Visual Cue Transformation
  8. 2026EmoSpace: Immersive Affective Image Generation Guided by Fine-Grained Emotion Prototypes