# Aether Weaver: Multimodal Affective Narrative Co-Generation with Dynamic Scene Graphs

arXiv: https://arxiv.org/abs/2507.21893 · HTML: https://mlsanity.com/paper/2507.21893/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 24 references that are in the map.
- First posted on arXiv 29 Jul 2025.

## Builds on (most cited first, 16 of 16)

- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [Image Generation from Scene Graphs](https://mlsanity.com/paper/1804.01622/) (2018)
- [MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models](https://mlsanity.com/paper/2010.00840/) (2020)
- [SEED-Story: Multimodal Long Story Generation with Large Language Model](https://mlsanity.com/paper/2407.08683/) (2024)
- [Learning to Reason for Long-Form Story Generation](https://mlsanity.com/paper/2503.22828/) (2025)
- [An Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era](https://mlsanity.com/paper/2210.03538/) (2022)
- [Evaluating Creative Short Story Generation in Humans and Large Language Models](https://mlsanity.com/paper/2411.02316/) (2024)
- [Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis](https://mlsanity.com/paper/2503.08741/) (2025)
- [ICM-Assistant: Instruction-tuning Multimodal Large Language Models for Rule-based Explainable Image Content Moderation](https://mlsanity.com/paper/2412.18216/) (2024)
- [Controllable Logical Hypothesis Generation for Abductive Reasoning in Knowledge Graphs](https://mlsanity.com/paper/2505.20948/) (2025)
- [Self-Emotion Blended Dialogue Generation in Social Simulation Agents](https://mlsanity.com/paper/2408.01633/) (2024)
- [An Exploratory Study on Multi-modal Generative AI in AR Storytelling](https://mlsanity.com/paper/2505.15973/) (2025)
- [Kahaani: A Multimodal Co-Creative Storytelling System](https://mlsanity.com/paper/2409.11261/) (2024)

## Built on by (most cited first, 1 of 1)

- [ExpressEdit: Fast Editing of Stylized Facial Expressions with Diffusion Models in Photoshop](https://mlsanity.com/paper/2604.03448/) (2026)

## Abstract (arXiv)

We introduce Aether Weaver, a novel, integrated framework for multimodal narrative co-generation that overcomes limitations of sequential text-to-visual pipelines. Our system concurrently synthesizes textual narratives, dynamic scene graph representations, visual scenes, and affective soundscapes, driven by a tightly integrated, co-generation mechanism. At its core, the Narrator, a large language model, generates narrative text and multimodal prompts, while the Director acts as a dynamic scene graph manager, and analyzes the text to build and maintain a structured representation of the story's world, ensuring spatio-temporal and relational consistency for visual rendering and subsequent narrative generation. Additionally, a Narrative Arc Controller guides the high-level story structure, influencing multimodal affective consistency, further complemented by an Affective Tone Mapper that ensures congruent emotional expression across all modalities. Through qualitative evaluations on a diverse set of narrative prompts encompassing various genres, we demonstrate that Aether Weaver significantly enhances narrative depth, visual fidelity, and emotional resonance compared to cascaded baseline approaches. This integrated framework provides a robust platform for rapid creative prototyping and immersive storytelling experiences.