# Object Isolated Attention for Consistent Story Visualization

arXiv: https://arxiv.org/abs/2503.23353 · HTML: https://mlsanity.com/paper/2503.23353/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 30 Mar 2025.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data](https://mlsanity.com/paper/2306.09344/) (2023)
- [Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models](https://mlsanity.com/paper/2305.18292/) (2023)
- [StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation](https://mlsanity.com/paper/2405.01434/) (2024)
- [StoryGAN: A Sequential Conditional GAN for Story Visualization](https://mlsanity.com/paper/1812.02784/) (2018)
- [Training-Free Consistent Text-to-Image Generation](https://mlsanity.com/paper/2402.03286/) (2024)
- [MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis](https://mlsanity.com/paper/2402.05408/) (2024)
- [Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models](https://mlsanity.com/paper/2306.00973/) (2023)
- [Make-A-Story: Visual Memory Conditioned Consistent Story Generation](https://mlsanity.com/paper/2211.13319/) (2022)
- [TaleCrafter: Interactive Story Visualization with Multiple Characters](https://mlsanity.com/paper/2305.18247/) (2023)
- [Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models](https://mlsanity.com/paper/2211.10950/) (2022)
- [HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models](https://mlsanity.com/paper/2312.14091/) (2023)
- [JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation](https://mlsanity.com/paper/2407.06187/) (2024)
- [Instant-3D: Instant Neural Radiance Field Training Towards On-Device AR/VR 3D Reconstruction](https://mlsanity.com/paper/2304.12467/) (2023)
- [FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition](https://mlsanity.com/paper/2405.13870/) (2024)
- [DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion](https://mlsanity.com/paper/2407.12899/) (2024)
- [Character-Centric Story Visualization via Visual Planning and Token Alignment](https://mlsanity.com/paper/2210.08465/) (2022)
- [Mini-DALLE3: Interactive Text to Image by Prompting Large Language Models](https://mlsanity.com/paper/2310.07653/) (2023)
- [StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion](https://mlsanity.com/paper/2404.05979/) (2024)
- [AutoStudio: Crafting Consistent Subjects in Multi-turn Interactive Image Generation](https://mlsanity.com/paper/2406.01388/) (2024)
- [TheaterGen: Character Management with LLM for Consistent Multi-turn Image Generation](https://mlsanity.com/paper/2404.18919/) (2024)

## Built on by (most cited first, 7 of 7)

- [HumanAesExpert: Advancing a Multi-Modality Foundation Model for Human Image Aesthetic Assessment](https://mlsanity.com/paper/2503.23907/) (2025)
- [FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusion](https://mlsanity.com/paper/2512.11274/) (2025)
- [AnimeGamer: Infinite Anime Life Simulation with Next Game State Prediction](https://mlsanity.com/paper/2504.01014/) (2025)
- [ViStoryBench: Comprehensive Benchmark Suite for Story Visualization](https://mlsanity.com/paper/2505.24862/) (2025)
- [Beyond the Golden Data: Resolving the Motion-Vision Quality Dilemma via Timestep Selective Training](https://mlsanity.com/paper/2603.25527/) (2026)
- [SPF-Portrait: Towards Pure Text-to-Portrait Customization with Semantic Pollution-Free Fine-Tuning](https://mlsanity.com/paper/2504.00396/) (2025)
- [LCG: Long-Context Consistent Image Generation with Sparse Relational Attention](https://mlsanity.com/paper/2606.26171/) (2026)

## Abstract (arXiv)

Open-ended story visualization is a challenging task that involves generating coherent image sequences from a given storyline. One of the main difficulties is maintaining character consistency while creating natural and contextually fitting scenes--an area where many existing methods struggle. In this paper, we propose an enhanced Transformer module that uses separate self attention and cross attention mechanisms, leveraging prior knowledge from pre-trained diffusion models to ensure logical scene creation. The isolated self attention mechanism improves character consistency by refining attention maps to reduce focus on irrelevant areas and highlight key features of the same character. Meanwhile, the isolated cross attention mechanism independently processes each character's features, avoiding feature fusion and further strengthening consistency. Notably, our method is training-free, allowing the continuous generation of new characters and storylines without re-tuning. Both qualitative and quantitative evaluations show that our approach outperforms current methods, demonstrating its effectiveness.