# Mixture of Diffusers for scene composition and high resolution image generation

arXiv: https://arxiv.org/abs/2302.02412 · HTML: https://mlsanity.com/paper/2302.02412/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 32 references that are in the map.
- First posted on arXiv 5 Feb 2023.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)

## Built on by (most cited first, 30 of 56)

- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [Grounded Text-to-Image Synthesis with Attention Refocusing](https://mlsanity.com/paper/2306.05427/) (2023)
- [Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformers](https://mlsanity.com/paper/2405.05945/) (2024)
- [Compositional 3D Scene Generation using Locally Conditioned Diffusion](https://mlsanity.com/paper/2303.12218/) (2023)
- [Expressive Text-to-Image Generation with Rich Text](https://mlsanity.com/paper/2304.06720/) (2023)
- [Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models](https://mlsanity.com/paper/2305.13921/) (2023)
- [CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusion](https://mlsanity.com/paper/2403.05121/) (2024)
- [Prompt-tuning latent diffusion models for inverse problems](https://mlsanity.com/paper/2310.01110/) (2023)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [FouriScale: A Frequency Perspective on Training-Free High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.12963/) (2024)
- [4K4DGen: Panoramic 4D Generation at 4K Resolution](https://mlsanity.com/paper/2406.13527/) (2024)
- [ControlMat: A Controlled Generative Approach to Material Capture](https://mlsanity.com/paper/2309.01700/) (2023)
- [MultiBooth: Towards Generating All Your Concepts in an Image from Text](https://mlsanity.com/paper/2404.14239/) (2024)
- [ElasticDiffusion: Training-free Arbitrary Size Image Generation through Global-Local Content Separation](https://mlsanity.com/paper/2311.18822/) (2023)
- [Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinement](https://mlsanity.com/paper/2411.06558/) (2024)
- [Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images](https://mlsanity.com/paper/2308.16582/) (2023)
- [StableCodec: Taming One-Step Diffusion for Extreme Image Compression](https://mlsanity.com/paper/2506.21977/) (2025)
- [HiDiffusion: Unlocking Higher-Resolution Creativity and Efficiency in Pretrained Diffusion Models](https://mlsanity.com/paper/2311.17528/) (2023)
- [Training-free Regional Prompting for Diffusion Transformers](https://mlsanity.com/paper/2411.02395/) (2024)
- [JointNet: Extending Text-to-Image Diffusion for Dense Distribution Modeling](https://mlsanity.com/paper/2310.06347/) (2023)
- [ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models](https://mlsanity.com/paper/2403.02084/) (2024)
- [ControlVideo: Conditional Control for One-shot Text-driven Video Editing and Beyond](https://mlsanity.com/paper/2305.17098/) (2023)
- [Bracket Diffusion: HDR Image Generation by Consistent LDR Denoising](https://mlsanity.com/paper/2405.14304/) (2024)
- [Inf-DiT: Upsampling Any-Resolution Image with Memory-Efficient Diffusion Transformer](https://mlsanity.com/paper/2405.04312/) (2024)
- [JointDiT: Enhancing RGB-Depth Joint Modeling with Diffusion Transformers](https://mlsanity.com/paper/2505.00482/) (2025)
- [AdaDiffSR: Adaptive Region-aware Dynamic Acceleration Diffusion Model for Real-World Image Super-Resolution](https://mlsanity.com/paper/2410.17752/) (2024)
- [TwinDiffusion: Enhancing Coherence and Efficiency in Panoramic Image Generation with Diffusion Models](https://mlsanity.com/paper/2404.19475/) (2024)
- [Noise Calibration: Plug-and-play Content-Preserving Video Enhancement using Pre-trained Video Diffusion Models](https://mlsanity.com/paper/2407.10285/) (2024)
- [Latent Swap Joint Diffusion for 2D Long-Form Latent Generation](https://mlsanity.com/paper/2502.05130/) (2025)
- [DMM: Building a Versatile Image Generation Model via Distillation-Based Model Merging](https://mlsanity.com/paper/2504.12364/) (2025)

## Abstract (arXiv)

Diffusion methods have been proven to be very effective to generate images while conditioning on a text prompt. However, and although the quality of the generated images is unprecedented, these methods seem to struggle when trying to generate specific image compositions. In this paper we present Mixture of Diffusers, an algorithm that builds over existing diffusion models to provide a more detailed control over composition. By harmonizing several diffusion processes acting on different regions of a canvas, it allows generating larger images, where the location of each object and style is controlled by a separate diffusion process.