mlsanityData to 7 Oct 2026

Composer: Creative and Controllable Image Synthesis with Composable Conditions

arXiv 2302.09778 ↗First posted 20 Feb 2023cs.CV cs.GRcited by 303

Not on the map. Why?

Cited by 173 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent large-scale generative models learned on big data are capable of synthesizing incredible images yet suffer from limited controllability. This work offers a new generation paradigm that allows flexible control of the output image, such as spatial layout and palette, while maintaining the synthesis quality and model creativity. With compositionality as the core idea, we first decompose an image into representative factors, and then train a diffusion model with all these factors as the conditions to recompose the input. At the inference stage, the rich intermediate representations work as composable elements, leading to a huge design space (i.e., exponentially proportional to the number of decomposed factors) for customizable content creation. It is noteworthy that our approach, which we call Composer, supports various levels of conditions, such as text description as the global information, depth map and sketch as the local guidance, color histogram for low-level details, etc. Besides improving controllability, we confirm that Composer serves as a general framework and facilitates a wide range of classical generative tasks without retraining. Code and models will be made available.

Key facts. Cited by 303 papers in the map (data through 7 Oct 2026). Builds on 44 of its 51 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

44 of 51 references.

References · 44

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
32 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  3. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  4. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  5. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  9. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  10. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  11. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  12. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  13. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  14. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  15. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  16. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  17. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  18. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  19. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  20. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  21. 2019Measuring Compositional Generalization: A Comprehensive Method on Realistic Datacited by 280
  22. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 276
  23. 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
  24. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  25. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  26. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 229
  27. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  28. 2021Pixel Difference Networks for Efficient Edge Detectioncited by 167
  29. 2022Pretraining is All You Need for Image-to-Image Translationcited by 160
  30. 2019Image Synthesis From Reconfigurable Layout and Stylecited by 85
  31. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  32. 2022Diffusion-Based Scene Graph to Image Generation with Masked Contrastive Pre-Trainingcited by 56

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 95; 2024: 120; 2025: 60; 2026: 28.

303 papers.

Sort:

By area:

  1. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  2. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  3. 2023ModelScope Text-to-Video Technical Reportcited by 651
  4. 2025VACE: All-in-One Video Creation and Editingcited by 467
  5. 2023VideoComposer: Compositional Video Synthesis with Motion Controllabilitycited by 461
  6. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
  7. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  8. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  9. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  10. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  11. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  12. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
38 more
  1. 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
  2. 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Drivingcited by 258
  3. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  4. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  5. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  6. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  7. 2023StableVideo: Text-driven Consistency-aware Diffusion Video Editingcited by 182
  8. 2023AnyText: Multilingual Visual Text Generation And Editingcited by 165
  9. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion modelscited by 159
  10. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 158
  11. 2024In-Context LoRA for Diffusion Transformerscited by 155
  12. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedbackcited by 153
  13. 2023Localizing Object-level Shape Variations with Text-to-Image Diffusion Modelscited by 142
  14. 2023Cross-Image Attention for Zero-Shot Appearance Transfercited by 135
  15. 2023GlyphControl: Glyph Conditional Control for Visual Text Generationcited by 127
  16. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  17. 2023Animate124: Animating One Image to 4D Dynamic Scenecited by 113
  18. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 111
  19. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  20. 2024MotionClone: Training-Free Motion Cloning for Controllable Video Generationcited by 104
  21. 2023BEVControl: Accurately Controlling Street-view Elements with Multi-perspective Consistency via BEV Sketch Layoutcited by 102
  22. 2023Cones 2: Customizable Image Synthesis with Multiple Subjectscited by 101
  23. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
  24. 2024Learning Temporally Consistent Video Depth from Video Diffusion Priorscited by 100
  25. 2024UniReal: Universal Image Generation and Editing via Learning Real-world Dynamicscited by 100
  26. 2023PreDiff: Precipitation Nowcasting with Latent Diffusion Modelscited by 96
  27. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 95
  28. 2023WoVoGen: World Volume-aware Diffusion for Controllable Multi-camera Driving Scene Generationcited by 93
  29. 2023VideoLCM: Video Latent Consistency Modelcited by 90
  30. 2023Animate-A-Story: Storytelling with Retrieval-Augmented Video Generationcited by 89
  31. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 86
  32. 2023DreamWaltz: Make a Scene with Complex 3D Animatable Avatarscited by 82
  33. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  34. 2025DreamO: A Unified Framework for Image Customizationcited by 82
  35. 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 81
  36. 2023FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesiscited by 79
  37. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
  38. 2023ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Modelscited by 75

50 most cited of 303 All 303 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Advancing Aesthetic Image Generation via Composition Transfer
  2. 2025IP-Composer: Semantic Composition of Visual Concepts
  3. 2023Divide and Compose with Score Based Generative Models
  4. 2025LVLM-Composer's Explicit Planning for Image Generation
  5. 2023Compositional Sculpting of Iterative Generative Processes
  6. 2025Compositional Image Synthesis with Inference-Time Scaling
  7. 2022The Stable Artist: Steering Semantics in Diffusion Latent Space
  8. 2023ControlCom: Controllable Image Composition using Diffusion Model