mlsanityData to 7 Oct 2026

Compositional Text-to-Image Generation with Dense Blob Representations

arXiv 2405.08246 ↗First posted 14 May 2024cs.CV cs.AI cs.LGcited by 38

Not on the map. Why?

Cited by 22 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Existing text-to-image models struggle to follow complex text prompts, raising the need for extra grounding inputs for better controllability. In this work, we propose to decompose a scene into visual primitives - denoted as dense blob representations - that contain fine-grained details of the scene while being modular, human-interpretable, and easy-to-construct. Based on blob representations, we develop a blob-grounded text-to-image diffusion model, termed BlobGEN, for compositional generation. Particularly, we introduce a new masked cross-attention module to disentangle the fusion between blob representations and visual features. To leverage the compositionality of large language models (LLMs), we introduce a new in-context learning approach to generate blob representations from text prompts. Our extensive experiments show that BlobGEN achieves superior zero-shot generation quality and better layout-guided controllability on MS-COCO. When augmented by LLMs, our method exhibits superior numerical and spatial correctness on compositional image generation benchmarks. Project page: https://blobgen-2d.github.io.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 51 of its 58 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

51 of 58 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  11. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  12. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
38 more
  1. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  7. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  9. 2021Grounded Language-Image Pre-trainingcited by 1,147
  10. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  11. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
  12. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  13. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  14. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  15. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  16. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  17. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  18. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  19. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  20. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  21. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 371
  22. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  23. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  24. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  25. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  26. 2023Generating Images with Multimodal Language Modelscited by 259
  27. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  28. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 253
  29. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 242
  30. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  31. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
  32. 2023Grounded Text-to-Image Synthesis with Attention Refocusingcited by 163
  33. 2023Emu: Generative Pretraining in Multimodalitycited by 156
  34. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  35. 2023Expressive Text-to-Image Generation with Rich Textcited by 89
  36. 2023Ranni: Taming Text-to-Image Diffusion for Accurate Instruction Followingcited by 77
  37. 2021Controllable and Compositional Generation with Latent-Space Energy-Based Modelscited by 64
  38. 2023Paragraph-to-Image Generation with Information-Enriched Diffusion Modelcited by 39

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 19; 2026: 6.

38 papers.

Sort:

By area:

  1. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 58
  2. 2024GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Modelscited by 28
  3. 2024DiffUHaul: A Training-Free Method for Object Dragging in Imagescited by 22
  4. 2025Through-The-Mask: Mask-based Motion Trajectories for Image-to-Video Generationcited by 20
  5. 2024Improving Compositional Attribute Binding in Text-to-Image Generative Models via Enhanced Text Embeddingscited by 18
  6. 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generationcited by 18
  7. 2025PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Modelscited by 18
  8. 2025BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representationscited by 15
  9. 2024Kaleido Diffusion: Improving Conditional Diffusion Models with Autoregressive Latent Modelingcited by 13
  10. 2025Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesiscited by 10
  11. 2025Be Decisive: Noise-Induced Layouts for Multi-Subject Generationcited by 10
  12. 2024Wavelets Are All You Need for Autoregressive Image Generationcited by 9
26 more
  1. 2024ROICtrl: Boosting Instance Control for Visual Generationcited by 9
  2. 2025SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulationcited by 9
  3. 2024Relations, Negations, and Numbers: Looking for Logic in Generative Text-to-Image Modelscited by 8
  4. 2025Beyond Simple Edits: X-Planner for Complex Instruction-Based Image Editingcited by 8
  5. 2025VSC: Visual Search Compositional Text-to-Image Diffusion Modelcited by 7
  6. 2025LOTS of Fashion! Multi-Conditioning for Image Generation via Sketch-Text Pairingcited by 7
  7. 2024Controllable Human Image Generation with Personalized Multi-Garmentscited by 5
  8. 2024GraPE: A Generate-Plan-Edit Framework for Compositional T2I Synthesiscited by 4
  9. 2025Inverse Virtual Try-On: Generating Multi-Category Product-Style Images from Clothed Individualscited by 4
  10. 2025BlobCtrl: Taming Controllable Blob for Element-level Image Editingcited by 2
  11. 2025LLMControl: Grounded Control of Text-to-Image Diffusion-based Synthesis with Multimodal LLMscited by 2
  12. 2024Generating Intermediate Representations for Compositional Text-To-Image Generationcited by 1
  13. 2025CardioComposer: Leveraging Differentiable Geometry for Compositional Control of Anatomical Diffusion Modelscited by 1
  14. 2025InfSplign: Inference-Time Spatial Alignment of Text-to-Image Diffusion Modelscited by 1
  15. 2026From Plans to Pixels: Learning to Plan and Orchestrate for Open-Ended Image Editingcited by 1
  16. 2026Envisioning Beyond the Few: Disentangled Semantics and Primitives for Few-Shot Atypical Layout-to-Image Generationcited by 1
  17. 2024Improving Fine-Grained Control via Aggregation of Multiple Diffusion Modelscited by 0
  18. 2025Comp-Attn: Present-and-Align Attention for Compositional Video Generationcited by 0
  19. 2025ComposeAnything: Composite Object Priors for Text-to-Image Generationcited by 0
  20. 2025MALeR: Improving Compositional Fidelity in Layout-Guided Generationcited by 0
  21. 2025Anatomica: Localized Control over Geometric and Topological Properties for Anatomical Diffusion Modelscited by 0
  22. 2025ClimaOoD: Improving Anomaly Segmentation via Physically Realistic Synthetic Datacited by 0
  23. 2026Multi-Level Conditioning by Pairing Localized Text and Sketch for Fashion Image Generationcited by 0
  24. 2026DrivePTS: A Progressive Learning Framework with Textual and Structural Enhancement for Driving Scene Generationcited by 0
  25. 2026Spatially-Grounded Text-to-Video Generation via Inference-Time Gradient-Free Optimizationcited by 0
  26. 2026Panoptic Scene Program Diffusion Transformercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts
  2. 2025Object-level Visual Prompts for Compositional Image Generation
  3. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models
  4. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing
  5. 2025HiCoGen: Hierarchical Compositional Text-to-Image Generation in Diffusion Models via Reinforcement Learning
  6. 2024All Seeds Are Not Equal: Enhancing Compositional Text-to-Image Generation with Reliable Random Seeds
  7. 2025CompAlign: Improving Compositional Text-to-Image Generation with a Complex Benchmark and Fine-Grained Feedback
  8. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty