mlsanityData to 7 Oct 2026

MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis

arXiv 2402.05408 ↗First posted 8 Feb 2024cs.CVcited by 123

Not on the map. Why?

Cited by 54 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a Multi-Instance Generation (MIG) task, simultaneously generating multiple instances with diverse controls in one image. Given a set of predefined coordinates and their corresponding descriptions, the task is to ensure that generated instances are accurately at the designated locations and that all instances' attributes adhere to their corresponding description. This broadens the scope of current research on Single-instance generation, elevating it to a more versatile and practical dimension. Inspired by the idea of divide and conquer, we introduce an innovative approach named Multi-Instance Generation Controller (MIGC) to address the challenges of the MIG task. Initially, we break down the MIG task into several subtasks, each involving the shading of a single instance. To ensure precise shading for each instance, we introduce an instance enhancement attention mechanism. Lastly, we aggregate all the shaded instances to provide the necessary information for accurately generating multiple instances in stable diffusion (SD). To evaluate how well generation models perform on the MIG task, we provide a COCO-MIG benchmark along with an evaluation pipeline. Extensive experiments were conducted on the proposed COCO-MIG benchmark, as well as on various commonly used benchmarks. The evaluation results illustrate the exceptional control capabilities of our model in terms of quantity, position, attribute, and interaction. Code and demos will be released at https://migcproject.github.io/.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 58 of its 69 references that are in the map. First posted on arXiv 8 Feb 2024.

Builds on

58 of 69 references.

References · 58

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023Segment Anythingcited by 8,158
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2022Classifier-Free Diffusion Guidancecited by 5,717
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  12. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
38 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  5. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  6. 2018Self-Attention with Relative Position Representationscited by 1,112
  7. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  8. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 774
  9. 2016Generative Adversarial Text to Image Synthesiscited by 735
  10. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  11. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 696
  12. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  13. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  14. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  15. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 640
  16. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 575
  17. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  18. 2022Visual Programming: Compositional visual reasoning without trainingcited by 507
  19. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  20. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  21. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  22. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  23. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
  24. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  25. 2023Diffusion Self-Guidance for Controllable Image Generationcited by 305
  26. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  27. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 280
  28. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  29. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  30. 2022SpaText: Spatio-Textual Representation for Controllable Image Generationcited by 217
  31. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  32. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  33. 2023TF-ICON: Diffusion-Based Training-Free Cross-Domain Image Compositioncited by 182
  34. 2023DiffuMask: Synthesizing Images with Pixel-level Annotations for Semantic Segmentation Using Diffusion Modelscited by 162
  35. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  36. 2023Wavelet-based Fourier Information Interaction with Frequency Diffusion Adjustment for Underwater Image Restorationcited by 86
  37. 2023SIFU: Side-view Conditioned Implicit Function for Real-world Usable Clothed Human Reconstructioncited by 77
  38. 2023DiffusionRig: Learning Personalized Priors for Facial Appearance Editingcited by 74

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 43; 2025: 48; 2026: 32.

123 papers.

Sort:

By area:

  1. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  2. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  3. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedbackcited by 153
  4. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 142
  5. 2024Robust Watermarking Using Generative Priors Against Image Editing: From Benchmarking to Advancescited by 105
  6. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 82
  7. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 58
  8. 2024VAR-CLIP: Text-to-Image Generator with Visual Auto-Regressive Modelingcited by 42
  9. 2025DiP: Taming Diffusion Models in Pixel Spacecited by 42
  10. 2024HiCo: Hierarchical Controllable Diffusion Model for Layout-to-image Generationcited by 39
  11. 2024Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinementcited by 37
  12. 20243DIS: Depth-Driven Decoupled Instance Synthesis for Text-to-Image Generationcited by 36
38 more
  1. 2025CreatiDesign: A Unified Multi-Conditional Diffusion Transformer for Creative Graphic Designcited by 36
  2. 2024MIGC++: Advanced Multi-Instance Generation Controller for Image Synthesiscited by 35
  3. 2024Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesiscited by 35
  4. 2024HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splattingcited by 33
  5. 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generationcited by 32
  6. 2024AeroGen: Enhancing Remote Sensing Object Detection with Diffusion-Driven Data Generationcited by 32
  7. 2025EliGen: Entity-Level Controlled Image Generation with Regional Attentioncited by 32
  8. 2024The Scene Language: Representing Scenes with Programs, Words, and Embeddingscited by 31
  9. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 30
  10. 2024A Survey on Personalized Content Synthesis with Diffusion Modelscited by 29
  11. 2025DragFlow: Unleashing DiT Priors with Region Based Supervision for Drag Editingcited by 29
  12. 2025DreamRenderer: Taming Multi-Instance Attribute Control in Large-Scale Text-to-Image Modelscited by 28
  13. 2024Learning A Physical-aware Diffusion Model Based on Transformer for Underwater Image Enhancementcited by 24
  14. 2024GrounDiT: Grounding Diffusion Transformers via Noisy Patch Transplantationcited by 24
  15. 2024Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activitycited by 22
  16. 2025From Zero to Detail: Deconstructing Ultra-High-Definition Image Restoration from Progressive Spectral Perspectivecited by 22
  17. 2025Investigating Text Insulation and Attention Mechanisms for Complex Visual Text Generationcited by 22
  18. 2024Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprintcited by 21
  19. 2024RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Modelscited by 18
  20. 2024Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generationcited by 18
  21. 2024IFAdapter: Instance Feature Control for Grounded Text-to-Image Generationcited by 18
  22. 2024Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxycited by 18
  23. 2025PlanGen: Towards Unified Layout Planning and Image Generation in Auto-Regressive Vision Language Modelscited by 18
  24. 2024ODGEN: Domain-specific Object Detection Data Generation with Diffusion Modelscited by 17
  25. 2025ContextGen: Contextual Layout Anchoring for Identity-Consistent Multi-Instance Generationcited by 17
  26. 2025AnyStory: Towards Unified Single and Multiple Subject Personalization in Text-to-Image Generationcited by 16
  27. 2025UltraHR-100K: Enhancing UHR Image Synthesis with A Large-Scale High-Quality Datasetcited by 16
  28. 2024VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generationcited by 15
  29. 2026LUVE : Latent-Cascaded Ultra-High-Resolution Video Generation with Dual Frequency Expertscited by 13
  30. 2024Scene Graph Disentanglement and Composition for Generalizable Complex Image Generationcited by 11
  31. 2025LaRender: Training-Free Occlusion Control in Image Generation via Latent Renderingcited by 11
  32. 2025Seg2Any: Open-set Segmentation-Mask-to-Image Generation with Precise Shape and Semantic Controlcited by 10
  33. 2025YOLO-Count: Differentiable Object Counting for Text-to-Image Generationcited by 10
  34. 2024ROICtrl: Boosting Instance Control for Visual Generationcited by 9
  35. 2025InstanceAssemble: Layout-Aware Image Generation via Instance Assembling Attentioncited by 9
  36. 2025OverLayBench: A Benchmark for Layout-to-Image Generation with Dense Overlapscited by 9
  37. 2025BideDPO: Conditional Image Generation with Simultaneous Text and Condition Alignmentcited by 9
  38. 2024SimuScope: Realistic Endoscopic Synthetic Dataset Generation through Surgical Simulation and Diffusion Modelscited by 8

50 most cited of 123 All 123 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024InstanceDiffusion: Instance-level Control for Image Generation
  2. 2026InstanceControl: Controllable Complex Image Generation without Instance Labeling
  3. 2025InstanceV: Instance-Level Video Generation
  4. 2023FineControlNet: Fine-level Text Control for Image Generation with Spatially Aligned Text Control Injection
  5. 2024Generating Compositional Scenes via Text-to-image RGBA Instance Generation
  6. 2023Training-Free Location-Aware Text-to-Image Synthesis
  7. 2024AnyControl: Create Your Artwork with Versatile Control on Text-to-Image Generation
  8. 2025A Two-Stage System for Layout-Controlled Image Generation using Large Language Models and Diffusion Models