mlsanityData to 14 Oct 2026

ConsistCompose: Unified Multimodal Layout Control for Image Composition

arXiv 2511.18333 ↗First posted 23 Nov 2025cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Unified multimodal models that couple visual understanding with image generation have advanced rapidly, yet most systems still focus on visual grounding-aligning language with image regions-while their generative counterpart, linguistic-embedded layout-grounded generation (LELG) for layout-controllable multi-instance generation, remains underexplored and limits precise compositional control. We present ConsistCompose, a unified multimodal framework that embeds layout coordinates directly into language prompts, enabling layout-controlled multi-instance image generation from Interleaved Image-Text within a single generative interface. We further construct ConsistCompose3M, a 3.4M multi-instance generation dataset with layout and identity annotations (2.6M text-guided and 0.8M image-guided data pairs) that provides large-scale supervision for layout-conditioned generation. Within this framework, LELG is instantiated through instance-coordinate binding prompts and coordinate-aware classifier-free guidance, which translate linguistic layout cues into precise spatial control without task-specific branches. Experiments on COCO-Position and MS-Bench show that ConsistCompose substantially improves spatial accuracy over layout-controlled baselines while preserving identity fidelity and competitive general multimodal understanding, establishing a unified paradigm for layout-controllable multimodal image generation.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 59 of its 62 references that are in the map. First posted on arXiv 23 Nov 2025.

Builds on

59 of 62 references.

References · 59

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  5. 2022Flow Matching for Generative Modelingcited by 5,702
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  7. 2024GPT-4o System Cardcited by 4,588
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  12. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
38 more
  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  2. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  3. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  6. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  7. 2025Qwen-Image Technical Reportcited by 1,044
  8. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  9. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  11. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  12. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  13. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  14. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  15. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  16. 2023MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuningcited by 487
  17. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  18. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  19. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  20. 2024OmniGen: Unified Image Generationcited by 394
  21. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  22. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 340
  23. 2021VITON-HD: High-Resolution Virtual Try-On via Misalignment-Aware Normalizationcited by 287
  24. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoningcited by 280
  25. 2025Transfer between Modalities with MetaQueriescited by 236
  26. 2022Sketch-Guided Text-to-Image Diffusion Modelscited by 180
  27. 2024InstanceDiffusion: Instance-level Control for Image Generationcited by 175
  28. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 154
  29. 2025HunyuanImage 3.0 Technical Reportcited by 149
  30. 2024MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidancecited by 144
  31. 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 124
  32. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
  33. 2025Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunitiescited by 85
  34. 2025Detect Anything via Next Point Predictioncited by 73
  35. 2025GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Datasetcited by 70
  36. 2024CreatiLayout: Siamese Multimodal Diffusion Transformer for Creative Layout-to-Image Generationcited by 60
  37. 2025LLMDet: Learning Strong Open-Vocabulary Object Detectors under the Supervision of Large Language Modelscited by 57
  38. 2025OminiControl2: Efficient Conditioning for Diffusion Transformerscited by 57

50 most cited of 59 All 59 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.