mlsanityData to 14 Oct 2026

COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models

arXiv 2606.28696 ↗First posted 27 Jun 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Composition is a high-level visual intent that governs where subjects are placed and how a scene is organized, yet current unified multimodal models remain unreliable at fine-grained composition recognition and struggle to turn such intent into controllable generation. We present COMPASS, the first unified multimodal framework that grounds composition-intent control in a single system spanning both composition perception and composition-guided generation, with a shared expert token τc as the central intent anchor. On the perception side, COMPASS injects composition expertise into an MoE backbone in a minimally invasive manner and distills the inferred intent into τc. On the generation side, COMPASS reuses τc as a global conditioning signal that steers the denoising trajectory, effectively converting passive composition analysis into explicit layout control. To support systematic instruction-following composition learning and evaluation at scale, we construct Comp-11, a large-scale dataset with an 11-class taxonomy and reasoning-augmented annotations. Extensive experiments show that COMPASS substantially improves category-level composition understanding and delivers more composition-consistent, prompt-faithful generation than strong baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 27 Jun 2026.

Builds on

36 of 44 references.

References · 36

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  4. 2025Qwen3-VL Technical Reportcited by 2,793
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  7. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  8. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  9. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  10. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  11. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  12. 2024Emu3: Next-Token Prediction is All You Needcited by 781
24 more
  1. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  2. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  3. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
  4. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  5. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  6. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  7. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  8. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  9. 2023Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfercited by 196
  10. 2024InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generationcited by 170
  11. 2021ArtEmis: Affective Language for Visual Artcited by 104
  12. 2024AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perceptioncited by 65
  13. 2024AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perceptioncited by 54
  14. 2024UNIAA: A Unified Multi-modal Image Aesthetic Assessment Baseline and Benchmarkcited by 42
  15. 2023LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Modelscited by 41
  16. 2025ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understandingcited by 41
  17. 2017Quantitative Analysis of Automatic Image Cropping Algorithms: A Dataset and Comparative Studycited by 27
  18. 2025StyleSSP: Sampling StartPoint Enhancement for Training-free Diffusion-based Method for Style Transfercited by 23
  19. 2025UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texturecited by 22
  20. 2021Image Composition Assessment with Saliency-augmented Multi-pattern Poolingcited by 21
  21. 2024Spatial-Semantic Collaborative Cropping for User Generated Contentcited by 13
  22. 2025Ming-Lite-Uni: Advancements in Unified Architecture for Natural Multimodal Interactioncited by 13
  23. 2023Learning Subject-Aware Cropping by Outpainting Professional Photoscited by 12
  24. 2025PhotoFramer: Multi-modal Image Composition Instructioncited by 9

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Advancing Aesthetic Image Generation via Composition Transfer
  2. 2026Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis
  3. 2025Evaluating Compositional Scene Understanding in Multimodal Generative Models
  4. 2026CoBind: Stage-Aware Compositional Binding for Training-Free Text-to-Image Generation
  5. 2026Compositional Video Generation via Inference-Time Guidance
  6. 2024Learning to Compose: Improving Object Centric Learning by Injecting Compositionality
  7. 2026MM-CondChain: A Programmatically Verified Benchmark for Visually Grounded Deep Compositional Reasoning
  8. 2025Explain Before You Answer: A Survey on Compositional Visual Reasoning