mlsanityData to 14 Oct 2026

CoMPaSS: Enhancing Spatial Understanding in Text-to-Image Diffusion Models

arXiv 2412.13195 ↗First posted 17 Dec 2024cs.CVcited by 7

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-to-image (T2I) diffusion models excel at generating photorealistic images but often fail to render accurate spatial relationships. We identify two core issues underlying this common failure: 1) the ambiguous nature of data concerning spatial relationships in existing datasets, and 2) the inability of current text encoders to accurately interpret the spatial semantics of input descriptions. We propose CoMPaSS, a versatile framework that enhances spatial understanding in T2I models. It first addresses data ambiguity with the Spatial Constraints-Oriented Pairing (SCOP) data engine, which curates spatially-accurate training data via principled constraints. To leverage these priors, CoMPaSS also introduces the Token ENcoding ORdering (TENOR) module, which preserves crucial token ordering information lost by text encoders, thereby reinforcing the prompt's linguistic structure. Extensive experiments on four popular T2I models (UNet and MMDiT-based) show CoMPaSS sets a new state of the art on key spatial benchmarks, with substantial relative gains on VISOR (+98%), T2I-CompBench Spatial (+67%), and GenEval Position (+131%). Code is available at https://github.com/blurgyy/CoMPaSS.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 68 of its 86 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

68 of 86 references.

Closest ancestors · 26

Same direction, newest first

  1. 2024CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching4 Apr
  2. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment8 Mar
  3. 2024Box It to Bind It: Unified Layout Control and Attribute Binding in T2I Diffusion Models27 Feb
  4. 2024Pick-and-Draw: Training-free Semantic Guidance for Text-to-Image Personalization30 Jan
  5. 2024InstantID: Zero-shot Identity-Preserving Generation in Seconds15 Jan
  6. 2023Rich Human Feedback for Text-to-Image Generation15 Dec
  7. 2023TokenCompose: Text-to-Image Diffusion with Token-level Supervision6 Dec
  8. 2023Paragraph-to-Image Generation with Information-Enriched Diffusion Model24 Nov
  9. 2023A Picture is Worth a Thousand Words: Principled Recaptioning Improves Image Generation25 Oct
  10. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment17 Oct
  11. 2023R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation13 Oct
  12. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models13 Aug
14 more
  1. 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
  2. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models23 May
  3. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  4. 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models11 Apr
  5. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  7. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  8. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  9. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models26 Oct
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  11. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  12. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  13. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  14. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec

Other references · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
30 more
  1. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  2. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  3. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  4. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  5. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,173
  6. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  7. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  8. 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
  9. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  10. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  11. 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 347
  12. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 332
  13. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  14. 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 328
  15. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolutioncited by 315
  16. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
  17. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
  18. 2022CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformerscited by 307
  19. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  20. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
  21. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 219
  22. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Modelscited by 141
  23. 2018Image Generation from Layoutcited by 105
  24. 2022Benchmarking Spatial Relationships in Text-to-Image Generationcited by 97
  25. 2016Scribbler: Controlling Deep Image Synthesis with Sketch and Colorcited by 78
  26. 2024TC-Bench: Benchmarking Temporal Compositionality in Text-to-Video and Image-to-Video Generationcited by 43
  27. 2024Scalable Ranked Preference Optimization for Text-to-Image Generationcited by 33
  28. 2024Diffusion-RPO: Aligning Diffusion Models through Relative Preference Optimizationcited by 32
  29. 2024Getting it Right: Improving Spatial Consistency in Text-to-Image Modelscited by 30
  30. 2016Convolutional Sketch Inversioncited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Progressive Prompt Detailing for Improved Alignment in Text-to-Image Generative Modelscited by 4
  2. 2025Data-Driven Loss Functions for Inference-Time Optimization in Text-to-Imagecited by 3
  3. 2026Circuit Mechanisms for Spatial Relation Generation in Diffusion Transformerscited by 3
  4. 2026Guided Trajectory Optimization with Sparse Scaling for Test-Time Diffusioncited by 2
  5. 2025ESPLoRA: Enhanced Spatial Precision with Low-Rank Adaption in Text-to-Image Diffusion Models for High-Definition Synthesiscited by 1
  6. 2026Anchoring and Steering Diffusion: Enhancing the Faithfulness of Text-to-Image Generation at Inference Timecited by 0
  7. 2026Math2Visual-X: A Modular Framework for Pedagogically Aligned Lower-Primary Math Visuals Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025InfSplign: Inference-Time Spatial Alignment of Text-to-Image Diffusion Models
  2. 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Models
  3. 2026Not all tokens contribute equally to diffusion learning
  4. 2025Spatial Transport Optimization by Repositioning Attention Map for Training-Free Text-to-Image Synthesis
  5. 2023Semantic Guidance Tuning for Text-To-Image Diffusion Models
  6. 2026SpatialBench-UC: Uncertainty-Aware Evaluation of Spatial Prompt Following in Text-to-Image Generation
  7. 2026Benchmarking Layout-Guided Diffusion Models through Unified Semantic-Spatial Evaluation in Closed and Open Settings
  8. 2024Diffusion Lens: Interpreting Text Encoders in Text-to-Image Pipelines