# Predicated Diffusion: Predicate Logic-Based Attention Guidance for Text-to-Image Diffusion Models arXiv: https://arxiv.org/abs/2311.16117 ยท HTML: https://mlsanity.com/paper/2311.16117/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 31 of its 45 references that are in the map. - First posted on arXiv 3 Oct 2023. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) ## Builds on (most cited first, 30 of 31) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022) - [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021) - [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015) - [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022) - [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016) - [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023) - [Compositional Visual Generation with Composable Diffusion Models](https://mlsanity.com/paper/2206.01714/) (2022) - [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022) - [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022) - [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022) - [BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion](https://mlsanity.com/paper/2307.10816/) (2023) - [Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment](https://mlsanity.com/paper/2306.08877/) (2023) - [Re-imagine the Negative Prompt Algorithm: Transform 2D Diffusion into 3D, alleviate Janus problem and Beyond](https://mlsanity.com/paper/2304.04968/) (2023) - [Harnessing Deep Neural Networks with Logic Rules](https://mlsanity.com/paper/1603.06318/) (2016) - [Directed Diffusion: Direct Control of Object Placement through Attention Guidance](https://mlsanity.com/paper/2302.13153/) (2023) - [Shape-Guided Diffusion with Inside-Outside Attention](https://mlsanity.com/paper/2212.00210/) (2022) - [Deep Learning with Logical Constraints](https://mlsanity.com/paper/2205.00523/) (2022) - [Training-Free Location-Aware Text-to-Image Synthesis](https://mlsanity.com/paper/2304.13427/) (2023) ## Built on by (most cited first, 16 of 16) - [A Cat Is A Cat (Not A Dog!): Unraveling Information Mix-ups in Text-to-Image Encoders through Causal Analysis and Embedding Optimization](https://mlsanity.com/paper/2410.00321/) (2024) - [Enhancing MMDiT-Based Text-to-Image Models for Similar Subject Generation](https://mlsanity.com/paper/2411.18301/) (2024) - [MoVer: Motion Verification for Motion Graphics Animations](https://mlsanity.com/paper/2502.13372/) (2025) - [Attention Overlap Is Responsible for The Entity Missing Problem in Text-to-image Diffusion Models!](https://mlsanity.com/paper/2410.20972/) (2024) - [Diffusion Beats Autoregressive: An Evaluation of Compositional Generation in Text-to-Image Models](https://mlsanity.com/paper/2410.22775/) (2024) - [Geometrical Properties of Text Token Embeddings for Strong Semantic Binding in Text-to-Image Generation](https://mlsanity.com/paper/2503.23011/) (2025) - [Attention in Diffusion Model: A Survey](https://mlsanity.com/paper/2504.03738/) (2025) - [Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models](https://mlsanity.com/paper/2510.05509/) (2025) - [Why Settle for Mid: A Probabilistic Viewpoint to Spatial Relationship Alignment in Text-to-image Models](https://mlsanity.com/paper/2506.23418/) (2025) - [Neuro-Symbolic Artificial Intelligence: A Task-Directed Survey in the Black-Box Models Era](https://mlsanity.com/paper/2603.03177/) (2026) - [HarmonPaint: Harmonized Training-Free Diffusion Inpainting](https://mlsanity.com/paper/2507.16732/) (2025) - [MetaLogic: Robustness Evaluation of Text-to-Image Models via Logically Equivalent Prompts](https://mlsanity.com/paper/2510.00796/) (2025) - [UNCAGE: Contrastive Attention Guidance for Masked Generative Transformers in Text-to-Image Generation](https://mlsanity.com/paper/2508.05399/) (2025) - [Feature Space Analysis by Guided Diffusion Model](https://mlsanity.com/paper/2509.07936/) (2025) - [FreeFuse: Multi-Subject LoRA Fusion via Adaptive Token-Level Routing at Test Time](https://mlsanity.com/paper/2510.23515/) (2025) - [AI-T2I: Aggregating-and-Isolating Cross-Attention to Diffusion Models for Text-to-Image Synthesis](https://mlsanity.com/paper/2605.25763/) (2026) ## Abstract (arXiv) Diffusion models have achieved remarkable results in generating high-quality, diverse, and creative images. However, when it comes to text-based image generation, they often fail to capture the intended meaning presented in the text. For instance, a specified object may not be generated, an unnecessary object may be generated, and an adjective may alter objects it was not intended to modify. Moreover, we found that relationships indicating possession between objects are often overlooked. While users' intentions in text are diverse, existing methods tend to specialize in only some aspects of these. In this paper, we propose Predicated Diffusion, a unified framework to express users' intentions. We consider that the root of the above issues lies in the text encoder, which often focuses only on individual words and neglects the logical relationships between them. The proposed method does not solely rely on the text encoder, but instead, represents the intended meaning in the text as propositions using predicate logic and treats the pixels in the attention maps as the fuzzy predicates. This enables us to obtain a differentiable loss function that makes the image fulfill the proposition by minimizing it. When compared to several existing methods, we demonstrated that Predicated Diffusion can generate images that are more faithful to various text prompts, as verified by human evaluators and pretrained image-text models.