Text-to-Image Alignment in Denoising-Based Models through Step Selection
Not on the map. Why?
Abstract (arXiv)
Visual generative AI models often encounter challenges related to text-image alignment and reasoning limitations. This paper presents a novel method for selectively enhancing the signal at critical denoising steps, optimizing image generation based on input semantics. Our approach addresses the shortcomings of early-stage signal modifications, demonstrating that adjustments made at later stages yield superior results. We conduct extensive experiments to validate the effectiveness of our method in producing semantically aligned images on Diffusion and Flow Matching model, achieving state-of-the-art performance. Our results highlight the importance of a judicious choice of sampling stage to improve performance and overall image alignment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 45 of its 51 references that are in the map. First posted on arXiv 24 Apr 2025.
Builds on
45 of 51 references.
References · 45
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
33 more
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
- 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
- 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 864
- 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
- 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
- 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
- 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
- 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
- 2022Diffusion Models already have a Semantic Latent Spacecited by 328
- 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
- 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
- 2022Perception Prioritized Training of Diffusion Modelscited by 303
- 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 294
- 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusioncited by 282
- 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 236
- 2023Teaching CLIP to Count to Tencited by 198
- 2023On the Importance of Noise Scheduling for Diffusion Modelscited by 194
- 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignmentcited by 171
- 2022Generative Modelling With Inverse Heat Dissipationcited by 167
- 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometrycited by 125
- 2024InitNO: Boosting Text-to-Image Diffusion Models via Initial Noise Optimizationcited by 123
- 2023Divide & Bind Your Attention for Improved Generative Semantic Nursingcited by 81
- 2023A-STAR: Test-time Attention Segregation and Retention for Text-to-image Synthesiscited by 78
- 2023A Picture is Worth a Thousand Words: Principled Recaptioning Improves Image Generationcited by 41
- 2023Exploiting the Signal-Leak Bias in Diffusion Modelscited by 36
- 2023TIAM -- A Metric for Evaluating Alignment in Text-to-Image Generationcited by 18
- 2024Spatial-Aware Latent Initialization for Controllable Image Generationcited by 15
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025SAGA: Learning Signal-Aligned Distributions for Improved Text-to-Image Generation
- 2026Alignment of Diffusion Model and Flow Matching for Text-to-Image Generation
- 2025Asynchronous Denoising Diffusion Models for Aligning Text-to-Image Generation
- 2024Information Theoretic Text-to-Image Alignment
- 2023Divide, Evaluate, and Refine: Evaluating and Improving Text-to-Image Alignment with Iterative VQA Feedback
- 2023Improving Compositional Text-to-image Generation with Large Vision-Language Models
- 2024Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis
- 2025Aligning Text to Image in Diffusion Models is Easier Than You Think