mlsanityData to 14 Oct 2026

Improving Compositional Generation with Diffusion Models Using Lift Scores

arXiv 2505.13740 ↗First posted 19 May 2025cs.LG cs.AI cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We introduce a novel resampling criterion using lift scores, for improving compositional generation in diffusion models. By leveraging the lift scores, we evaluate whether generated samples align with each single condition and then compose the results to determine whether the composed prompt is satisfied. Our key insight is that lift scores can be efficiently approximated using only the original diffusion model, requiring no additional training or external modules. We develop an optimized variant that achieves relatively lower computational overhead during inference while maintaining effectiveness. Through extensive experiments, we demonstrate that lift scores significantly improved the condition alignment for compositional generation across 2D synthetic data, CLEVR position tasks, and text-to-image synthesis. Our code is available at http://rainorangelemon.github.io/complift.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 39 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

33 of 39 references.

Closest ancestors · 15

Same direction, newest first

  1. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps16 Jan
  2. 2024Not All Noises Are Created Equally:Diffusion Noise Selection and Optimization19 Jul
  3. 2024Diffusion Rejection Sampling28 May
  4. 2023Correcting Diffusion Generation through Resampling10 Dec
  5. 2023Interpretable Diffusion via Information Decomposition12 Oct
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  7. 2023Restart Sampling for Improving Generative Processes26 Jun
  8. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC22 Feb
  9. 2022Refining Generative Process with Discriminator Guidance in Score-based Diffusion Models28 Nov
  10. 2022Classifier-Free Diffusion Guidance26 Jul
  11. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
3 more
  1. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  5. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  6. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  7. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  8. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  9. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  10. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  11. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  12. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 313
6 more
  1. 2023Your Diffusion Model is Secretly a Zero-Shot Classifiercited by 306
  2. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Modelscited by 83
  3. 2023Robust Classification via a Single Diffusion Modelcited by 79
  4. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesiscited by 51
  5. 2023If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selectioncited by 40
  6. 2023Separate-and-Enhance: Compositional Finetuning for Text2Image Diffusion Modelscited by 19

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026TILT: Improving Compositional Generation in Diffusion Models with a Model-Intrinsic Reward
  2. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
  3. 2026Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate
  4. 2025Composition and Alignment of Diffusion Models using Constrained Learning
  5. 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation
  6. 2024All Seeds Are Not Equal: Enhancing Compositional Text-to-Image Generation with Reliable Random Seeds
  7. 2024Diffusion Beats Autoregressive: An Evaluation of Compositional Generation in Text-to-Image Models
  8. 2025Role Bias in Diffusion Models: Diagnosing and Mitigating through Intermediate Decomposition