mlsanityData to 7 Oct 2026

Compositional Sculpting of Iterative Generative Processes

arXiv 2309.16115 ↗First posted 28 Sep 2023cs.LGcited by 26

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

High training costs of generative models and the need to fine-tune them for specific tasks have created a strong interest in model reuse and composition. A key challenge in composing iterative generative processes, such as GFlowNets and diffusion models, is that to realize the desired target distribution, all steps of the generative process need to be coordinated, and satisfy delicate balance conditions. In this work, we propose Compositional Sculpting: a general approach for defining compositions of iterative generative processes. We then introduce a method for sampling from these compositions built on classifier guidance. We showcase ways to accomplish compositional sculpting in both GFlowNets and diffusion models. We highlight two binary operations unicodex2014 the harmonic mean (p1 otimes p2) and the contrast (p1 unicodex25D1,p2) between pairs, and the generalization of these operations to multiple component distributions. We offer empirical results on image and molecular generation tasks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 51 of its 73 references that are in the map. First posted on arXiv 28 Sep 2023.

Builds on

51 of 73 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC22 Feb
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Probabilistic Models19 Jun
  7. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 44

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  9. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  11. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
32 more
  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2017Neural Message Passing for Quantum Chemistrycited by 3,460
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  9. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  10. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2022Reproducible scaling laws for contrastive language-image learningcited by 1,163
  13. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  14. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
  15. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 591
  16. 2019Graph Transformer Networkscited by 520
  17. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  18. 2021GFlowNet Foundationscited by 286
  19. 2019Compositionality decomposed: how do neural networks generalise?cited by 264
  20. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  21. 2022Scaling Up Models and Data with textttt5x and textttseqiocited by 187
  22. 2021MARS: Markov Molecular Sampling for Multi-objective Drug Discoverycited by 125
  23. 2022Multi-Objective GFlowNetscited by 100
  24. 2023A theory of continuous generative flow networkscited by 98
  25. 2021Learning to Compose Visual Relationscited by 72
  26. 2021Unsupervised Learning of Compositional Energy Conceptscited by 71
  27. 2022Testing Relational Understanding in Text-Guided Image Generationcited by 59
  28. 2017Generative Models of Visually Grounded Imaginationcited by 44
  29. 2023Unsupervised Compositional Concepts Discovery with Text-to-Image Generative Modelscited by 36
  30. 2017AdaGAN: Boosting Generative Modelscited by 34
  31. 2022Unifying Generative Models with GFlowNets and Beyondcited by 24
  32. 2017Boosted Generative Modelscited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 9; 2025: 4; 2026: 11.

26 papers.

Sort:

By area:

  1. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  2. 2023Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Modelscited by 46
  3. 2024Compositional Generative Modeling: A Single Model is Not All You Needcited by 41
  4. 2025Generative Trajectory Stitching through Diffusion Compositioncited by 38
  5. 2024Diffusion Models as Constrained Samplers for Optimization with Unknown Constraintscited by 33
  6. 2023RadEdit: stress-testing biomedical vision models via diffusion image editingcited by 28
  7. 2024Compositional Image Decomposition with Diffusion Modelscited by 12
  8. 2024On Divergence Measures for Training GFlowNetscited by 6
  9. 2025Product of Experts for Visual Generationcited by 6
  10. 2024Few-Shot Task Learning through Inverse Generative Modelingcited by 4
  11. 2024Streaming Bayes GFlowNetscited by 4
  12. 2026Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulationcited by 4
14 more
  1. 2024Embarrassingly Parallel GFlowNetscited by 3
  2. 2025Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNetscited by 2
  3. 2026Compositional Visual Planning via Inference-Time Diffusion Scalingcited by 2
  4. 2025Ctrl-VI: Controllable Video Synthesis via Variational Inferencecited by 1
  5. 2026Logical Guidance for the Exact Composition of Diffusion Modelscited by 1
  6. 2026Long-Text-to-Image Generation via Compositional Prompt Decompositioncited by 1
  7. 2024Improving Fine-Grained Control via Aggregation of Multiple Diffusion Modelscited by 0
  8. 2026Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generationcited by 0
  9. 2026LPH-VTON: Resolving the Structure-Texture Dilemma of Virtual Try-On via Latent Process Handovercited by 0
  10. 2026Diffusion Domain Expansion: Learning to Coordinate Pre-trained Diffusion Modelscited by 0
  11. 2026Divide-and-Denoise: A Game-Theoretic Method for Fairly Composing Diffusion Modelscited by 0
  12. 2026Path-dependent Discrete Amortized Inferencecited by 0
  13. 2026PathGuide: Dynamic Classifier-Free Guidance via On-Policy Transport Alignmentcited by 0
  14. 2026Evidence-Aligned Local Composition of Discrete Experts for Sequence Restorationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
  2. 2023Exploring Compositional Visual Generation with Latent Classifier Guidance
  3. 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation
  4. 2025Improving Compositional Generation with Diffusion Models Using Lift Scores
  5. 2026TILT: Improving Compositional Generation in Diffusion Models with a Model-Intrinsic Reward
  6. 2025Composition and Alignment of Diffusion Models using Constrained Learning
  7. 2026Catastrophic Compositional Generation: Why Vanilla Diffusion Models Fail to Extrapolate
  8. 2025How Compositional Generalization and Creativity Improve as Diffusion Models are Trained