# Compositional Sculpting of Iterative Generative Processes

arXiv: https://arxiv.org/abs/2309.16115 · HTML: https://mlsanity.com/paper/2309.16115/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 51 of its 73 references that are in the map.
- First posted on arXiv 28 Sep 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 51)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)

## Built on by (most cited first, 27 of 27)

- [Amortizing intractable inference in diffusion models for vision, language, and control](https://mlsanity.com/paper/2405.20971/) (2024)
- [Visual Anagrams: Generating Multi-View Optical Illusions with Diffusion Models](https://mlsanity.com/paper/2311.17919/) (2023)
- [Compositional Generative Modeling: A Single Model is Not All You Need](https://mlsanity.com/paper/2402.01103/) (2024)
- [Generative Trajectory Stitching through Diffusion Composition](https://mlsanity.com/paper/2503.05153/) (2025)
- [Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints](https://mlsanity.com/paper/2402.18012/) (2024)
- [RadEdit: stress-testing biomedical vision models via diffusion image editing](https://mlsanity.com/paper/2312.12865/) (2023)
- [Compositional Image Decomposition with Diffusion Models](https://mlsanity.com/paper/2406.19298/) (2024)
- [On Divergence Measures for Training GFlowNets](https://mlsanity.com/paper/2410.09355/) (2024)
- [Product of Experts for Visual Generation](https://mlsanity.com/paper/2506.08894/) (2025)
- [Few-Shot Task Learning through Inverse Generative Modeling](https://mlsanity.com/paper/2411.04987/) (2024)
- [Streaming Bayes GFlowNets](https://mlsanity.com/paper/2411.05899/) (2024)
- [Lucid-XR: An Extended-Reality Data Engine for Robotic Manipulation](https://mlsanity.com/paper/2605.00244/) (2026)
- [Embarrassingly Parallel GFlowNets](https://mlsanity.com/paper/2406.03288/) (2024)
- [Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNets](https://mlsanity.com/paper/2508.15442/) (2025)
- [Compositional Visual Planning via Inference-Time Diffusion Scaling](https://mlsanity.com/paper/2603.02646/) (2026)
- [Ctrl-VI: Controllable Video Synthesis via Variational Inference](https://mlsanity.com/paper/2510.07670/) (2025)
- [Logical Guidance for the Exact Composition of Diffusion Models](https://mlsanity.com/paper/2602.05549/) (2026)
- [Long-Text-to-Image Generation via Compositional Prompt Decomposition](https://mlsanity.com/paper/2604.18258/) (2026)
- [PathGuide: Dynamic Classifier-Free Guidance via On-Policy Transport Alignment](https://mlsanity.com/paper/2608.29107/) (2026)
- [Improving Fine-Grained Control via Aggregation of Multiple Diffusion Models](https://mlsanity.com/paper/2410.01262/) (2024)
- [Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation](https://mlsanity.com/paper/2602.21565/) (2026)
- [LPH-VTON: Resolving the Structure-Texture Dilemma of Virtual Try-On via Latent Process Handover](https://mlsanity.com/paper/2605.14874/) (2026)
- [Diffusion Domain Expansion: Learning to Coordinate Pre-trained Diffusion Models](https://mlsanity.com/paper/2605.23275/) (2026)
- [Divide-and-Denoise: A Game-Theoretic Method for Fairly Composing Diffusion Models](https://mlsanity.com/paper/2606.14756/) (2026)
- [Path-dependent Discrete Amortized Inference](https://mlsanity.com/paper/2608.08644/) (2026)
- [Evidence-Aligned Local Composition of Discrete Experts for Sequence Restoration](https://mlsanity.com/paper/2609.05801/) (2026)
- [Mixture of GFlowNets](https://mlsanity.com/paper/2610.17007/) (2026)

## Abstract (arXiv)

High training costs of generative models and the need to fine-tune them for specific tasks have created a strong interest in model reuse and composition. A key challenge in composing iterative generative processes, such as GFlowNets and diffusion models, is that to realize the desired target distribution, all steps of the generative process need to be coordinated, and satisfy delicate balance conditions. In this work, we propose Compositional Sculpting: a general approach for defining compositions of iterative generative processes. We then introduce a method for sampling from these compositions built on classifier guidance. We showcase ways to accomplish compositional sculpting in both GFlowNets and diffusion models. We highlight two binary operations unicodex2014 the harmonic mean (p1 otimes p2) and the contrast (p1 unicodex25D1,p2) between pairs, and the generalization of these operations to multiple component distributions. We offer empirical results on image and molecular generation tasks.