# Generative Blocks World: Moving Things Around in Pictures

arXiv: https://arxiv.org/abs/2506.20703 · HTML: https://mlsanity.com/paper/2506.20703/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 60 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 30 of 45)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation](https://mlsanity.com/paper/2212.00774/) (2022)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior](https://mlsanity.com/paper/2303.14184/) (2023)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold](https://mlsanity.com/paper/2305.10973/) (2023)
- [BSP-Net: Generating Compact Meshes via Binary Space Partitioning](https://mlsanity.com/paper/1911.06971/) (2019)
- [DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models](https://mlsanity.com/paper/2307.02421/) (2023)
- [Learning Shape Abstractions by Assembling Volumetric Primitives](https://mlsanity.com/paper/1612.00404/) (2016)
- [CvxNet: Learnable Convex Decomposition](https://mlsanity.com/paper/1909.05736/) (2019)
- [Stable Flow: Vital Layers for Training-Free Image Editing](https://mlsanity.com/paper/2411.14430/) (2024)
- [CSGNet: Neural Shape Parser for Constructive Solid Geometry](https://mlsanity.com/paper/1712.08290/) (2017)
- [LayoutNet: Reconstructing the 3D Room Layout from a Single RGB Image](https://mlsanity.com/paper/1803.08999/) (2018)
- [LooseControl: Lifting ControlNet for Generalized Depth Conditioning](https://mlsanity.com/paper/2312.03079/) (2023)
- [OBJECT 3DIT: Language-guided 3D-aware Image Editing](https://mlsanity.com/paper/2307.11073/) (2023)
- [Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D](https://mlsanity.com/paper/2312.02190/) (2023)
- [Generative Models: What Do They Know? Do They Know Things? Let's Find Out!](https://mlsanity.com/paper/2311.17137/) (2023)
- [Image Sculpting: Precise Object Editing with 3D Geometry Control](https://mlsanity.com/paper/2401.01702/) (2024)
- [Towards High-Fidelity Single-view Holistic Reconstruction of Indoor Scenes](https://mlsanity.com/paper/2207.08656/) (2022)
- [StyleGAN knows Normal, Depth, Albedo, and More](https://mlsanity.com/paper/2306.00987/) (2023)
- [Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitives](https://mlsanity.com/paper/2307.05473/) (2023)

## Built on by (most cited first, 2 of 2)

- [Coupled Diffusion Sampling for Training-Free Multi-View Image Editing](https://mlsanity.com/paper/2510.14981/) (2025)
- [Thinking in Boxes: 3D Editing in Real Images Made Easy](https://mlsanity.com/paper/2606.20556/) (2026)

## Abstract (arXiv)

We describe Generative Blocks World to interact with the scene of a generated image by manipulating simple geometric abstractions. Our method represents scenes as assemblies of convex 3D primitives, and the same scene can be represented by different numbers of primitives, allowing an editor to move either whole structures or small details. Once the scene geometry has been edited, the image is generated by a flow-based method, which is conditioned on depth and a texture hint. Our texture hint takes into account the modified 3D primitives, exceeding the texture-consistency provided by existing techniques. These texture hints (a) allow accurate object and camera moves and (b) preserve the identity of objects. Our experiments demonstrate that our approach outperforms prior works in visual fidelity, editability, and compositional generalization.