Thinking in Boxes: 3D Editing in Real Images Made Easy
Not on the map. Why?
Abstract (arXiv)
Text and 2D-conditioning interfaces provide weak, ambiguous control over spatial transformations in image editing -- particularly under large object motions and camera changes. Prior work has used 3D primitives such as boxes, but only as loose conditioning signals indicating approximate object location rather than specifying the transformation. We instead use 3D boxes as structured specifications: the user provides the input and output boxes of the edit, casting editing as a well-posed geometry problem. This ``thinking in boxes'' interface, where each box face is color-coded to convey 3D orientation, gives precise control over translation, rotation, scaling, and viewpoint changes in real images while preserving scene and object identity, and recovering previously unseen object regions. To ground transformations in scene appearance, we introduce a depth-aligned planar floor as a global reference frame, shaded with depth-aware cues. Conditioned on this structure, an image generator produces consistent results under large transformations. Trained in two stages -- on synthetic multi-object scenes and a small set of real-world videos from Objectron -- the system generalizes to complex, in-the-wild real images. Our method operates directly on real photographs and substantially outperforms recent state-of-the-art methods on large 3D edits.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 69 of its 74 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
69 of 74 references.
References · 69
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
38 more
- 2025DINOv3cited by 1,297
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
- 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
- 2023Emergent Correspondence from Image Diffusioncited by 437
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
- 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
- 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
- 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
- 2025SAM 3D: 3Dfy Anything in Imagescited by 230
- 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
- 2025Native and Compact Structured Latents for 3D Generationcited by 157
- 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
- 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
- 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
- 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
- 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
- 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
- 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
- 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
- 2025OminiControl2: Efficient Conditioning for Diffusion Transformerscited by 56
- 2025TokenVerse: Versatile Multi-concept Personalization in Token Modulation Spacecited by 51
- 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
- 2024Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videoscited by 38
- 2025LightLab: Controlling Light Sources in Images with Diffusion Modelscited by 36
- 2024GeoDiffuser: Geometry-Based Image Editing with Diffusion Modelscited by 34
- 2024GoodDrag: Towards Good Practices for Drag Editing with Diffusion Modelscited by 33
- 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
- 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25
50 most cited of 69 All 69 (JSON)
Built on by
Not cited yet.