mlsanityData to 7 Oct 2026

Thinking in Boxes: 3D Editing in Real Images Made Easy

arXiv 2606.20556 ↗First posted 18 Jun 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Text and 2D-conditioning interfaces provide weak, ambiguous control over spatial transformations in image editing -- particularly under large object motions and camera changes. Prior work has used 3D primitives such as boxes, but only as loose conditioning signals indicating approximate object location rather than specifying the transformation. We instead use 3D boxes as structured specifications: the user provides the input and output boxes of the edit, casting editing as a well-posed geometry problem. This ``thinking in boxes'' interface, where each box face is color-coded to convey 3D orientation, gives precise control over translation, rotation, scaling, and viewpoint changes in real images while preserving scene and object identity, and recovering previously unseen object regions. To ground transformations in scene appearance, we introduce a depth-aligned planar floor as a global reference frame, shaded with depth-aware cues. Conditioned on this structure, an image generator produces consistent results under large transformations. Trained in two stages -- on synthetic multi-object scenes and a small set of real-world videos from Objectron -- the system generalizes to complex, in-the-wild real images. Our method operates directly on real photographs and substantially outperforms recent state-of-the-art methods on large 3D edits.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 69 of its 74 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

69 of 74 references.

References · 69

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  5. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  8. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  9. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  10. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  11. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  12. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
38 more
  1. 2025DINOv3cited by 1,297
  2. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  3. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  4. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  5. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  6. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  7. 2025SAM 3: Segment Anything with Conceptscited by 835
  8. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  9. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  10. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  11. 2023Emergent Correspondence from Image Diffusioncited by 437
  12. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  13. 2023AnyDoor: Zero-shot Object-level Image Customizationcited by 411
  14. 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 375
  15. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  16. 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
  17. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editingcited by 293
  18. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  19. 2023DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Modelscited by 193
  20. 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Modelscited by 179
  21. 2025Native and Compact Structured Latents for 3D Generationcited by 157
  22. 2020Objectron: A Large Scale Dataset of Object-Centric Videos in the Wild with Pose Annotationscited by 155
  23. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
  24. 2023Prodigy: An Expeditiously Adaptive Parameter-Free Learnercited by 129
  25. 2025EasyControl: Adding Efficient and Flexible Control for Diffusion Transformercited by 118
  26. 2024Stable Flow: Vital Layers for Training-Free Image Editingcited by 111
  27. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  28. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 62
  29. 2023Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3Dcited by 59
  30. 2025OminiControl2: Efficient Conditioning for Diffusion Transformerscited by 56
  31. 2025TokenVerse: Versatile Multi-concept Personalization in Token Modulation Spacecited by 51
  32. 2023Alchemist: Parametric Control of Material Properties with Diffusion Modelscited by 38
  33. 2024Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videoscited by 38
  34. 2025LightLab: Controlling Light Sources in Images with Diffusion Modelscited by 36
  35. 2024GeoDiffuser: Geometry-Based Image Editing with Diffusion Modelscited by 34
  36. 2024GoodDrag: Towards Good Practices for Drag Editing with Diffusion Modelscited by 33
  37. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  38. 2025Generating Multi-Image Synthetic Data for Text-to-Image Customizationcited by 25

50 most cited of 69 All 69 (JSON)

Built on by

Not cited yet.