mlsanityData to 7 Oct 2026

Image Sculpting: Precise Object Editing with 3D Geometry Control

arXiv 2401.01702 ↗First posted 2 Jan 2024cs.GR cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

We present Image Sculpting, a new framework for editing 2D images by incorporating tools from 3D geometry and graphics. This approach differs markedly from existing methods, which are confined to 2D spaces and typically rely on textual instructions, leading to ambiguity and limited control. Image Sculpting converts 2D objects into 3D, enabling direct interaction with their 3D geometry. Post-editing, these objects are re-rendered into 2D, merging into the original image to produce high-fidelity results through a coarse-to-fine enhancement process. The framework supports precise, quantifiable, and physically-plausible editing options such as pose editing, rotation, translation, 3D composition, carving, and serial addition. It marks an initial step towards combining the creative freedom of generative models with the precision of graphics pipelines.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 62 of its 88 references that are in the map. First posted on arXiv 2 Jan 2024.

Builds on

62 of 88 references.

References · 62

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2023Segment Anythingcited by 8,158
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Generative Adversarial Networkscited by 5,603
  12. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
38 more
  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  11. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  12. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  13. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  14. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  15. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  16. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  17. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  18. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,204
  19. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  20. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  21. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  22. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 990
  23. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  24. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  25. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  26. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  27. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  28. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  29. 2019Interpreting the Latent Space of GANs for Semantic Face Editingcited by 691
  30. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  31. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  32. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  33. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  34. 2015Holistically-Nested Edge Detectioncited by 590
  35. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  36. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  37. 2015A Neural Algorithm of Artistic Stylecited by 480
  38. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 9; 2025: 24; 2026: 12.

45 papers.

Sort:

By area:

  1. 2024PhysMotion: Physics-Grounded Dynamics From a Single Imagecited by 45
  2. 2024LeviTor: 3D Trajectory Oriented Image-to-Video Synthesiscited by 39
  3. 2024Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videoscited by 38
  4. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  5. 2025ObjectMover: Generative Object Movement with Video Priorcited by 30
  6. 2024Videoshop: Localized Semantic Video Editing with Noise-Extrapolated Diffusion Inversioncited by 27
  7. 2025VideoHandles: Editing 3D Object Compositions in Videos Using Video Generative Priorscited by 14
  8. 2024Customizing Text-to-Image Diffusion with Object Viewpoint Controlcited by 13
  9. 2025Training-free Geometric Image Editing on Diffusion Modelscited by 13
  10. 20253D-Fixup: Advancing Photo Editing with 3D Priorscited by 12
  11. 2025ORIDa: Object-centric Real-world Image Composition Datasetcited by 11
  12. 2025I2V3D: Controllable image-to-video generation with 3D guidancecited by 9
33 more
  1. 2025GenSpace: Benchmarking Spatially-Aware Image Generationcited by 9
  2. 2025SceneDesigner: Controllable Multi-Object Image Generation with 9-DoF Pose Manipulationcited by 9
  3. 2025Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxycited by 8
  4. 20253D Object Manipulation in a Single Image using Generative Modelscited by 7
  5. 2025BlenderFusion: 3D-Grounded Visual Editing and Generative Compositingcited by 7
  6. 2024DragGaussian: Enabling Drag-style Manipulation on 3D Gaussian Representationcited by 6
  7. 2024Smart Vision-Language Reasonerscited by 5
  8. 2025MObI: Multimodal Object Inpainting Using Diffusion Modelscited by 4
  9. 2025Attention in Diffusion Model: A Surveycited by 4
  10. 2026SeeThrough3D: Occlusion Aware 3D Control in Text-to-Image Generationcited by 4
  11. 2025Anti-Inpainting: A Proactive Defense Approach against Malicious Diffusion-based Inpainters under Unknown Conditionscited by 3
  12. 2025Free-Form Scene Editor: Enabling Multi-Round Object Manipulation like in a 3D Enginecited by 3
  13. 2026Ctrl&Shift: High-Quality Geometry-Aware Object Manipulation in Visual Generationcited by 3
  14. 2026Direct 3D-Aware Object Insertion via Decomposed Visual Proxiescited by 3
  15. 2025BlobCtrl: Taming Controllable Blob for Element-level Image Editingcited by 2
  16. 2025Generative Blocks World: Moving Things Around in Picturescited by 2
  17. 2025PromptFlare: Prompt-Generalized Defense via Cross-Attention Decoy in Diffusion-Based Inpaintingcited by 2
  18. 2025GeoDiffusion: A Training-Free Framework for Accurate 3D Geometric Conditioning in Image Generationcited by 2
  19. 2025Neural USD: An object-centric framework for iterative editing and controlcited by 2
  20. 2025GimbalDiffusion: Gravity-Aware Camera Control for Video Generationcited by 2
  21. 2026Talk2Move: Reinforcement Learning for Text-Instructed Object-Level Geometric Transformation in Scenescited by 2
  22. 2026Geometric Image Editing via Effects-Sensitive In-Context Inpainting with Diffusion Transformerscited by 1
  23. 2026GeoEdit: Geometry-Aware Object Editing via Dual-Branch Denoisingcited by 1
  24. 2024DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editingcited by 0
  25. 2025MotionDiff: Training-free Zero-shot Interactive Motion Editing via Flow-assisted Multi-view Diffusioncited by 0
  26. 2025FreeInsert: Personalized Object Insertion with Geometric and Style Controlcited by 0
  27. 20253DProxyImg: Controllable 3D-Aware Animation Synthesis from Single Image via 2D-3D Aligned Proxy Embeddingcited by 0
  28. 2026LightMover: Generative Light Movement with Color and Intensity Controlscited by 0
  29. 2026ObjectMorpher: 3D-Aware Image Editing via Deformable 3DGS Modelscited by 0
  30. 2026BoxCtrl: 3D-Aware Visual Prompting for Geometric Image Editingcited by 0
  31. 2026ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inferencecited by 0
  32. 2026SpatialDiff: 3D-Aware Object Movement via Implicit Spatial Modelingcited by 0
  33. 2026OREO: Fidelity Alignment in 3D Generation via On-the-fly Rendering-Editing Optimizationcited by 0