mlsanityData to 7 Oct 2026

Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects

arXiv 2412.00518 ↗First posted 30 Nov 2024cs.CV cs.GRcited by 43

Not on the map. Why?

Abstract (arXiv)

We propose a generative technique to edit 3D shapes, represented as meshes, NeRFs, or Gaussian Splats, in approximately 3 seconds, without the need for running an SDS type of optimization. Our key insight is to cast 3D editing as a multiview image inpainting problem, as this representation is generic and can be mapped back to any 3D representation using the bank of available Large Reconstruction Models. We explore different fine-tuning strategies to obtain both multiview generation and inpainting capabilities within the same diffusion model. In particular, the design of the inpainting mask is an important factor of training an inpainting model, and we propose several masking strategies to mimic the types of edits a user would perform on a 3D shape. Our approach takes 3D generative editing from hours to seconds and produces higher-quality results compared to previous works.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 50 of its 58 references that are in the map. First posted on arXiv 30 Nov 2024.

Builds on

50 of 58 references.

References · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  8. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  9. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  10. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  11. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  12. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 924
38 more
  1. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
  2. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  3. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  4. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  5. 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 682
  6. 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 638
  7. 2023Zero123++: a Single Image to Consistent Multi-view Diffusion Base Modelcited by 516
  8. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  9. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 482
  10. 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 421
  11. 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Modelcited by 376
  12. 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 344
  13. 2021Text2Mesh: Text-Driven Neural Stylization for Meshescited by 332
  14. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 303
  15. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 283
  16. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  17. 2023ImageDream: Image-Prompt Multi-view Diffusion for 3D Generationcited by 256
  18. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  19. 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 214
  20. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 154
  21. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  22. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  23. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
  24. 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 114
  25. 2023TextDeformer: Geometry Manipulation using Text Guidancecited by 94
  26. 2023FocalDreamer: Text-driven 3D Editing via Focal-fusion Assemblycited by 75
  27. 2023SKED: Sketch-guided Text-based 3D Editingcited by 72
  28. 2023NeRFiller: Completing Scenes via Generative 3D Inpaintingcited by 62
  29. 2023Progressive3D: Progressively Local Editing for Text-to-3D Content Creation with Complex Semantic Promptscited by 51
  30. 2023Direct2.5: Diverse Text-to-3D Generation via Multi-view 2.5D Diffusioncited by 50
  31. 2024Generic 3D Diffusion Adapter Using Controlled Multi-View Editingcited by 46
  32. 2024Score Distillation via Reparametrized DDIMcited by 41
  33. 20233D Paintbrush: Local Stylization of 3D Shapes with Cascaded Score Distillationcited by 34
  34. 2024MVInpainter: Learning Multi-View Consistent Inpainting to Bridge 2D and 3D Editingcited by 30
  35. 2024MeshUp: Multi-Target Mesh Deformation via Blended Score Distillationcited by 17
  36. 2024MatAtlas: Text-driven Consistent Geometry Texturing and Material Assignmentcited by 14
  37. 2024MagicClay: Sculpting Meshes With Generative Neural Fieldscited by 13
  38. 2023ROAR: Robust Adaptive Reconstruction of Shapes Using Planar Projectionscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 26.

43 papers.

Sort:

By area:

  1. 2025VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Spacecited by 40
  2. 2025NANO3D: A Training-Free Approach for Efficient 3D Editing Without Maskscited by 29
  3. 2025EditP23: 3D Editing via Propagation of Image Prompts to Multi-Viewcited by 19
  4. 2025LL3M: Large Language 3D Modelerscited by 18
  5. 2025Towards Scalable and Consistent 3D Editingcited by 9
  6. 2025SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modelingcited by 9
  7. 2025AnchorFlow: Training-Free 3D Editing via Latent Anchor-Aligned Flowscited by 8
  8. 2025MeshPad: Interactive Sketch-Conditioned Artist-Reminiscent Mesh Generation and Editingcited by 6
  9. 2025Feedforward 3D Editing via Text-Steerable Image-to-3Dcited by 6
  10. 2025FreeInsert: Disentangled Text-Guided Object Insertion in 3D Gaussian Scene without Spatial Priorscited by 5
  11. 2025SplatPainter: Interactive Authoring of 3D Gaussians from 2D Edits via Test-Time Trainingcited by 5
  12. 2025Coupled Diffusion Sampling for Training-Free Multi-View Image Editingcited by 4
31 more
  1. 2025GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherencecited by 4
  2. 2026VecSet-Edit: Unleashing Pre-trained LRM for Mesh Editing from Single Imagecited by 4
  3. 2025CraftMesh: High-Fidelity Generative Mesh Manipulation via Poisson Seamless Fusioncited by 3
  4. 2025Native 3D Editing with Full Attentioncited by 3
  5. 2026ShapeUP: Scalable Image-Conditioned 3D Editingcited by 3
  6. 2026Vinedresser3D: Agentic Text-guided 3D Editingcited by 3
  7. 2026ArtisanGS: Interactive Tools for Gaussian Splat Selection with AI and Human in the Loopcited by 2
  8. 2026TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Maskingcited by 2
  9. 2026EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformerscited by 2
  10. 2026TanGO: Training-Free 3D Editing via Tangent-Space Guidance and Optimizationcited by 2
  11. 2026Axolotl3D: a Unified Framework for Faithful 3D Shape Completioncited by 2
  12. 2026Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapescited by 1
  13. 2026Velocity-Space 3D Asset Editingcited by 1
  14. 2026EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learningcited by 1
  15. 2026ViSculpt: Visual-Centric Agentic Geometry Editingcited by 1
  16. 2025ObjFiller3D: Scaling 3D Object Inpainting to Dense Multi-View Consistencycited by 0
  17. 2025Mesh RAG: Retrieval Augmentation for Autoregressive Mesh Generationcited by 0
  18. 2026Human Interaction-Aware 3D Reconstruction from a Single Imagecited by 0
  19. 2026MeshOn: Intersection-Free Mesh-to-Mesh Compositioncited by 0
  20. 2026MeshReGen: A Unified 3D Geometry Regeneration Frameworkcited by 0
  21. 2026DreamEdit3D: Personalization of Multi-View Diffusion Models for 3D Editingcited by 0
  22. 2026CompoSE: Compositional Synthesis and Editing of 3D Shapes via Part-Aware Controlcited by 0
  23. 2026GLUT: 3D Gaussian Lookup Table for Continuous Color Transformationcited by 0
  24. 2026Arbor: Explicit Geometric Conditioning for Controllable 3D Asset Generationcited by 0
  25. 2026CoIn: Comprehensive 2D-3D Inpainting with Gaussian Splatting Guidancecited by 0
  26. 2026Restore3D: Breathing Life into Broken Objects with Shape and Texture Restorationcited by 0
  27. 2026CNS-Edit++: Category-Agnostic 3D Editing with Coupled Neural Shape Representationcited by 0
  28. 2026Hunyuan3D-Buffalo 1.0: A Unified Multimodal Model for Scalable 3D Generation, Understanding, and Editingcited by 0
  29. 2026RADmesh: Remesh-Aware Mesh Deformationcited by 0
  30. 2026SemanticSlider3D: Training-Free Continuous Semantic Editing for 3D Objectscited by 0
  31. 2026Learning 3D Editing without Paired Supervision via Generative Prior Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Inpaint4DNeRF: Promptable Spatio-Temporal NeRF Inpainting with Generative Diffusion Models
  2. 2023Inpaint3D: 3D Scene Content Generation using 2D Inpainting Diffusion
  3. 2025Geometry-Aware Diffusion Models for Multiview Scene Inpainting
  4. 2024Generative Object Insertion in Gaussian Splatting with a Multi-View Diffusion Model
  5. 20263D-GIMP: When 3D Gaussian Inpainting Meets PatchMatch
  6. 2024MvDrag3D: Drag-based Creative 3D Editing via Multi-view Generation-Reconstruction Priors
  7. 2026WINGS: Reference-Free Gaussian Splatting Inpainting with 3D-Native Generative Priors
  8. 2024InFusion: Inpainting 3D Gaussians via Learning Depth Completion from Diffusion Prior