# Instant3dit: Multiview Inpainting for Fast Editing of 3D Objects

arXiv: https://arxiv.org/abs/2412.00518 · HTML: https://mlsanity.com/paper/2412.00518/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 58 references that are in the map.
- First posted on arXiv 30 Nov 2024.

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Objaverse: A Universe of Annotated 3D Objects](https://mlsanity.com/paper/2212.08051/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [MVDream: Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2308.16512/) (2023)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024)
- [LRM: Large Reconstruction Model for Single Image to 3D](https://mlsanity.com/paper/2311.04400/) (2023)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [Wonder3D: Single Image to 3D using Cross-Domain Diffusion](https://mlsanity.com/paper/2310.15008/) (2023)
- [Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation](https://mlsanity.com/paper/2303.13873/) (2023)
- [SyncDreamer: Generating Multiview-consistent Images from a Single-view Image](https://mlsanity.com/paper/2309.03453/) (2023)
- [Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model](https://mlsanity.com/paper/2310.15110/) (2023)
- [Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures](https://mlsanity.com/paper/2211.07600/) (2022)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models](https://mlsanity.com/paper/2404.07191/) (2024)
- [Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model](https://mlsanity.com/paper/2311.06214/) (2023)
- [GS-LRM: Large Reconstruction Model for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.19702/) (2024)
- [Text2Mesh: Text-Driven Neural Stylization for Meshes](https://mlsanity.com/paper/2112.03221/) (2021)
- [TEXTure: Text-Guided Texturing of 3D Shapes](https://mlsanity.com/paper/2302.01721/) (2023)
- [GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image](https://mlsanity.com/paper/2403.12013/) (2024)
- [CLIP-Mesh: Generating textured meshes from text using pretrained image-text models](https://mlsanity.com/paper/2203.13333/) (2022)
- [ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation](https://mlsanity.com/paper/2312.02201/) (2023)
- [GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models](https://mlsanity.com/paper/2310.08529/) (2023)

## Built on by (most cited first, 30 of 44)

- [VoxHammer: Training-Free Precise and Coherent 3D Editing in Native 3D Space](https://mlsanity.com/paper/2508.19247/) (2025)
- [NANO3D: A Training-Free Approach for Efficient 3D Editing Without Masks](https://mlsanity.com/paper/2510.15019/) (2025)
- [EditP23: 3D Editing via Propagation of Image Prompts to Multi-View](https://mlsanity.com/paper/2506.20652/) (2025)
- [LL3M: Large Language 3D Modelers](https://mlsanity.com/paper/2508.08228/) (2025)
- [Towards Scalable and Consistent 3D Editing](https://mlsanity.com/paper/2510.02994/) (2025)
- [AnchorFlow: Training-Free 3D Editing via Latent Anchor-Aligned Flows](https://mlsanity.com/paper/2511.22357/) (2025)
- [SpaceControl: Introducing Test-Time Spatial Control to 3D Generative Modeling](https://mlsanity.com/paper/2512.05343/) (2025)
- [MeshPad: Interactive Sketch-Conditioned Artist-Reminiscent Mesh Generation and Editing](https://mlsanity.com/paper/2503.01425/) (2025)
- [SplatPainter: Interactive Authoring of 3D Gaussians from 2D Edits via Test-Time Training](https://mlsanity.com/paper/2512.05354/) (2025)
- [Feedforward 3D Editing via Text-Steerable Image-to-3D](https://mlsanity.com/paper/2512.13678/) (2025)
- [GENA3D: Generative Amodal 3D Modeling by Bridging 2D Priors and 3D Coherence](https://mlsanity.com/paper/2511.21945/) (2025)
- [FreeInsert: Disentangled Text-Guided Object Insertion in 3D Gaussian Scene without Spatial Priors](https://mlsanity.com/paper/2505.01322/) (2025)
- [Coupled Diffusion Sampling for Training-Free Multi-View Image Editing](https://mlsanity.com/paper/2510.14981/) (2025)
- [Native 3D Editing with Full Attention](https://mlsanity.com/paper/2511.17501/) (2025)
- [VecSet-Edit: Unleashing Pre-trained LRM for Mesh Editing from Single Image](https://mlsanity.com/paper/2602.04349/) (2026)
- [ShapeUP: Scalable Image-Conditioned 3D Editing](https://mlsanity.com/paper/2602.05676/) (2026)
- [CraftMesh: High-Fidelity Generative Mesh Manipulation via Poisson Seamless Fusion](https://mlsanity.com/paper/2509.13688/) (2025)
- [Vinedresser3D: Agentic Text-guided 3D Editing](https://mlsanity.com/paper/2602.19542/) (2026)
- [EVA01: Unified Native 3D Understanding and Generation via Mixture-of-Transformers](https://mlsanity.com/paper/2605.16745/) (2026)
- [Axolotl3D: a Unified Framework for Faithful 3D Shape Completion](https://mlsanity.com/paper/2607.20660/) (2026)
- [ArtisanGS: Interactive Tools for Gaussian Splat Selection with AI and Human in the Loop](https://mlsanity.com/paper/2602.10173/) (2026)
- [TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking](https://mlsanity.com/paper/2604.01207/) (2026)
- [Velocity-Space 3D Asset Editing](https://mlsanity.com/paper/2605.07385/) (2026)
- [TanGO: Training-Free 3D Editing via Tangent-Space Guidance and Optimization](https://mlsanity.com/paper/2607.14927/) (2026)
- [Mesh RAG: Retrieval Augmentation for Autoregressive Mesh Generation](https://mlsanity.com/paper/2511.16807/) (2025)
- [Beyond Prompts: Unconditional 3D Inversion for Out-of-Distribution Shapes](https://mlsanity.com/paper/2604.14914/) (2026)
- [MeshReGen: A Unified 3D Geometry Regeneration Framework](https://mlsanity.com/paper/2604.28134/) (2026)
- [EditVerse3D: High-Quality 3D Object Editing with Region-Aware Learning](https://mlsanity.com/paper/2607.07187/) (2026)
- [ViSculpt: Visual-Centric Agentic Geometry Editing](https://mlsanity.com/paper/2608.24169/) (2026)
- [ObjFiller3D: Scaling 3D Object Inpainting to Dense Multi-View Consistency](https://mlsanity.com/paper/2508.18271/) (2025)

## Abstract (arXiv)

We propose a generative technique to edit 3D shapes, represented as meshes, NeRFs, or Gaussian Splats, in approximately 3 seconds, without the need for running an SDS type of optimization. Our key insight is to cast 3D editing as a multiview image inpainting problem, as this representation is generic and can be mapped back to any 3D representation using the bank of available Large Reconstruction Models. We explore different fine-tuning strategies to obtain both multiview generation and inpainting capabilities within the same diffusion model. In particular, the design of the inpainting mask is an important factor of training an inpainting model, and we propose several masking strategies to mimic the types of edits a user would perform on a 3D shape. Our approach takes 3D generative editing from hours to seconds and produces higher-quality results compared to previous works.