Beyond Voxel 3D Editing: Learning from 3D Masks and Self-Constructed Data
Not on the map. Why?
Abstract (arXiv)
3D editing refers to the ability to apply local or global modifications to 3D assets. Effective 3D editing requires maintaining semantic consistency by performing localized changes according to prompts, while also preserving local invariance so that unchanged regions remain consistent with the original. However, existing approaches have significant limitations: multi-view editing methods incur losses when projecting back to 3D, while voxel-based editing is constrained in both the regions that can be modified and the scale of modifications. Moreover, the lack of sufficiently large editing datasets for training and evaluation remains a challenge. To address these challenges, we propose a Beyond Voxel 3D Editing (BVE) framework with a self-constructed large-scale dataset specifically tailored for 3D editing. Building upon this dataset, our model enhances a foundational image-to-3D generative architecture with lightweight, trainable modules, enabling efficient injection of textual semantics without the need for expensive full-model retraining. Furthermore, we introduce an annotation-free 3D masking strategy to preserve local invariance, maintaining the integrity of unchanged regions during editing. Extensive experiments demonstrate that BVE achieves superior performance in generating high-quality, text-aligned 3D assets, while faithfully retaining the visual characteristics of the original input.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 66 of its 83 references that are in the map. First posted on arXiv 15 Apr 2026.
Builds on
66 of 83 references.
References · 66
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
- 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
- 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
38 more
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusioncited by 689
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Imagecited by 645
- 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 560
- 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
- 2024InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Modelscited by 433
- 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
- 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
- 2025Hunyuan3D 2.0: Scaling Diffusion Models for High Resolution Textured 3D Assets Generationcited by 356
- 2023One-2-3-45++: Fast Single Image to 3D Objects with Consistent Multi-View Generation and 3D Diffusioncited by 283
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 268
- 2023Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representationcited by 228
- 2024CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Modelcited by 215
- 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Modelscited by 213
- 2024Direct3D: Scalable Image-to-3D Generation via 3D Latent Diffusion Transformercited by 210
- 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusioncited by 202
- 2024MeshAnything: Artist-Created Mesh Generation with Autoregressive Transformerscited by 144
- 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 135
- 2025Hunyuan3D 2.5: Towards High-Fidelity 3D Assets Generation with Ultimate Detailscited by 125
- 2024CraftsMan3D: High-fidelity Mesh Generation with 3D Native Generation and Interactive Geometry Refinercited by 123
- 2023Vox-E: Text-guided Voxel Editing of 3D Objectscited by 115
- 2024MV-Adapter: Multi-view Consistent Image Generation Made Easycited by 112
- 2025Direct3D-S2: Gigascale 3D Generation Made Easy with Spatial Sparse Attentioncited by 110
- 20243DTopia-XL: Scaling High-quality 3D Asset Generation via Primitive Diffusioncited by 101
- 2024DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editingcited by 94
- 2024Meshtron: High-Fidelity, Artist-Like 3D Mesh Generation at Scalecited by 89
- 2025Step1X-3D: Towards High-Fidelity and Controllable Generation of Textured 3D Assetscited by 87
- 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 84
- 2024BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolationcited by 83
- 2023SKED: Sketch-guided Text-based 3D Editingcited by 73
- 2023EpiDiff: Enhancing Multi-View Synthesis via Localized Epipolar-Constrained Diffusioncited by 72
- 2024PartGen: Part-level 3D Generation and Reconstruction with Multi-View Diffusion Modelscited by 62
- 2024SeedEdit: Align Image Re-Generation to Image Editingcited by 61
- 2024LT3SD: Latent Trees for 3D Scene Diffusioncited by 55
- 2024Generic 3D Diffusion Adapter Using Controlled Multi-View Editingcited by 47
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.