mlsanityData to 14 Oct 2026

ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context

arXiv 2310.09965 ↗First posted 15 Oct 2023cs.CV cs.GRcited by 17

Not on the map. Why?

Abstract (arXiv)

Neural Radiance Fields (NeRFs) have recently emerged as a popular option for photo-realistic object capture due to their ability to faithfully capture high-fidelity volumetric content even from handheld video input. Although much research has been devoted to efficient optimization leading to real-time training and rendering, options for interactive editing NeRFs remain limited. We present a very simple but effective neural network architecture that is fast and efficient while maintaining a low memory footprint. This architecture can be incrementally guided through user-friendly image-based edits. Our representation allows straightforward object selection via semantic feature distillation at the training stage. More importantly, we propose a local 3D-aware image context to facilitate view-consistent image editing that can then be distilled into fine-tuned NeRFs, via geometric and appearance adjustments. We evaluate our setup on a variety of examples to demonstrate appearance and geometric edits and report 10-30x speedup over concurrent work focusing on text-guided NeRF editing. Video results can be seen on our project webpage at https://proteusnerf.github.io.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 48 of its 59 references that are in the map. First posted on arXiv 15 Oct 2023.

Builds on

48 of 59 references.

References · 48

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
36 more
  1. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
  2. 2021Vision Transformers for Dense Predictioncited by 1,803
  3. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,613
  4. 2022TensoRF: Tensorial Radiance Fieldscited by 1,280
  5. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  6. 2022Diffusion Models in Vision: A Surveycited by 1,015
  7. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 964
  8. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 905
  9. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 746
  10. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  11. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 600
  12. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  13. 2021Zero-Shot Text-Guided Object Generation with Dream Fieldscited by 513
  14. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 511
  15. 2022Latent-NeRF for Shape-Guided Generation of 3D Shapes and Texturescited by 506
  16. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  17. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 468
  18. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 371
  19. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 370
  20. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 219
  21. 2022Neural Feature Fusion Fields: 3D Distillation of Self-Supervised 2D Image Representationscited by 196
  22. 2023DreamEditor: Text-Driven 3D Scene Editing with Neural Fieldscited by 155
  23. 2022StylizedNeRF: Consistent 3D Scene Stylization as Stylized NeRF via 2D-3D Mutual Learningcited by 140
  24. 2023TextMesh: Generation of Realistic 3D Meshes From Text Promptscited by 135
  25. 2022NeRF-Art: Text-Driven Neural Radiance Fields Stylizationcited by 126
  26. 2021Stylizing 3D Scene via Implicit Representation and HyperNetworkcited by 103
  27. 2022ReLU Fields: The Little Non-linearity That Couldcited by 87
  28. 2022Deforming Radiance Fields with Cagescited by 78
  29. 2021Learning to Stylize Novel Viewscited by 76
  30. 2023Nerfbusters: Removing Ghostly Artifacts from Casually Captured NeRFscited by 76
  31. 2022Neural Volumetric Object Selectioncited by 61
  32. 2022PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fieldscited by 45
  33. 2020A Survey on Trajectory Data Management, Analytics, and Learningcited by 41
  34. 2023Blending-NeRF: Text-Driven Localized Editing in Neural Radiance Fieldscited by 28
  35. 2023RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenescited by 25
  36. 2023Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fieldscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 6; 2026: 4.

17 papers.

Sort:

By area:

  1. 2024DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editingcited by 94
  2. 2023SHAP-EDITOR: Instruction-guided Latent 3D Editing in Secondscited by 37
  3. 2025Fast Multi-view Consistent 3D Editing with Video Priorscited by 12
  4. 2024NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFscited by 11
  5. 2024ICE-G: Image Conditional Editing of 3D Gaussian Splatscited by 11
  6. 2024NeRF-Insert: 3D Local Editing with Multimodal Control Signalscited by 4
  7. 2025SVG-Head: Hybrid Surface-Volumetric Gaussians for High-Fidelity Head Reconstruction and Real-Time Editingcited by 4
  8. 2026Edit in 2D, Verify in 3D: Reinforcement Learning for Multi-view Consistent Scene Editingcited by 4
  9. 2024IReNe: Instant Recoloring of Neural Radiance Fieldscited by 3
  10. 2025DisCo3D: Distilling Multi-View Consistency for 3D Scene Editingcited by 3
  11. 20253D-Consistent Multi-View Editing by Correspondence Guidancecited by 2
  12. 2024InfoGaussian: Structure-Aware Dynamic Gaussians through Lightweight Information Shapingcited by 1
5 more
  1. 2025InstructMix2Mix: Consistent Sparse-View Editing Through Multi-View Model Personalizationcited by 1
  2. 2026VIRGi: View-dependent Instant Recoloring of 3D Gaussians Splatscited by 1
  3. 2025InStyle: Instant Appearance Stylization of 3D Shapescited by 0
  4. 2026Catalyst4D: High-Fidelity 3D-to-4D Scene Editing via Dynamic Propagationcited by 0
  5. 2026GeM-NR: Geometry-Aware Multi-View Editing for Nonrigid Scene Changescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024DATENeRF: Depth-Aware Text-based Editing of NeRFs
  2. 2024SealD-NeRF: Interactive Pixel-Level Editing for Dynamic Scenes by Neural Radiance Fields
  3. 2023RePaint-NeRF: NeRF Editting via Semantic Masks and Diffusion Models
  4. 2024ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields
  5. 20234D-Editor: Interactive Object-level Editing in Dynamic Neural Radiance Fields via Semantic Distillation
  6. 2023DynVideo-E: Harnessing Dynamic NeRF for Large-Scale Motion- and View-Change Human-Centric Video Editing
  7. 2023ED-NeRF: Efficient Text-Guided Editing of 3D Scene with Latent Space NeRF
  8. 2023Inpaint4DNeRF: Promptable Spatio-Temporal NeRF Inpainting with Generative Diffusion Models