mlsanityData to 7 Oct 2026

NeRFDeformer: NeRF Transformation from a Single View via 3D Scene Flows

arXiv 2406.10543 ↗First posted 15 Jun 2024cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

We present a method for automatically modifying a NeRF representation based on a single observation of a non-rigid transformed version of the original scene. Our method defines the transformation as a 3D flow, specifically as a weighted linear blending of rigid transformations of 3D anchor points that are defined on the surface of the scene. In order to identify anchor points, we introduce a novel correspondence algorithm that first matches RGB-based pairs, then leverages multi-view information and 3D reprojection to robustly filter false positives in two steps. We also introduce a new dataset for exploring the problem of modifying a NeRF scene through a single observation. Our dataset ( https://github.com/nerfdeformer/nerfdeformer ) contains 113 synthetic scenes leveraging 47 3D assets. We show that our proposed method outperforms NeRF editing methods as well as diffusion-based methods, and we also explore different methods for filtering correspondences.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 50 of its 61 references that are in the map. First posted on arXiv 15 Jun 2024.

Builds on

50 of 61 references.

References · 50

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,522
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  8. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  9. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  10. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  11. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  12. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
38 more
  1. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  2. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  3. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  4. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  5. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  6. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  7. 2023Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priorscited by 395
  8. 2021CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fieldscited by 369
  9. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  10. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  11. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  12. 2022FlowFormer: A Transformer Architecture for Optical Flowcited by 273
  13. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  14. 2021Editing Conditional Radiance Fieldscited by 222
  15. 2022NeRF-Editing: Geometry Editing of Neural Radiance Fieldscited by 216
  16. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  17. 2022ASpanFormer: Detector-Free Image Matching with Adaptive Span Transformercited by 170
  18. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  19. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
  20. 2023BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objectscited by 144
  21. 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 141
  22. 2023Distilled Feature Fields Enable Few-Shot Language-Guided Manipulationcited by 130
  23. 2023SINE: Semantic-driven Image-based NeRF Editing with Prior-guided Editing Fieldcited by 93
  24. 2020Object-Centric Neural Scene Renderingcited by 90
  25. 2021Decomposing 3D Scenes into Objects via Unsupervised Volume Segmentationcited by 88
  26. 2022NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fieldscited by 86
  27. 2016VolumeDeform: Real-time Volumetric Non-rigid Reconstructioncited by 75
  28. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  29. 2021CoNeRF: Controllable Neural Radiance Fieldscited by 65
  30. 2022Parallel Inversion of Neural Radiance Fields for Robust Pose Estimationcited by 55
  31. 2022IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesiscited by 50
  32. 2020Neural Non-Rigid Trackingcited by 49
  33. 2022PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fieldscited by 43
  34. 2023I2-SDF: Intrinsic Indoor Scene Reconstruction and Editing via Raytracing in Neural SDFscited by 42
  35. 2022LaTeRF: Label and Text Driven Object Radiance Fieldscited by 38
  36. 2023Pseudo-Generalized Dynamic View Synthesis from a Videocited by 27
  37. 2023Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fieldscited by 18
  38. 2022RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Controlcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Editing Implicit and Explicit Representations of Radiance Fields: A Surveycited by 1