mlsanityData to 13 Oct 2026

ShaRF: Shape-conditioned Radiance Fields from a Single View

arXiv 2102.08860 ↗First posted 17 Feb 2021cs.CV cs.GRcited by 92

Not on the map. Why?

Abstract (arXiv)

We present a method for estimating neural scenes representations of objects given only a single image. The core of our method is the estimation of a geometric scaffold for the object and its use as a guide for the reconstruction of the underlying radiance field. Our formulation is based on a generative process that first maps a latent code to a voxelized shape, and then renders it to an image, with the object appearance being controlled by a second latent code. During inference, we optimize both the latent codes and the networks to fit a test image of a new object. The explicit disentanglement of shape and appearance allows our model to be fine-tuned given a single image. We can then render new views in a geometrically consistent manner and they represent faithfully the input object. Additionally, our method is able to generalize to images outside of the training domain (more realistic renderings and even real photographs). Finally, the inferred geometric scaffold is itself an accurate estimate of the object's 3D shape. We demonstrate in several experiments the effectiveness of our approach in both synthetic and real images.

Key facts. Cited by 92 papers in the map (data through 13 Oct 2026). Builds on 37 of its 59 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

37 of 59 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,000
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,231
  5. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,686
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,039
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,581
  8. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,269
  9. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
  10. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 976
  11. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  12. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
25 more
  1. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  2. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 650
  3. 2020GANSpace: Discovering Interpretable GAN Controlscited by 612
  4. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  5. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  6. 2016Generative Visual Manipulation on the Natural Image Manifoldcited by 348
  7. 2020State of the Art on Neural Renderingcited by 277
  8. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 269
  9. 2020Free View Synthesiscited by 231
  10. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  11. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  12. 2020MetaSDF: Meta-learning Signed Distance Functionscited by 214
  13. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  14. 2019BSP-Net: Generating Compact Meshes via Binary Space Partitioningcited by 206
  15. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  16. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  17. 2020Stable View Synthesiscited by 165
  18. 2017Optimizing the Latent Space of Generative Networkscited by 157
  19. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  20. 2018Unsupervised Learning of Shape and Pose with Differentiable Point Cloudscited by 89
  21. 2017Neural Face Editing with Intrinsic Image Disentanglingcited by 70
  22. 2015Multi-view 3D Models from Single Images with a Convolutional Networkcited by 57
  23. 2020Equivariant Neural Renderingcited by 48
  24. 2020Neural Volume Rendering: NeRF And Beyondcited by 44
  25. 2019Front2Back: Single View 3D Shape Reconstruction via Front to Back Predictioncited by 17

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 17; 2022: 20; 2023: 29; 2024: 19; 2025: 6; 2026: 1.

92 papers.

Sort:

By area:

  1. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  2. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  3. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  4. 2021Neural Fields in Visual Computing and Beyondcited by 539
  5. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 521
  6. 2021Advances in Neural Renderingcited by 365
  7. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 362
  8. 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
  9. 2021Urban Radiance Fieldscited by 263
  10. 20243D Gaussian Splatting as Markov Chain Monte Carlocited by 214
  11. 2021Neural Rays for Occlusion-aware Image-based Renderingcited by 192
  12. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 180
38 more
  1. 2023GauHuman: Articulated Gaussian Splatting from Monocular Human Videoscited by 168
  2. 2021Unconstrained Scene Generation with Locally Conditioned Radiance Fieldscited by 130
  3. 2021NeSF: Neural Semantic Fields for Generalizable Semantic Segmentation of 3D Scenescited by 119
  4. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  5. 2022Vision Transformer for NeRF-Based View Synthesis from a Single Input Imagecited by 104
  6. 2021Unsupervised Discovery of Object Radiance Fieldscited by 101
  7. 2023GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fieldscited by 99
  8. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  9. 2023PC2: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstructioncited by 84
  10. 2021Demystifying the Draft EU Artificial Intelligence Actcited by 82
  11. 2021Learning to Stylize Novel Viewscited by 76
  12. 2021DIVeR: Real-time and Accurate Neural Radiance Fields with Deterministic Integration for Volume Renderingcited by 60
  13. 2022AutoRF: Learning 3D Object Radiance Fields from Single View Observationscited by 59
  14. 2022CLA-NeRF: Category-Level Articulated Neural Radiance Fieldcited by 55
  15. 2024MeshFormer: High-Quality Mesh Generation with 3D-Guided Reconstruction Modelcited by 55
  16. 2022Surface Reconstruction from Point Clouds by Learning Predictive Context Priorscited by 54
  17. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
  18. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  19. 2023FeatureNeRF: Learning Generalizable NeRFs by Distilling Foundation Modelscited by 47
  20. 2022Aug-NeRF: Training Stronger Neural Radiance Fields with Triple-Level Physically-Grounded Augmentationscited by 46
  21. 2023ReLight My NeRF: A Dataset for Novel View Synthesis and Relighting of Real World Objectscited by 42
  22. 2023MixNeRF: Modeling a Ray with Mixture Density for Novel View Synthesis from Sparse Inputscited by 40
  23. 2022Fast Learning Radiance Fields by Shooting Much Fewer Rayscited by 38
  24. 2022Transframer: Arbitrary Frame Prediction with Generative Modelscited by 37
  25. 2024SpaRP: Fast 3D Object Reconstruction and Pose Estimation from Sparse Viewscited by 35
  26. 2023OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fieldscited by 34
  27. 2022S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpointcited by 30
  28. 2022Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categoriescited by 29
  29. 2023FlipNeRF: Flipped Reflection Rays for Few-shot Novel View Synthesiscited by 29
  30. 2021Fast and Explicit Neural View Synthesiscited by 28
  31. 2023Unsupervised Inference of Signed Distance Functions from Single Sparse Point Clouds without Learning Priorscited by 28
  32. 2023Learning Signed Distance Functions from Noisy 3D Point Clouds via Noise to Noise Mappingcited by 27
  33. 2023Pseudo-Generalized Dynamic View Synthesis from a Videocited by 27
  34. 2023Point'n Move: Interactive Scene Object Manipulation on Gaussian Splatting Radiance Fieldscited by 27
  35. 2021Neural Radiance Fields Approach to Deep Multi-View Photometric Stereocited by 25
  36. 2025Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Surveycited by 25
  37. 2023ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretrainingcited by 24
  38. 2024Real3D: Scaling Up Large Reconstruction Models with Real-World Imagescited by 24

50 most cited of 92 All 92 (JSON)