mlsanityData to 14 Oct 2026

Shape, Pose, and Appearance from a Single Image via Bootstrapped Radiance Field Inversion

arXiv 2211.11674 ↗First posted 21 Nov 2022cs.CV cs.AI cs.GRcited by 30

Not on the map. Why?

Abstract (arXiv)

Neural Radiance Fields (NeRF) coupled with GANs represent a promising direction in the area of 3D reconstruction from a single view, owing to their ability to efficiently model arbitrary topologies. Recent work in this area, however, has mostly focused on synthetic datasets where exact ground-truth poses are known, and has overlooked pose estimation, which is important for certain downstream applications such as augmented reality (AR) and robotics. We introduce a principled end-to-end reconstruction framework for natural images, where accurate ground-truth poses are not available. Our approach recovers an SDF-parameterized 3D shape, pose, and appearance from a single image of an object, without exploiting multiple views during training. More specifically, we leverage an unconditional 3D-aware generator, to which we apply a hybrid inversion scheme where a model produces a first guess of the solution which is then refined via optimization. Our framework can de-render an image in as few as 10 steps, enabling its use in practical scenarios. We demonstrate state-of-the-art results on a variety of real and synthetic benchmarks.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 63 of its 72 references that are in the map. First posted on arXiv 21 Nov 2022.

Builds on

63 of 72 references.

References · 63

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2017Mask R-CNNcited by 7,277
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  6. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  8. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  9. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  10. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
  11. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  12. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
38 more
  1. 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
  2. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  3. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  4. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  5. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  6. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  7. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
  8. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 689
  9. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  10. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  11. 2021StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesiscited by 477
  12. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
  13. 2022GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Imagescited by 446
  14. 2019PointRend: Image Segmentation as Renderingcited by 440
  15. 2017Neural 3D Mesh Renderercited by 437
  16. 2021Pivotal Tuning for Latent-based Editing of Real Imagescited by 427
  17. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  18. 2021GAN Inversion: A Surveycited by 367
  19. 2016Adversarial Feature Learningcited by 350
  20. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  21. 2022Novel View Synthesis with Diffusion Modelscited by 300
  22. 2021StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generationcited by 289
  23. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  24. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  25. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  26. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  27. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  28. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  29. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  30. 2020Shape and Viewpoint without Keypointscited by 92
  31. 2017Hierarchical Surface Prediction for 3D Object Reconstructioncited by 89
  32. 2022GIRAFFE HD: A High-Resolution 3D-aware Generative Modelcited by 86
  33. 2021LOLNeRF: Learn from One Lookcited by 84
  34. 2022Pix2NeRF: Unsupervised Conditional π-GAN for Single Image to Neural Radiance Fields Translationcited by 73
  35. 2021Unsupervised 3D Shape Completion through GAN Inversioncited by 70
  36. 2020Leveraging 2D Data to Learn Textured 3D Mesh Generationcited by 69
  37. 2020SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Imagescited by 68
  38. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 66

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 13; 2025: 4; 2026: 3.

30 papers.

Sort:

By area:

  1. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
  2. 2025MUSt3R: Multi-view Network for Stereo 3D Reconstructioncited by 141
  3. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 118
  4. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 68
  5. 2024AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstructioncited by 52
  6. 2023MonoNPHM: Dynamic Head Reconstruction from Monocular Videoscited by 35
  7. 2024Neural Fields in Robotics: A Surveycited by 21
  8. 2024NeRFs in Robotics: A Surveycited by 16
  9. 2023SOCS: Semantically-aware Object Coordinate Space for Category-Level 6D Object Pose Estimation under Large Shape Variationscited by 13
  10. 2023Slice3D: Multi-Slice, Occlusion-Revealing, Single View 3D Reconstructioncited by 13
  11. 2025Orientation Matters: Making 3D Generative Models Orientation-Alignedcited by 13
  12. 2023MOHO: Learning Single-view Hand-held Object Reconstruction with Multi-view Occlusion-Aware Supervisioncited by 12
18 more
  1. 2023MFOS: Model-Free & One-Shot Object Pose Estimationcited by 10
  2. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  3. 2024VQA-Diff: Exploiting VQA and Diffusion for Zero-Shot Image-to-3D Vehicle Asset Generation in Autonomous Drivingcited by 9
  4. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  5. 2023WildFusion: Learning 3D-Aware Latent Diffusion Models in View Spacecited by 8
  6. 2025Is Single-View Mesh Reconstruction Ready for Robotics?cited by 6
  7. 2023Learning Effective NeRFs and SDFs Representations with 3D Generative Adversarial Networks for 3D Object Generationcited by 3
  8. 2024SUP-NeRF: A Streamlined Unification of Pose Estimation and NeRF for Monocular 3D Object Reconstructioncited by 3
  9. 2024G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Imagescited by 3
  10. 2025MADrive: Memory-Augmented Driving Scene Modelingcited by 3
  11. 2026S-MUSt3R: Sliding Multi-view 3D Reconstructioncited by 3
  12. 2023360^circ Reconstruction From a Single Image Using Space Carved Outpaintingcited by 2
  13. 2024MomentsNeRF: Leveraging Orthogonal Moments for Few-Shot Neural Renderingcited by 2
  14. 2024Predicting the Encoding Error of SIRENscited by 2
  15. 2023Multi-View Unsupervised Image Generation with Cross Attention Guidancecited by 1
  16. 20243DArticCyclists: Generating Synthetic Articulated 8D Pose-Controllable Cyclist Data for Computer Vision Applicationscited by 1
  17. 2026Learning Positive-Incentive Point Sampling in Neural Implicit Fields for Object Pose Estimationcited by 0
  18. 2026SEMAGIC: Learning Semantically Consistent Deformable 3D Representations from In-the-Wild Imagescited by 0