mlsanityData to 7 Oct 2026

AutoRF: Learning 3D Object Radiance Fields from Single View Observations

arXiv 2204.03593 ↗First posted 7 Apr 2022cs.CVcited by 59

Not on the map. Why?

Abstract (arXiv)

We introduce AutoRF - a new approach for learning neural 3D object representations where each object in the training set is observed by only a single view. This setting is in stark contrast to the majority of existing works that leverage multiple views of the same object, employ explicit priors during training, or require pixel-perfect annotations. To address this challenging setting, we propose to learn a normalized, object-centric representation whose embedding describes and disentangles shape, appearance, and pose. Each encoding provides well-generalizable, compact information about the object of interest, which is decoded in a single-shot into a new target view, thus enabling novel view synthesis. We further improve the reconstruction quality by optimizing shape and appearance codes at test time by fitting the representation tightly to the input image. In a series of experiments, we show that our method generalizes well to unseen objects, even across different datasets of challenging real-world street scenes such as nuScenes, KITTI, and Mapillary Metropolis.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). Builds on 32 of its 45 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

32 of 45 references.

References · 32

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2017Mask R-CNNcited by 7,259
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  7. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  8. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  9. 2018Panoptic Segmentationcited by 796
  10. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  11. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  12. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
20 more
  1. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  2. 2021Advances in Neural Renderingcited by 365
  3. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  4. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  5. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  6. 2019Disentangling Monocular 3D Object Detectioncited by 215
  7. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  8. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  9. 2017Optimizing the Latent Space of Generative Networkscited by 157
  10. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  11. 2019Autolabeling 3D Objects with Differentiable Rendering of SDF Shape Priorscited by 74
  12. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
  13. 2020Leveraging 2D Data to Learn Textured 3D Mesh Generationcited by 69
  14. 2021Unsupervised Learning of 3D Object Categories from Videos in the Wildcited by 67
  15. 2021GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detectioncited by 66
  16. 2021FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modellingcited by 47
  17. 2020Implicit Feature Networks for Texture Completion from Partial 3D Datacited by 31
  18. 2020SAMP: Shape and Motion Priors for 4D Vehicle Reconstructioncited by 19
  19. 2020Seeing Behind Objects for 3D Multi-Object Tracking in RGB-D Sequencescited by 13
  20. 2020Improving Panoptic Segmentation at All Scalescited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 29; 2024: 14; 2025: 2; 2026: 3.

59 papers.

Sort:

By area:

  1. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  2. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  3. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  4. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  5. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  6. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  7. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  8. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  9. 2022DM-NeRF: 3D Scene Geometry Decomposition and Manipulation from 2D Imagescited by 69
  10. 2024MultiDiff: Consistent Novel View Synthesis from a Single Imagecited by 67
  11. 2023Behind the Scenes: Density Fields for Single View Reconstructioncited by 63
  12. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
38 more
  1. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  2. 2023Points-to-3D: Bridging the Gap between Sparse Points and Shape-Controllable Text-to-3D Generationcited by 39
  3. 2024Img2CAD: Conditioned 3D CAD Model Generation from Single Image with Structured Visual Geometrycited by 35
  4. 2022Common Pets in 3D: Dynamic New-View Synthesis of Real-Life Deformable Categoriescited by 29
  5. 2024G3R: Gradient Guided Generalizable Reconstructioncited by 29
  6. 2023BeyondPixels: A Comprehensive Review of the Evolution of Neural Radiance Fieldscited by 27
  7. 2023UrbanGIRAFFE: Representing Urban Scenes as Compositional Generative Neural Feature Fieldscited by 26
  8. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  9. 2023GINA-3D: Learning to Generate Implicit Neural Assets in the Wildcited by 25
  10. 2023Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Modelscited by 22
  11. 2024TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshescited by 20
  12. 2023NViST: In the Wild New View Synthesis from a Single Image with Transformerscited by 19
  13. 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
  14. 2022A Survey on Deep Generative 3D-aware Image Synthesiscited by 16
  15. 2023HyP-NeRF: Learning Improved NeRF Priors using a HyperNetworkcited by 14
  16. 2023AutoRecon: Automated 3D Object Discovery and Reconstructioncited by 13
  17. 2023NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localizationcited by 13
  18. 2023Iterative Superquadric Recomposition of 3D Objects from Multiple Viewscited by 13
  19. 2023Towards Viewpoint Robustness in Bird's Eye View Segmentationcited by 13
  20. 20233D Data Augmentation for Driving Scenes on Cameracited by 12
  21. 2024Unsupervised Discovery of Object-Centric Neural Fieldscited by 12
  22. 2023Semantic-aware Occlusion Filtering Neural Radiance Fields in the Wildcited by 9
  23. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  24. 2024Generative Artificial Intelligence Meets Synthetic Aperture Radar: A Surveycited by 9
  25. 2024UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulationcited by 7
  26. 2022ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapescited by 6
  27. 2022One-Shot Neural Fields for 3D Object Understandingcited by 5
  28. 2026GenAssets: Generating in-the-wild 3D Assets in Latent Spacecited by 4
  29. 2023Car-Studio: Learning Car Radiance Fields from Single-View and Endless In-the-wild Imagescited by 3
  30. 2023MetaDreamer: Efficient Text-to-3D Creation With Disentangling Geometry and Texturecited by 3
  31. 2023ProSGNeRF: Progressive Dynamic Neural Scene Graph with Frequency Modulated Foundation Model in Urban Scenescited by 3
  32. 2024SUP-NeRF: A Streamlined Unification of Pose Estimation and NeRF for Monocular 3D Object Reconstructioncited by 3
  33. 2024Dynamic EventNeRF: Reconstructing General Dynamic Scenes from Multi-view RGB and Event Streamscited by 3
  34. 2022FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fieldscited by 2
  35. 2023CarPatch: A Synthetic Benchmark for Radiance Field Evaluation on Vehicle Componentscited by 2
  36. 2023Partial-View Object View Synthesis via Filtered Inversioncited by 1
  37. 2023Parcel3D: Shape Reconstruction from Single RGB Images for Applications in Transportation Logisticscited by 1
  38. 2024Category-level Neural Field for Reconstruction of Partially Observed Objects in Indoor Environmentcited by 1

50 most cited of 59 All 59 (JSON)