mlsanityData to 7 Oct 2026

Behind the Scenes: Density Fields for Single View Reconstruction

arXiv 2301.07668 ↗First posted 18 Jan 2023cs.CVcited by 63

Not on the map. Why?

Abstract (arXiv)

Inferring a meaningful geometric scene representation from a single image is a fundamental problem in computer vision. Approaches based on traditional depth map prediction can only reason about areas that are visible in the image. Currently, neural radiance fields (NeRFs) can capture true 3D including color, but are too complex to be generated from a single image. As an alternative, we propose to predict implicit density fields. A density field maps every location in the frustum of the input image to volumetric density. By directly sampling color from the available views instead of storing color in the density field, our scene representation becomes significantly less complex compared to NeRFs, and a neural network can predict it in a single forward pass. The prediction network is trained through self-supervision from only video data. Our formulation allows volume rendering to perform both depth prediction and novel view synthesis. Through experiments, we show that our method is able to predict meaningful geometry for regions that are occluded in the input image. Additionally, we demonstrate the potential of our approach on three datasets for depth prediction and novel-view synthesis.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 51 of its 63 references that are in the map. First posted on arXiv 18 Jan 2023.

Builds on

51 of 63 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  6. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  7. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2020Neural Sparse Voxel Fieldscited by 959
  11. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 862
  12. 2016Unsupervised Monocular Depth Estimation with Left-Right Consistencycited by 819
38 more
  1. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  2. 2020Convolutional Occupancy Networkscited by 760
  3. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  4. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  5. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  6. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  7. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  8. 20193D Packing for Self-Supervised Monocular Depth Estimationcited by 437
  9. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  10. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  11. 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 301
  12. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  13. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  14. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  15. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  16. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  17. 2021InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Renderingcited by 180
  18. 2016View Synthesis by Appearance Flowcited by 172
  19. 2015Learning Depth from Single Monocular Images Using Deep Convolutional Neural Fieldscited by 169
  20. 2022BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimationcited by 166
  21. 2018Unsupervised Learning of Monocular Depth Estimation and Visual Odometry with Deep Feature Reconstructioncited by 159
  22. 2022NeW CRFs: Neural Window Fully-connected CRFs for Monocular Depth Estimationcited by 151
  23. 2020Feature-metric Loss for Self-supervised Learning of Depth and Egomotioncited by 140
  24. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  25. 2019Self-Supervised Monocular Depth Hintscited by 127
  26. 2022DepthFormer: Exploiting Long-Range Correlation and Local Information for Accurate Monocular Depth Estimationcited by 116
  27. 2017Semi-Supervised Deep Learning for Monocular Depth Map Predictioncited by 109
  28. 2018Every Pixel Counts ++: Joint Learning of Geometry and Motion with 3D Holistic Understandingcited by 101
  29. 2018Deep Virtual Stereo Odometry: Leveraging Deep Depth Prediction for Monocular Direct Sparse Odometrycited by 85
  30. 2018Layer-structured 3D Scene Inference via View Synthesiscited by 82
  31. 2020MonoRec: Semi-Supervised Dense Reconstruction in Dynamic Environments from a Single Moving Cameracited by 59
  32. 2022AutoRF: Learning 3D Object Radiance Fields from Single View Observationscited by 59
  33. 2020Forget About the LiDAR: Self-Supervised Depth Estimators with MED Probability Volumescited by 50
  34. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  35. 2020Bidirectional Attention Network for Monocular Depth Estimationcited by 48
  36. 2022De-rendering 3D Objects in the Wildcited by 31
  37. 2019Object-Driven Multi-Layer Scene Decomposition From a Single Imagecited by 26
  38. 2018Peeking Behind Objects: Layered Depth Prediction from a Single Imagecited by 23

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 12; 2024: 24; 2025: 17; 2026: 10.

63 papers.

Sort:

By area:

  1. 20243DGStream: On-the-Fly Training of 3D Gaussians for Efficient Streaming of Photo-Realistic Free-Viewpoint Videoscited by 156
  2. 2024HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splattingcited by 141
  3. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  4. 2023RenderOcc: Vision-Centric 3D Occupancy Prediction with 2D Rendering Supervisioncited by 126
  5. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  6. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 83
  7. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  8. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understandingcited by 46
  9. 2023ViP-NeRF: Visibility Prior for Sparse Input Neural Radiance Fieldscited by 43
  10. 2023MonoNeRD: NeRF-like Representations for Monocular 3D Object Detectioncited by 33
  11. 2023MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representationscited by 33
  12. 2023Sat2Density: Faithful Density Learning from Satellite-Ground Image Pairscited by 29
38 more
  1. 2024Vision-based 3D occupancy prediction in autonomous driving: a review and outlookcited by 29
  2. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  3. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  4. 2024Forging Vision Foundation Models for Autonomous Driving: Challenges, Methodologies, and Opportunitiescited by 24
  5. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  6. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 23
  7. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19
  8. 2024FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priorscited by 18
  9. 2024Gaussian Splatting in Stylecited by 16
  10. 2024LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenescited by 14
  11. 2024Know Your Neighbors: Improving Single-View Reconstruction via Spatial Vision-Language Reasoningcited by 12
  12. 2025Feed-Forward SceneDINO for Unsupervised Semantic Scene Completioncited by 11
  13. 2024OccScene: Semantic Occupancy-based Cross-task Mutual Learning for 3D Scene Generationcited by 10
  14. 2024Semantically-aware Neural Radiance Fields for Visual Scene Understanding: A Comprehensive Reviewcited by 9
  15. 2024A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervisioncited by 9
  16. 2025High-Fidelity Novel View Synthesis via Splatting-Guided Diffusioncited by 9
  17. 2024ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Predictioncited by 8
  18. 2024UrbanCAD: Towards Highly Controllable and Photorealistic 3D Vehicles for Urban Scene Simulationcited by 7
  19. 2025LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoningcited by 7
  20. 2024Sketch2NeRF: Multi-view Sketch-guided Text-to-3D Generationcited by 6
  21. 2024Boosting Self-Supervision for Single-View Scene Completion via Knowledge Distillationcited by 6
  22. 2024CATSplat: Context-Aware Transformer with Spatial Guidance for Generalizable 3D Gaussian Splatting from A Single-View Imagecited by 6
  23. 2024GSRender: Deduplicated Occupancy Prediction via Weakly Supervised 3D Gaussian Splattingcited by 6
  24. 2026OccAny: Generalized Unconstrained Urban 3D Occupancycited by 6
  25. 2023A Simple Framework for 3D Occupancy Estimation in Autonomous Drivingcited by 4
  26. 2025Pointmap-Conditioned Diffusion for Consistent Novel View Synthesiscited by 4
  27. 2025RendBEV: Semantic Novel View Synthesis for Self-Supervised Bird's Eye View Segmentationcited by 3
  28. 2025Niagara: Normal-Integrated Geometric Affine Field for Scene Reconstruction from a Single Viewcited by 3
  29. 2025Uncertainty-Aware Diffusion Guided Refinement of 3D Scenescited by 3
  30. 2025TwoSquared: 4D Generation from 2D Image Pairscited by 3
  31. 2025IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposalscited by 3
  32. 2023LiDAR View Synthesis for Robust Vehicle Navigation Without Expert Labelscited by 2
  33. 2023ADU-Depth: Attention-based Distillation with Uncertainty Modeling for Depth Estimationcited by 2
  34. 2023Novel View Synthesis with View-Dependent Effects from a Single Imagecited by 2
  35. 2025Scale-GS: Efficient Scalable Gaussian Splatting via Redundancy-filtering Training on Streaming Contentcited by 2
  36. 2026Guardians of the Hair: Rescuing Soft Boundaries in Depth, Stereo, and Novel Viewscited by 2
  37. 2024LetsMap: Unsupervised Representation Learning for Semantic BEV Mappingcited by 1
  38. 2025Dream-to-Recon: Monocular 3D Reconstruction with Diffusion-Depth Distillation from Single Imagescited by 1

50 most cited of 63 All 63 (JSON)