mlsanityData to 7 Oct 2026

Layer-structured 3D Scene Inference via View Synthesis

arXiv 1807.10264 ↗First posted 26 Jul 2018cs.CVcited by 82

Not on the map. Why?

Abstract (arXiv)

We present an approach to infer a layer-structured 3D representation of a scene from a single input image. This allows us to infer not only the depth of the visible pixels, but also to capture the texture and depth for content in the scene that is not directly visible. We overcome the challenge posed by the lack of direct supervision by instead leveraging a more naturally available multi-view supervisory signal. Our insight is to use view synthesis as a proxy task: we enforce that our representation (inferred from a single image), when rendered from a novel perspective, matches the true observed image. We present a learning framework that operationalizes this insight using a new, differentiable novel view renderer. We provide qualitative and quantitative validation of our approach in two different settings, and demonstrate that we can learn to capture the hidden aspects of a scene.

Key facts. Cited by 82 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 16; 2023: 20; 2024: 9; 2025: 14; 2026: 1.

82 papers.

Sort:

By area:

  1. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  2. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  3. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  4. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  5. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 215
  6. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  7. 2021Point-Based Neural Rendering with Per-View Optimizationcited by 188
  8. 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
  9. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  10. 2023F2-NeRF: Fast Neural Radiance Field Training with Free Camera Trajectoriescited by 118
  11. 2024Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Imagecited by 113
  12. 2021NeRF-SR: High-Quality Neural Radiance Fields using Supersamplingcited by 95
38 more
  1. 2021PixelSynth: Generating a 3D-Consistent Experience from a Single Imagecited by 82
  2. 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
  3. 2021GeoSim: Realistic Video Simulation via Geometry-Aware Composition for Self-Drivingcited by 75
  4. 2023Behind the Scenes: Density Fields for Single View Reconstructioncited by 63
  5. 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
  6. 2025Bolt3D: Generating 3D Scenes in Secondscited by 57
  7. 2022Look Outside the Room: Synthesizing A Consistent Long-Term 3D Scene Video from A Single Imagecited by 55
  8. 2022DiffDreamer: Towards Consistent Unsupervised Single-view Scene Extrapolation with Conditional Diffusion Modelscited by 51
  9. 2021Omnimatte: Associating Objects and Their Effects in Videocited by 50
  10. 2021Learning Neural Light Fields with Ray-Space Embedding Networkscited by 49
  11. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  12. 20213D Photo Stylization: Learning to Generate Stylized Novel Views from a Single Imagecited by 45
  13. 2023Persistent Nature: A Generative Model of Unbounded 3D Worldscited by 44
  14. 2021Depth Completion with Twin Surface Extrapolation at Occlusion Boundariescited by 41
  15. 2025WonderTurbo: Generating Interactive 3D World in 0.72 Secondscited by 31
  16. 2025SynCity: Training-Free Generation of 3D Worldscited by 30
  17. 2023CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulationcited by 26
  18. 2024Video2Game: Real-time, Interactive, Realistic and Browser-Compatible Environment from a Single Videocited by 26
  19. 20233D Cinemagraphy from a Single Imagecited by 23
  20. 2024Invisible Stitch: Generating Smooth 3D Scenes with Depth Inpaintingcited by 23
  21. 2023Reconstructing Objects in-the-wild for Realistic Sensor Simulationcited by 21
  22. 2023Decoupling Dynamic Monocular Videos for Dynamic View Synthesiscited by 17
  23. 2021Layout-Guided Novel View Synthesis from a Single Indoor Panoramacited by 16
  24. 20223D Moments from Near-Duplicate Photoscited by 15
  25. 2021Self-Supervised Visibility Learning for Novel View Synthesiscited by 14
  26. 2023Structural Multiplane Image: Bridging Neural View Synthesis and 3D Reconstructioncited by 12
  27. 2024ExtraNeRF: Visibility-Aware View Extrapolation of Neural Radiance Fields with Diffusion Modelscited by 12
  28. 2024BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenescited by 12
  29. 2022Stereo Magnification with Multi-Layer Imagescited by 11
  30. 2022Bridging Implicit and Explicit Geometric Transformation for Single-Image View Synthesiscited by 11
  31. 2023Multiscale Representation for Real-Time Anti-Aliasing Neural Renderingcited by 11
  32. 2023ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Fieldcited by 11
  33. 2022PanoDepth: A Two-Stage Approach for Monocular Omnidirectional Depth Estimationcited by 9
  34. 2021What's Behind the Couch? Directed Ray Distance Functions (DRDF) for 3D Scene Reconstructioncited by 7
  35. 2023MPI-Flow: Learning Realistic Optical Flow with Multiplane Imagescited by 7
  36. 2025LaRI: Layered Ray Intersections for Single-view 3D Geometric Reasoningcited by 7
  37. 2025WonderFree: Enhancing Novel View Quality and Cross-View Consistency for 3D Scene Explorationcited by 7
  38. 2022Learning A Locally Unified 3D Point Cloud for View Synthesiscited by 6

50 most cited of 82 All 82 (JSON)