mlsanityData to 14 Oct 2026

Learning to Synthesize a 4D RGBD Light Field from a Single Image

arXiv 1708.03292 ↗First posted 10 Aug 2017cs.CV cs.GRcited by 60

Not on the map. Why?

Abstract (arXiv)

We present a machine learning algorithm that takes as input a 2D RGB image and synthesizes a 4D RGBD light field (color and depth of the scene in each ray direction). For training, we introduce the largest public light field dataset, consisting of over 3300 plenoptic camera light fields of scenes containing flowers and plants. Our synthesis pipeline consists of a convolutional neural network (CNN) that estimates scene geometry, a stage that renders a Lambertian light field using that geometry, and a second CNN that predicts occluded rays and non-Lambertian effects. Our algorithm builds on recent view synthesis methods, but is unique in predicting RGBD for each light field ray and improving unsupervised single image depth estimation by enforcing consistency of ray depths that should intersect the same scene point. Please see our supplementary video at https://youtu.be/yLCvWoQLnms

Key facts. Cited by 60 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Aug 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 20; 2022: 11; 2023: 18; 2024: 6; 2025: 3; 2026: 2.

60 papers.

Sort:

By area:

  1. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 740
  2. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
  3. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 216
  4. 2021InfoNeRF: Ray Entropy Minimization for Few-Shot Neural Volume Renderingcited by 182
  5. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  6. 2023Consistent View Synthesis with Pose-Guided Diffusion Modelscited by 117
  7. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 112
  8. 2021Light Field Neural Renderingcited by 108
  9. 2021Geometry-Free View Synthesis: Transformers and no 3D Priorscited by 100
  10. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  11. 2022Single-View View Synthesis in the Wild with Learned Adaptive Multiplane Imagescited by 59
  12. 2023Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collectionscited by 54
38 more
  1. 2021SLIDE: Single Image 3D Photography with Soft Layering and Depth-aware Inpaintingcited by 48
  2. 2021UnsupervisedR&R: Unsupervised Point Cloud Registration via Differentiable Renderingcited by 44
  3. 2025Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordancecited by 19
  4. 2023Hyperspectral Image Super Resolution with Real Unaligned RGB Guidancecited by 13
  5. 2023PCR-CG: Point Cloud Registration via Deep Explicit Color and Geometrycited by 13
  6. 2021Neural Relightable Participating Media Renderingcited by 12
  7. 2022A Survey on RGB-D Datasetscited by 11
  8. 2022Bridging Implicit and Explicit Geometric Transformation for Single-Image View Synthesiscited by 11
  9. 2023SinMPI: Novel View Synthesis from a Single Image with Expanded Multiplane Imagescited by 11
  10. 2021Temporal-MPI: Enabling Multi-Plane Images for Dynamic Scene Modelling via Temporal Basis Learningcited by 8
  11. 2023LightSpeed: Light and Fast Neural Light Fields on Mobile Devicescited by 8
  12. 2024UC-NeRF: Uncertainty-aware Conditional Neural Radiance Fields from Endoscopic Sparse Viewscited by 8
  13. 2021Recognizing Scenes from Novel Viewpointscited by 5
  14. 2025PolarAnything: Diffusion-based Polarimetric Image Synthesiscited by 5
  15. 2022Acquiring a Dynamic Light Field through a Single-Shot Coded Imagecited by 4
  16. 2023Single-View View Synthesis with Self-Rectified Pseudo-Stereocited by 4
  17. 2024CMC: Few-shot Novel View Synthesis via Cross-view Multiplane Consistencycited by 4
  18. 2021Learning Dynamic Interpolation for Extremely Sparse Light Fields with Wide Baselinescited by 3
  19. 2022Content-aware Warping for View Synthesiscited by 3
  20. 2023Unsupervised Light Field Depth Estimation via Multi-view Feature Matching with Occlusion Predictioncited by 3
  21. 2023ReShader: View-Dependent Highlights for Single Image View-Synthesiscited by 3
  22. 2021Learning to compose 6-DoF omnidirectional videos using multi-sphere imagescited by 2
  23. 2021Scale-Consistent Fusion: from Heterogeneous Local Sampling to Global Immersive Renderingcited by 2
  24. 2021Deep Learning based Novel View Synthesiscited by 2
  25. 2022Equivariant Light Field Convolution and Transformercited by 2
  26. 2023LoLep: Single-View View Synthesis with Locally-Learned Planes and Self-Attention Occlusion Inferencecited by 2
  27. 2023Anisotropic Neural Representation Learning for High-Quality Neural Renderingcited by 2
  28. 2024PanoDreamer: Optimization-Based Single Image to 360 3D Scene With Diffusioncited by 2
  29. 2021Machine learning method for light field refocusingcited by 1
  30. 2022Cross-Camera Deep Colorizationcited by 1
  31. 2022MiNL: Micro-images based Neural Representation for Light Fieldscited by 1
  32. 2023Single-view Neural Radiance Fields with Depth Teachercited by 1
  33. 2023Rethinking Directional Integration in Neural Radiance Fieldscited by 1
  34. 2024Sampling for View Synthesis: From Local Light Field Fusion to Neural Radiance Fields and Beyondcited by 1
  35. 2021A Learned Compact and Editable Light Field Representationcited by 0
  36. 2021FaDIV-Syn: Fast Depth-Independent View Synthesis using Soft Masks and Implicit Blendingcited by 0
  37. 2021Dynamic Scene Novel View Synthesis via Deferred Spatio-temporal Consistencycited by 0
  38. 2022Unsupervised Learning Based Focal Stack Camera Depth Estimationcited by 0

50 most cited of 60 All 60 (JSON)