mlsanityData to 7 Oct 2026

Fast and Explicit Neural View Synthesis

arXiv 2107.05775 ↗First posted 12 Jul 2021cs.CV cs.GR cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

We study the problem of novel view synthesis from sparse source observations of a scene comprised of 3D objects. We propose a simple yet effective approach that is neither continuous nor implicit, challenging recent trends on view synthesis. Our approach explicitly encodes observations into a volumetric representation that enables amortized rendering. We demonstrate that although continuous radiance field representations have gained a lot of attention due to their expressive power, our simple approach obtains comparable or even better novel view reconstruction quality comparing with state-of-the-art baselines while increasing rendering speed by over 400x. Our model is trained in a category-agnostic manner and does not require scene-specific optimization. Therefore, it is able to generalize novel view synthesis to object categories not seen during training. In addition, we show that with our simple formulation, we can use view synthesis as a self-supervision signal for efficient learning of 3D geometry without explicit 3D supervision.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 45 of its 61 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

45 of 61 references.

References · 45

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  7. 2015Spatial Transformer Networkscited by 1,423
  8. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  9. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  10. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  11. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 784
  12. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
33 more
  1. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  2. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  3. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  4. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  5. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  6. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  7. 2019PointRend: Image Segmentation as Renderingcited by 439
  8. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  9. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  10. 2019Neural Point-Based Graphicscited by 326
  11. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  12. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  13. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 215
  14. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206
  15. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  16. 2017Learning a Multi-View Stereo Machinecited by 194
  17. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  18. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  19. 2016View Synthesis by Appearance Flowcited by 172
  20. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  21. 2020DeRF: Decomposed Radiance Fieldscited by 163
  22. 2018Visual Object Networks: Image Generation with Disentangled 3D Representationcited by 124
  23. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  24. 2018Extreme View Synthesiscited by 94
  25. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  26. 2019Neural Point Cloud Rendering via Multi-Plane Projectioncited by 60
  27. 2020Equivariant Neural Renderingcited by 47
  28. 2018Learning Spatial Common Sense with Geometry-Aware Recurrent Networkscited by 35
  29. 2020Deep Octree-based CNNs with Output-Guided Skip Connections for 3D Shape and Scene Completioncited by 33
  30. 2020ESRGAN+ : Further Improving Enhanced Super-Resolution Generative Adversarial Networkcited by 27
  31. 2019Transformable Bottleneck Networkscited by 23
  32. 2019Embodied Language Grounding with 3D Visual Feature Representationscited by 20
  33. 2020Tracking Emerges by Looking Around Static Scenes, with Neural 3D Mappingcited by 6

Built on by

Cited by year

Citing papers per year: 2022: 9; 2023: 11; 2024: 7; 2025: 1.

28 papers.

Sort:

By area:

  1. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  2. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  3. 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 179
  4. 2022GAUDI: A Neural Architect for Immersive 3D Scene Generationcited by 136
  5. 2024latentSplat: Autoencoding Variational Gaussians for Fast Generalizable 3D Reconstructioncited by 114
  6. 2023Viewset Diffusion: (0-)Image-Conditioned 3D Generative Models from 2D Datacited by 113
  7. 2022Vision Transformer for NeRF-Based View Synthesis from a Single Input Imagecited by 104
  8. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  9. 2022Compressing Explicit Voxel Grid Representations: fast NeRFs become also smallcited by 48
  10. 2022FWD: Real-time Novel View Synthesis with Forward Warping and Depthcited by 39
  11. 2022Transframer: Arbitrary Frame Prediction with Generative Modelscited by 37
  12. 2022Neural Groundplans: Persistent Neural Scene Representations from a Single Imagecited by 17
16 more
  1. 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
  2. 2023HyP-NeRF: Learning Improved NeRF Priors using a HyperNetworkcited by 14
  3. 2023SimNP: Learning Self-Similarity Priors Between Neural Pointscited by 11
  4. 20223inGAN: Learning a 3D Generative Model from Images of a Self-similar Scenecited by 9
  5. 2023Neural Spectro-polarimetric Fieldscited by 9
  6. 2024A Lesson in Splats: Teacher-Guided Diffusion for 3D Gaussian Splats Generation with 2D Supervisioncited by 9
  7. 2022NeuralPassthrough: Learned Real-Time View Synthesis for VRcited by 8
  8. 2024Gaussian Scenes: Pose-Free Sparse-View Scene Reconstruction using Depth-Enhanced Diffusion Priorscited by 8
  9. 2024Sp2360: Sparse-view 360 Scene Reconstruction using Cascaded 2D Diffusion Priorscited by 7
  10. 2024A Pixel Is Worth More Than One 3D Gaussians in Single-View 3D Reconstructioncited by 4
  11. 2024UniGS: Modeling Unitary 3D Gaussians for Novel View Synthesis from Sparse-view Imagescited by 3
  12. 2023360^circ Reconstruction From a Single Image Using Space Carved Outpaintingcited by 2
  13. 2023Mask-Based Modeling for Neural Radiance Fieldscited by 1
  14. 2025Interpretable Single-View 3D Gaussian Splatting using Unsupervised Hierarchical Disentangled Representation Learningcited by 1
  15. 2023Depth self-supervision for single image novel view synthesiscited by 0
  16. 2024GD^2-NeRF: Generative Detail Compensation via GAN and Diffusion for One-shot Generalizable Neural Radiance Fieldscited by 0