mlsanityData to 7 Oct 2026

FWD: Real-time Novel View Synthesis with Forward Warping and Depth

arXiv 2206.08355 ↗First posted 16 Jun 2022cs.CVcited by 39

Not on the map. Why?

Abstract (arXiv)

Novel view synthesis (NVS) is a challenging task requiring systems to generate photorealistic images of scenes from new viewpoints, where both quality and speed are important for applications. Previous image-based rendering (IBR) methods are fast, but have poor quality when input views are sparse. Recent Neural Radiance Fields (NeRF) and generalizable variants give impressive results but are not real-time. In our paper, we propose a generalizable NVS method with sparse inputs, called FWD, which gives high-quality synthesis in real-time. With explicit depth and differentiable rendering, it achieves competitive results to the SOTA methods with 130-1000x speedup and better perceptual quality. If available, we can seamlessly integrate sensor depth during either training or inference to improve image quality while retaining real-time speed. With the growing prevalence of depths sensors, we hope that methods making use of depth will become increasingly useful.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 73 of its 104 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

73 of 104 references.

References · 73

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  8. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  10. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  12. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
38 more
  1. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  3. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  4. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  7. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  8. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  9. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  10. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  11. 2020Neural Sparse Voxel Fieldscited by 959
  12. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  13. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  14. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  15. 2018Self-Attention Generative Adversarial Networkscited by 809
  16. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  17. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  18. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  19. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  20. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  21. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  22. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  23. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  24. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  25. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  26. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  27. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  28. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  29. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  30. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  31. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  32. 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 239
  33. 2021DONeRF: Towards Real-Time Rendering of Compact Neural Radiance Fields using Depth Oracle Networkscited by 232
  34. 2020Free View Synthesiscited by 231
  35. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  36. 2020Consistent Video Depth Estimationcited by 220
  37. 2021NeX: Real-time View Synthesis with Neural Basis Expansioncited by 215
  38. 2018Stereo Magnification: Learning View Synthesis using Multiplane Imagescited by 206

50 most cited of 73 All 73 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 15; 2024: 13; 2025: 4; 2026: 1.

39 papers.

Sort:

By area:

  1. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  2. 2023Splatter Image: Ultra-Fast Single-View 3D Reconstructioncited by 359
  3. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  4. 2023GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesiscited by 170
  5. 2022Vision Transformer for NeRF-Based View Synthesis from a Single Input Imagecited by 104
  6. 2023SHERF: Generalizable Human NeRF from a Single Imagecited by 88
  7. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  8. 2023Real-Time Radiance Fields for Single-Image Portrait View Synthesiscited by 64
  9. 2024Advances in 3D Generation: A Surveycited by 62
  10. 2024AffordanceLLM: Grounding Affordance from Vision Language Modelscited by 55
  11. 2022SceneRF: Self-Supervised Monocular 3D Scene Reconstruction with Radiance Fieldscited by 49
  12. 2023Real-Time Neural Rasterization for Large Scenescited by 37
27 more
  1. 2023SurfelNeRF: Neural Surfel Radiance Fields for Online Photorealistic Reconstruction of Indoor Scenescited by 30
  2. 2023Fast View Synthesis of Casual Videos with Soup-of-Planescited by 22
  3. 2024GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Viewscited by 22
  4. 2024GSD: View-Guided Gaussian Splatting Diffusion for 3D Reconstructioncited by 18
  5. 2023Decoupling Dynamic Monocular Videos for Dynamic View Synthesiscited by 17
  6. 2024Quark: Real-time, High-resolution, and General Neural View Synthesiscited by 16
  7. 2023MapNeRF: Incorporating Map Priors into Neural Radiance Fields for Driving View Simulationcited by 15
  8. 2023HyP-NeRF: Learning Improved NeRF Priors using a HyperNetworkcited by 14
  9. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  10. 2024BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenescited by 12
  11. 2024Tele-Aloha: A Low-budget and High-authenticity Telepresence System Using Sparse RGB Camerascited by 11
  12. 2023GNeSF: Generalizable Neural Semantic Fieldscited by 9
  13. 2022Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimationcited by 8
  14. 2022Learning A Locally Unified 3D Point Cloud for View Synthesiscited by 6
  15. 2023LiveNVS: Neural View Synthesis on Live RGB-D Streamscited by 6
  16. 2024HashPoint: Accelerated Point Searching and Sampling for Neural Renderingcited by 4
  17. 2023Enhancing Interpretable Object Abstraction via Clustering-based Slot Initializationcited by 3
  18. 2024UniGS: Modeling Unitary 3D Gaussians for Novel View Synthesis from Sparse-view Imagescited by 3
  19. 2024Spatial Annealing for Efficient Few-shot Neural Renderingcited by 2
  20. 2024SGCNeRF: Few-Shot Neural Rendering via Sparse Geometric Consistency Guidancecited by 1
  21. 2025Confident Splatting: Confidence-Based Compression of 3D Gaussian Splatting via Learnable Beta Distributionscited by 1
  22. 2026PRISM: Feed-Forward Single-Image 3D Reconstruction via Geometric Warp-Residual Modelingcited by 1
  23. 2023Depth self-supervision for single image novel view synthesiscited by 0
  24. 2024Incorporating dense metric depth into neural 3D representations for view synthesis and relightingcited by 0
  25. 2024GMT: Enhancing Generalizable Neural Rendering via Geometry-Driven Multi-Reference Texture Transfercited by 0
  26. 2025Geometry-guided Online 3D Video Synthesis with Multi-View Temporal Consistencycited by 0
  27. 2025GaussianGAN: Real-Time Photorealistic controllable Human Avatarscited by 0