mlsanityData to 7 Oct 2026

Point-Based Neural Rendering with Per-View Optimization

arXiv 2109.02369 ↗First posted 6 Sep 2021cs.CV cs.GRcited by 188

Not on the map. Why?

Abstract (arXiv)

There has recently been great interest in neural rendering methods. Some approaches use 3D geometry reconstructed with Multi-View Stereo (MVS) but cannot recover from the errors of this process, while others directly learn a volumetric neural representation, but suffer from expensive training and inference. We introduce a general approach that is initialized with MVS, but allows further optimization of scene properties in the space of input views, including depth and reprojected features, resulting in improved novel-view synthesis. A key element of our approach is our new differentiable point-based pipeline, based on bi-directional Elliptical Weighted Average splatting, a probabilistic depth test and effective camera selection. We use these elements together in our neural renderer, that outperforms all previous methods both in quality and speed in almost all scenes we tested. Our pipeline can be applied to multi-view harmonization and stylization in addition to novel-view synthesis.

Key facts. Cited by 188 papers in the map (data through 7 Oct 2026). Builds on 28 of its 71 references that are in the map. First posted on arXiv 6 Sep 2021.

Builds on

28 of 71 references.

References · 28

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2017Deep Setscited by 1,394
  3. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  4. 2017Deep Image Priorcited by 1,097
  5. 2020Neural Sparse Voxel Fieldscited by 959
  6. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  7. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  8. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  9. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  10. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  11. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  12. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
16 more
  1. 2019SynSin: End-to-end View Synthesis from a Single Imagecited by 364
  2. 2019Neural Point-Based Graphicscited by 326
  3. 2020State of the Art on Neural Renderingcited by 276
  4. 2019DeepView: View Synthesis with Learned Gradient Descentcited by 253
  5. 2019Differentiable Surface Splatting for Point-based Geometry Processingcited by 238
  6. 2020Free View Synthesiscited by 231
  7. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  8. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  9. 2020Stable View Synthesiscited by 165
  10. 2018The Contextual Loss for Image Transformation with Non-Aligned Datacited by 135
  11. 2018Extreme View Synthesiscited by 94
  12. 2018Layer-structured 3D Scene Inference via View Synthesiscited by 82
  13. 2020MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Imagescited by 64
  14. 2017Artistic style transfer for videos and spherical imagescited by 45
  15. 2020Interactive Video Stylization Using Few-Shot Patch-Based Trainingcited by 43
  16. 2020Light Stage Super-Resolution: Continuous High-Frequency Relightingcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 9; 2022: 11; 2023: 45; 2024: 71; 2025: 38; 2026: 14.

188 papers.

Sort:

By area:

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  3. 20242D Gaussian Splatting for Geometrically Accurate Radiance Fieldscited by 1,061
  4. 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 684
  5. 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 668
  6. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
  7. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  8. 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
  9. 2023LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPScited by 420
  10. 2023Gaussian Grouping: Segment and Edit Anything in 3D Scenescited by 387
  11. 2021Advances in Neural Renderingcited by 365
  12. 2023Compact 3D Gaussian Representation for Radiance Fieldcited by 365
38 more
  1. 2023Feature 3DGS: Supercharging 3D Gaussian Splatting to Enable Distilled Feature Fieldscited by 361
  2. 2021NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Imagescited by 336
  3. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
  4. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 257
  5. 2023GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Modelscited by 255
  6. 2022Panoptic Neural Fields: A Semantic Object-Aware Neural Scene Representationcited by 243
  7. 2024A Survey on 3D Gaussian Splattingcited by 221
  8. 2023BakedSDF: Meshing Neural SDFs for Real-Time View Synthesiscited by 220
  9. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
  10. 2023Depth-Regularized Optimization for 3D Gaussian Splatting in Few-Shot Imagescited by 172
  11. 2023GauHuman: Articulated Gaussian Splatting from Monocular Human Videoscited by 168
  12. 2024High-quality Surface Reconstruction using Gaussian Surfelscited by 159
  13. 2023Compact 3D Scene Representation via Self-Organizing Gaussian Gridscited by 147
  14. 20243D Gaussian Ray Tracing: Fast Tracing of Particle Scenescited by 138
  15. 2024DN-Splatter: Depth and Normal Priors for Gaussian Splatting and Meshingcited by 136
  16. 2023Relightable Gaussian Codec Avatarscited by 126
  17. 2023Drivable 3D Gaussian Avatarscited by 120
  18. 2024Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splattingcited by 111
  19. 2023FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embeddingcited by 107
  20. 2023Learning to Render Novel Views from Wide-Baseline Stereo Pairscited by 103
  21. 20243D Gaussian as a New Era: A Surveycited by 102
  22. 2022NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstructioncited by 99
  23. 2023GauFRe: Gaussian Deformation Fields for Real-time Dynamic Novel View Synthesiscited by 95
  24. 20234K4D: Real-Time 4D View Synthesis at 4K Resolutioncited by 91
  25. 20243DGUT: Enabling Distorted Cameras and Secondary Rays in Gaussian Splattingcited by 87
  26. 2022Differentiable Point-Based Radiance Fields for Efficient View Synthesiscited by 86
  27. 2023Neural Point Catacaustics for Novel-View Synthesis of Reflectionscited by 81
  28. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  29. 2023Adaptive Shells for Efficient Neural Radiance Field Renderingcited by 69
  30. 2023SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Explorationcited by 62
  31. 2024Advances in 3D Generation: A Surveycited by 62
  32. 2023Gaussian Shell Maps for Efficient 3D Human Generationcited by 61
  33. 2021StyleMesh: Style Transfer for Indoor 3D Scene Reconstructionscited by 58
  34. 2024Click-Gaussian: Interactive Segmentation to Any 3D Gaussianscited by 58
  35. 2023Tetra-NeRF: Representing Neural Radiance Fields Using Tetrahedracited by 57
  36. 2024Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesiscited by 56
  37. 2023Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenescited by 55
  38. 2024SplatFields: Neural Gaussian Splats for Sparse 3D and 4D Reconstructioncited by 52

50 most cited of 188 All 188 (JSON)