mlsanityData to 7 Oct 2026

VEGS: View Extrapolation of Urban Scenes in 3D Gaussian Splatting using Learned Priors

arXiv 2407.02945 ↗First posted 3 Jul 2024cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

Neural rendering-based urban scene reconstruction methods commonly rely on images collected from driving vehicles with cameras facing and moving forward. Although these methods can successfully synthesize from views similar to training camera trajectory, directing the novel view outside the training camera distribution does not guarantee on-par performance. In this paper, we tackle the Extrapolated View Synthesis (EVS) problem by evaluating the reconstructions on views such as looking left, right or downwards with respect to training camera distributions. To improve rendering quality for EVS, we initialize our model by constructing dense LiDAR map, and propose to leverage prior scene knowledge such as surface normal estimator and large-scale diffusion model. Qualitative and quantitative comparisons demonstrate the effectiveness of our methods on EVS. To the best of our knowledge, we are the first to address the EVS problem in urban scene reconstruction. Link to our project page: https://vegs3d.github.io/.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 39 of its 46 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

39 of 46 references.

References · 39

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  8. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  9. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  11. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  12. 2018Demystifying MMD GANscited by 1,268
27 more
  1. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  2. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  3. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  4. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  5. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  6. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  7. 2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
  8. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  9. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  10. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  11. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  12. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  13. 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
  14. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 396
  15. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  16. 2021Neural RGB-D Surface Reconstructioncited by 312
  17. 2023FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularizationcited by 310
  18. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  19. 2023MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Drivingcited by 170
  20. 2022Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentationcited by 152
  21. 2023DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Modelscited by 127
  22. 2023S-NeRF: Neural Radiance Fields for Street Viewscited by 116
  23. 2022Neural 3D Reconstruction in the Wildcited by 79
  24. 2021A Survey on Simulators for Testing Self-Driving Carscited by 26
  25. 2022Ray Priors through Reprojection: Improving Neural Radiance Fields for Novel View Extrapolationcited by 26
  26. 2023GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fieldscited by 26
  27. 2023NeRFVS: Neural Radiance Fields for Free View Synthesis via Geometry Scaffoldscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 16; 2026: 4.

25 papers.

Sort:

By area:

  1. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  2. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  3. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  4. 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representationcited by 36
  5. 2025MTGS: Multi-Traversal Gaussian Splattingcited by 17
  6. 2024Extrapolated Urban View Synthesis Benchmarkcited by 13
  7. 2024Driving View Synthesis on Free-form Trajectories with Generative Priorcited by 11
  8. 2025Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Surveycited by 9
  9. 2025LSD-3D: Large-Scale 3D Driving Scene Generation with Geometry Groundingcited by 5
  10. 2025Pointmap-Conditioned Diffusion for Consistent Novel View Synthesiscited by 4
  11. 2026GenAssets: Generating in-the-wild 3D Assets in Latent Spacecited by 4
  12. 2025VDEGaussian: Video Diffusion Enhanced 4D Gaussian Splatting for Dynamic Urban Scenes Modelingcited by 2
13 more
  1. 2025JRN-Geo: A Joint Perception Network based on RGB and Normal images for Cross-view Geo-localizationcited by 2
  2. 2025sshELF: Single-Shot Hierarchical Extrapolation of Latent Features for 3D Reconstruction from Sparse-Viewscited by 1
  3. 2025FPGS: Feed-Forward Semantic-aware Photorealistic Style Transfer of Large-Scale Gaussian Splattingcited by 1
  4. 2025Hybrid Gaussian Splatting for Novel Urban View Synthesiscited by 1
  5. 2025DIAL-GS: Dynamic Instance Aware Reconstruction for Label-free Street Scenes with 4D Gaussian Splattingcited by 1
  6. 2025FaithFusion: Harmonizing Reconstruction and Generation via Pixel-wise Information Gaincited by 1
  7. 2025SurfFill: Completion of LiDAR Point Clouds via Gaussian Surfel Splattingcited by 1
  8. 20253D Gaussian Splatting against Moving Objects for High-Fidelity Street Scene Reconstructioncited by 0
  9. 2025UrbanCraft: Urban View Extrapolation via Hierarchical Sem-Geometric Priorscited by 0
  10. 2025ArbiViewGen: Controllable Arbitrary Viewpoint Camera Data Generation for Autonomous Driving via Stable Diffusion Modelscited by 0
  11. 2026Enhancing Novel View Synthesis via Geometry Grounded Set Diffusioncited by 0
  12. 2026Diffusion-guided Generalizable Enhancer for Urban Scene Reconstructioncited by 0
  13. 2026G2ARD-GS: Geometry-Guided Anchor-Regularized Gaussian Splatting Distillationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior
  2. 2025LiDAR-GS++:Improving LiDAR Gaussian Reconstruction via Diffusion Priors
  3. 2025MuDG: Taming Multi-modal Diffusion with Gaussian Splatting for Urban Scene Reconstruction
  4. 2024UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations
  5. 2026L2D2-GS: Learning to Densify for Feedforward Dynamic Gaussian Scene Reconstruction
  6. 2024FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training
  7. 2026Geo-EVS: Geometry-Conditioned Extrapolative View Synthesis for Autonomous Driving
  8. 2025D2GS: Dense Depth Regularization for LiDAR-free Urban Scene Reconstruction