mlsanityData to 7 Oct 2026

Urban Radiance Fields

arXiv 2111.14643 ↗First posted 29 Nov 2021cs.CV cs.GRcited by 263

Not on the map. Why?

Abstract (arXiv)

The goal of this work is to perform 3D reconstruction and novel view synthesis from data captured by scanning platforms commonly deployed for world mapping in urban outdoor environments (e.g., Street View). Given a sequence of posed RGB images and lidar sweeps acquired by cameras and scanners moving through an outdoor scene, we produce a model from which 3D surfaces can be extracted and novel RGB images can be synthesized. Our approach extends Neural Radiance Fields, which has been demonstrated to synthesize realistic novel images for small scenes in controlled settings, with new methods for leveraging asynchronously captured lidar data, for addressing exposure variation between captured images, and for leveraging predicted image segmentations to supervise densities on rays pointing at the sky. Each of these three extensions provides significant performance improvements in experiments on Street View data. Our system produces state-of-the-art 3D surface reconstructions and synthesizes higher quality novel views in comparison to both traditional methods (e.g.~COLMAP) and recent neural representations (e.g.~Mip-NeRF).

Key facts. Cited by 263 papers in the map (data through 7 Oct 2026). Builds on 41 of its 72 references that are in the map. First posted on arXiv 29 Nov 2021.

Builds on

41 of 72 references.

References · 41

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  5. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  8. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  9. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  10. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  11. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  12. 2021Volume Rendering of Neural Implicit Surfacescited by 990
29 more
  1. 2020Neural Sparse Voxel Fieldscited by 959
  2. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  3. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  4. 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 857
  5. 2021MVSNeRF: Fast Generalizable Radiance Field Reconstruction from Multi-View Stereocited by 735
  6. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  7. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  8. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  9. 2021Common Objects in 3D: Large-Scale Learning and Evaluation of Real-life 3D Category Reconstructioncited by 646
  10. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  11. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  12. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  13. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  14. 2021iMAP: Implicit Mapping and Positioning in Real-Timecited by 476
  15. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  16. 2021In-Place Scene Labelling and Understanding with Implicit Scene Representationcited by 411
  17. 2021Neural RGB-D Surface Reconstructioncited by 312
  18. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  19. 2021Learning Object-Compositional Neural Radiance Field for Editable Scene Renderingcited by 228
  20. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  21. 2021NerfingMVS: Guided Optimization of Neural Radiance Fields for Indoor Multi-view Stereocited by 191
  22. 2021Stereo Radiance Fields (SRF): Learning View Synthesis for Sparse Views of Novel Scenescited by 190
  23. 2015DeepStereo: Learning to Predict New Views from the World's Imagerycited by 185
  24. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  25. 2021MINE: Towards Continuous Depth MPI with NeRF for Novel View Synthesiscited by 133
  26. 2021GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worldscited by 98
  27. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  28. 2020Neural Volume Rendering: NeRF And Beyondcited by 44
  29. 2017Mesh-based 3D Textured Urban Mappingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 23; 2023: 87; 2024: 92; 2025: 38; 2026: 19.

263 papers.

Sort:

By area:

  1. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 396
  2. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
  3. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  4. 2021BungeeNeRF: Progressive Neural Radiance Field for Extreme Multi-scale Scene Renderingcited by 228
  5. 2024A Survey on 3D Gaussian Splattingcited by 221
  6. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
  7. 2023EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervisioncited by 216
  8. 2023Language Embedded 3D Gaussians for Open-Vocabulary Scene Understandingcited by 205
  9. 2024CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussianscited by 190
  10. 2023MatrixCity: A Large-scale City Dataset for City-scale Neural Rendering and Beyondcited by 185
  11. 2023MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Drivingcited by 170
  12. 2023Point-SLAM: Dense Neural Point Cloud-based SLAMcited by 160
38 more
  1. 2024OmniRe: Omni Urban Scene Reconstructioncited by 159
  2. 2022NeRF: Neural Radiance Field in 3D Vision: A Comprehensive Review (Updated Post-Gaussian Splatting)cited by 156
  3. 2023SUDS: Scalable Urban Dynamic Scenescited by 151
  4. 2021Efficient Neural Radiance Fields for Interactive Free-viewpoint Videocited by 150
  5. 2024WildGaussians: 3D Gaussian Splatting in the Wildcited by 150
  6. 2022Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representationcited by 149
  7. 2023NeuRAD: Neural Rendering for Autonomous Drivingcited by 146
  8. 2023Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Renderingcited by 143
  9. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 136
  10. 2023SelfOcc: Self-Supervised Vision-Based 3D Occupancy Predictioncited by 133
  11. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  12. 2021NeRFReN: Neural Radiance Fields with Reflectionscited by 121
  13. 2023StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Viewscited by 120
  14. 2023S-NeRF: Neural Radiance Fields for Street Viewscited by 116
  15. 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusioncited by 105
  16. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Drivingcited by 105
  17. 2022DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenescited by 103
  18. 2023Progressively Optimized Local Radiance Fields for Robust View Synthesiscited by 103
  19. 2023RobustNeRF: Ignoring Distractors with Robust Lossescited by 101
  20. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  21. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  22. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
  23. 2022Neural 3D Reconstruction in the Wildcited by 79
  24. 2023NeuralField-LDM: Scene Generation with Hierarchical Latent Diffusion Modelscited by 76
  25. 2023Learning Neural Volumetric Representations of Dynamic Humans in Minutescited by 70
  26. 2024MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenescited by 70
  27. 2023Forward Flow for Novel View Synthesis of Dynamic Scenescited by 67
  28. 2022Removing Objects From Neural Radiance Fieldscited by 66
  29. 2024SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Drivingcited by 66
  30. 2023Neural LiDAR Fields for Novel View Synthesiscited by 65
  31. 2024SWAG: Splatting in the Wild images with Appearance-conditioned Gaussianscited by 61
  32. 2024SpotlessSplats: Ignoring Distractors in 3D Gaussian Splattingcited by 61
  33. 2023NeRF2: Neural Radio-Frequency Radiance Fieldscited by 56
  34. 2023Urban Radiance Field Representation with Deformable Neural Mesh Primitivescited by 55
  35. 2023NeO 360: Neural Fields for Sparse View Synthesis of Outdoor Scenescited by 53
  36. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53
  37. 2024FreeVS: Generative View Synthesis on Free Driving Trajectorycited by 52
  38. 2024DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensuscited by 51

50 most cited of 263 All 263 (JSON)