mlsanityData to 7 Oct 2026

EmerNeRF: Emergent Spatial-Temporal Scene Decomposition via Self-Supervision

arXiv 2311.02077 ↗First posted 3 Nov 2023cs.CVcited by 216

Not on the map. Why?

Cited by 34 in End-to-end autonomous driving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present EmerNeRF, a simple yet powerful approach for learning spatial-temporal representations of dynamic driving scenes. Grounded in neural fields, EmerNeRF simultaneously captures scene geometry, appearance, motion, and semantics via self-bootstrapping. EmerNeRF hinges upon two core components: First, it stratifies scenes into static and dynamic fields. This decomposition emerges purely from self-supervision, enabling our model to learn from general, in-the-wild data sources. Second, EmerNeRF parameterizes an induced flow field from the dynamic field and uses this flow field to further aggregate multi-frame features, amplifying the rendering precision of dynamic objects. Coupling these three fields (static, dynamic, and flow) enables EmerNeRF to represent highly-dynamic scenes self-sufficiently, without relying on ground truth object annotations or pre-trained models for dynamic object segmentation or optical flow estimation. Our method achieves state-of-the-art performance in sensor simulation, significantly outperforming previous methods when reconstructing static (+2.93 PSNR) and dynamic (+3.70 PSNR) scenes. In addition, to bolster EmerNeRF's semantic generalization, we lift 2D visual foundation model features into 4D space-time and address a general positional bias in modern Transformers, significantly boosting 3D perception performance (e.g., 37.50% relative improvement in occupancy prediction accuracy on average). Finally, we construct a diverse and challenging 120-sequence dataset to benchmark neural fields under extreme and highly-dynamic settings.

Key facts. Cited by 216 papers in the map (data through 7 Oct 2026). Builds on 34 of its 36 references that are in the map. First posted on arXiv 3 Nov 2023.

Builds on

34 of 36 references.

References · 34

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  7. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  8. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  9. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  10. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  11. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  12. 2023LERF: Language Embedded Radiance Fieldscited by 538
22 more
  1. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 485
  2. 2022Decomposing NeRF for Editing via Feature Field Distillationcited by 370
  3. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  4. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  5. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 325
  6. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  7. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  8. 2021Urban Radiance Fieldscited by 263
  9. 2022DynIBaR: Neural Dynamic Image-Based Renderingcited by 224
  10. 2023MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenescited by 215
  11. 2023Tracking Everything Everywhere All at Oncecited by 190
  12. 2022Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-timecited by 164
  13. 2022CLIP-Fields: Weakly Supervised Semantic Fields for Robotic Memorycited by 164
  14. 2023SUDS: Scalable Urban Dynamic Scenescited by 151
  15. 2022D2NeRF: Self-Supervised Decoupling of Dynamic and Static Objects from a Monocular Videocited by 136
  16. 2023StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Viewscited by 120
  17. 2021Neural Scene Flow Priorcited by 91
  18. 2023NerfAcc: Efficient Sampling Accelerates NeRFscited by 90
  19. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  20. 20204D Visualization of Dynamic Events from Unconstrained Multi-View Videoscited by 66
  21. 2023VL-Fields: Towards Language-Grounded Neural Implicit Spatial Representationscited by 23
  22. 2022Neural Groundplans: Persistent Neural Scene Representations from a Single Imagecited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 74; 2025: 89; 2026: 47.

216 papers.

Sort:

By area:

  1. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
  2. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  3. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  4. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 218
  5. 2023AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into Onecited by 165
  6. 2024OmniRe: Omni Urban Scene Reconstructioncited by 159
  7. 2024WildGaussians: 3D Gaussian Splatting in the Wildcited by 150
  8. 2023Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Renderingcited by 143
  9. 2024HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splattingcited by 141
  10. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  11. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  12. 2024NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wildcited by 85
38 more
  1. 2024HUGSIM: A Real-Time, Photo-Realistic and Closed-Loop Simulator for Autonomous Drivingcited by 84
  2. 2025Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Modelscited by 80
  3. 2024SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Drivingcited by 66
  4. 2024LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometrycited by 54
  5. 2024DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Inputcited by 53
  6. 2024InfiniCube: Unbounded and Controllable Dynamic 3D Driving Scene Generation with World-Guided Video Modelscited by 53
  7. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53
  8. 2024AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstructioncited by 52
  9. 2024FreeVS: Generative View Synthesis on Free Driving Trajectorycited by 52
  10. 2024StreetCrafter: Street View Synthesis with Controllable Video Diffusion Modelscited by 49
  11. 2024DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenescited by 45
  12. 2024Denoising Vision Transformerscited by 40
  13. 2025DiST-4D: Disentangled Spatiotemporal Diffusion with Metric Depth for 4D Driving Scene Generationcited by 39
  14. 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representationcited by 36
  15. 2024Dynamic 3D Gaussian Fields for Urban Areascited by 35
  16. 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
  17. 2025Segment Any Motion in Videoscited by 34
  18. 2024DistillNeRF: Perceiving 3D Scenes from Single-Glance Images by Distilling Neural Fields and Foundation Model Featurescited by 33
  19. 2024Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstructioncited by 32
  20. 2024FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenescited by 30
  21. 2024G3R: Gradient Guided Generalizable Reconstructioncited by 29
  22. 2025DRAWER: Digital Reconstruction and Articulation With Environment Realismcited by 29
  23. 2024S-NeRF++: Autonomous Driving Simulation via Neural Reconstruction and Generationcited by 28
  24. 2024Photorealistic Object Insertion with Diffusion-Guided Inverse Renderingcited by 27
  25. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 27
  26. 2025Scenario Dreamer: Vectorized Latent Diffusion for Generating Driving Simulation Environmentscited by 27
  27. 2025ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstructioncited by 27
  28. 2024Feat2GS: Probing Visual Foundation Models with Gaussian Splattingcited by 26
  29. 2025CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Drivingcited by 25
  30. 2024Let Occ Flow: Self-Supervised 3D Occupancy Flow Predictioncited by 23
  31. 2025GaussianDWM: 3D Gaussian Driving World Model for Unified Scene Understanding and Multi-Modal Generationcited by 23
  32. 2023Dynamic LiDAR Re-simulation using Compositional Neural Fieldscited by 22
  33. 2024PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priorscited by 21
  34. 2024Neural Fields in Robotics: A Surveycited by 21
  35. 2024DynOMo: Online Point Tracking by Dynamic Online Monocular Gaussian Reconstructioncited by 20
  36. 2024SplatFlow: Self-Supervised Dynamic Gaussian Splatting in Neural Motion Flow Field for Autonomous Drivingcited by 20
  37. 2025EVolSplat: Efficient Volume-based Gaussian Splatting for Urban View Synthesiscited by 20
  38. 2024Neural Radiance Field in Autonomous Driving: A Surveycited by 19

50 most cited of 216 All 216 (JSON)