mlsanityData to 7 Oct 2026

textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving

arXiv 2405.20323 ↗First posted 30 May 2024cs.CV cs.AIcited by 105

On the map: World models → World models for autonomous driving

Abstract (arXiv)

Photorealistic 3D reconstruction of street scenes is a critical technique for developing real-world simulators for autonomous driving. Despite the efficacy of Neural Radiance Fields (NeRF) for driving scenes, 3D Gaussian Splatting (3DGS) emerges as a promising direction due to its faster speed and more explicit representation. However, most existing street 3DGS methods require tracked 3D vehicle bounding boxes to decompose the static and dynamic elements for effective reconstruction, limiting their applications for in-the-wild scenarios. To facilitate efficient 3D scene reconstruction without costly annotations, we propose a self-supervised street Gaussian (textitS3Gaussian) method to decompose dynamic and static elements from 4D consistency. We represent each scene with 3D Gaussians to preserve the explicitness and further accompany them with a spatial-temporal field network to compactly model the 4D dynamics. We conduct extensive experiments on the challenging Waymo-Open dataset to evaluate the effectiveness of our method. Our textitS3Gaussian demonstrates the ability to decompose static and dynamic scenes and achieves the best performance without using 3D annotations. Code is available at: https://github.com/nnanhuang/S3Gaussian/.

Key facts. Cited by 105 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 58 of its 68 references that are in the map. First posted on arXiv 30 May 2024.

Builds on

58 of 68 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting2 Jan
  2. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers31 Mar
  3. 2017CARLA: An Open Urban Driving Simulator10 Nov

Other references · 55

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  3. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  4. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  5. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  6. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  7. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  8. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  9. 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
  10. 2022Planning-oriented Autonomous Drivingcited by 1,068
  11. 2021Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstructioncited by 961
  12. 2023Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstructioncited by 753
38 more
  1. 2023K-Planes: Explicit Radiance Fields in Space, Time, and Appearancecited by 742
  2. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  3. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  4. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  5. 2023Nerfstudio: A Modular Framework for Neural Radiance Field Developmentcited by 596
  6. 2023HexPlane: A Fast Representation for Dynamic Scenescited by 582
  7. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 517
  8. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  9. 2023Tri-Perspective View for Vision-Based 3D Semantic Occupancy Predictioncited by 404
  10. 2021Mega-NeRF: Scalable Construction of Large-Scale NeRFs for Virtual Fly-Throughscited by 396
  11. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  12. 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 359
  13. 2022Fast Dynamic Radiance Fields with Time-Aware Neural Voxelscited by 344
  14. 2021Deep ViT Features as Dense Visual Descriptorscited by 327
  15. 2019Neural Point-Based Graphicscited by 326
  16. 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 310
  17. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  18. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  19. 2021Urban Radiance Fieldscited by 263
  20. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  21. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  22. 2019Differentiable Surface Splatting for Point-based Geometry Processingcited by 238
  23. 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
  24. 2022NoPe-NeRF: Optimising Neural Radiance Field with No Pose Priorcited by 235
  25. 2022Tensor4D : Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Renderingcited by 217
  26. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  27. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  28. 2023MARS: An Instance-aware, Modular and Realistic Simulator for Autonomous Drivingcited by 170
  29. 2023SUDS: Scalable Urban Dynamic Scenescited by 151
  30. 2023NeuRAD: Neural Rendering for Autonomous Drivingcited by 146
  31. 2020PnPNet: End-to-End Perception and Prediction with Tracking in the Loopcited by 143
  32. 2023Periodic Vibration Gaussian: Dynamic Urban Scene Reconstruction and Real-time Renderingcited by 143
  33. 2023StreetSurf: Extending Multi-view Implicit Surface Reconstruction to Street Viewscited by 120
  34. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 116
  35. 2023S-NeRF: Neural Radiance Fields for Street Viewscited by 116
  36. 2021NeRF for Outdoor Scene Relightingcited by 114
  37. 2021HDR-NeRF: High Dynamic Range Neural Radiance Fieldscited by 113
  38. 2023Forecast-MAE: Self-supervised Pre-training for Motion Forecasting with Masked Autoencoderscited by 72

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 48; 2026: 35.

105 papers.

Sort:

By area:

  1. 2025Streaming 4D Visual Geometry Transformercited by 147
  2. 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representationcited by 123
  3. 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
  4. 2024ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restorationcited by 89
  5. 20253D and 4D World Modeling: A Surveycited by 64
  6. 2024STORM: Spatio-Temporal Reconstruction Model for Large-Scale Outdoor Scenescited by 53
  7. 2024GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splattingcited by 50
  8. 2024DeSiRe-GS: 4D Street Gaussians for Static-Dynamic Decomposition and Surface Reconstruction for Urban Driving Scenescited by 45
  9. 2025ReconDreamer++: Harmonizing Generative and Reconstructive Models for Driving Scene Representationcited by 36
  10. 2024DreamDrive: Generative 4D Scene Modeling from Street View Imagescited by 35
  11. 2025Segment Any Motion in Videoscited by 34
  12. 2024FreeSim: Toward Free-viewpoint Camera Simulation in Driving Scenescited by 30
38 more
  1. 2024DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Drivingcited by 27
  2. 2025CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Drivingcited by 25
  3. 2024PixelGaussian: Generalizable 3D Gaussian Reconstruction from Arbitrary Viewscited by 18
  4. 2024Stag-1: Towards Realistic 4D Driving Simulation with Video Generation Modelcited by 18
  5. 20244D Gaussian Splatting: Modeling Dynamic Scenes with Native 4D Primitivescited by 15
  6. 2024Bench2Drive-R: Turning Real World Data into Reactive Closed-Loop Autonomous Driving Benchmark by Generative Modelcited by 14
  7. 2025Rethinking Driving World Model as Synthetic Data Generator for Perception Taskscited by 14
  8. 2024EMD: Explicit Motion Modeling for High-Quality Street Gaussian Splattingcited by 13
  9. 2025ManipDreamer3D : Synthesizing Plausible Robotic Manipulation Video with Occupancy-aware 3D Trajectorycited by 12
  10. 2026WAM4D: Fast 4D World Action Model via Spatial Register Tokenscited by 12
  11. 2024Driving View Synthesis on Free-form Trajectories with Generative Priorcited by 11
  12. 2025ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidancecited by 11
  13. 2025CRUISE: Cooperative Reconstruction and Editing in V2X Scenarios using Gaussian Splattingcited by 11
  14. 2025WorldSplat: Gaussian-Centric Feed-Forward 4D Scene Generation for Autonomous Drivingcited by 11
  15. 2026VGGT-World: Transforming VGGT into an Autoregressive Geometry World Modelcited by 11
  16. 2026CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Drivingcited by 10
  17. 2024GazeGaussian: High-Fidelity Gaze Redirection with 3D Gaussian Splattingcited by 9
  18. 2025Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Surveycited by 9
  19. 2024LiHi-GS: LiDAR-Supervised Gaussian Splatting for Highway Driving Scene Reconstructioncited by 8
  20. 2025GS-LIVO: Real-Time LiDAR, Inertial, and Visual Multi-sensor Fused Odometry with Gaussian Mappingcited by 8
  21. 2025Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streamingcited by 8
  22. 2025MuDG: Taming Multi-modal Diffusion with Gaussian Splatting for Urban Scene Reconstructioncited by 8
  23. 2025UniSplat: Unified Spatio-Temporal Fusion via 3D Latent Scaffolds for Dynamic Driving Scene Reconstructioncited by 8
  24. 2024GSPR: Multimodal Place Recognition Using 3D Gaussian Splatting for Autonomous Drivingcited by 7
  25. 2024UrbanGS: Semantic-Guided Gaussian Splatting for Urban Scene Reconstructioncited by 7
  26. 2025Flux4D: Flow-based Unsupervised 4D Reconstructioncited by 7
  27. 2026ParkGaussian: Surround-view 3D Gaussian Splatting for Autonomous Parkingcited by 7
  28. 2025GauRast: Enhancing GPU Triangle Rasterizers to Accelerate 3D Gaussian Splattingcited by 6
  29. 2025From Coarse to Fine: Learnable Discrete Wavelet Transforms for Efficient 3D Gaussian Splattingcited by 6
  30. 2025ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimationcited by 6
  31. 2025OG-Gaussian: Occupancy Based Street Gaussians for Autonomous Drivingcited by 5
  32. 2025B'ezierGS: Dynamic Urban Scene Reconstruction with B'ezier Curve Gaussian Splattingcited by 5
  33. 2025LT-Gaussian: Long-Term Map Update Using 3D Gaussian Splatting for Autonomous Drivingcited by 5
  34. 2025VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splattingcited by 5
  35. 2026ReconDrive: Fast Feed-Forward 4D Gaussian Splatting for Autonomous Driving Scene Reconstructioncited by 5
  36. 2026Xiaomi Auto World Model: A Joint World Model Integrating Reconstruction and Generation for Autonomous Drivingcited by 5
  37. 2024MixedGaussianAvatar: Realistically and Geometrically Accurate Head Avatar via Mixed 2D-3D Gaussianscited by 4
  38. 2024Street Gaussians without 3D Object Trackercited by 4

50 most cited of 105 All 105 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SGD: Street View Synthesis with Gaussian Splatting and Diffusion Prior
  2. 2025DrivingScene: A Multi-Task Online Feed-Forward 3D Gaussian Splatting Method for Dynamic Driving Scenes
  3. 2026DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing
  4. 2024UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations
  5. 2026L2D2-GS: Learning to Densify for Feedforward Dynamic Gaussian Scene Reconstruction
  6. 2025RGE-GS: Reward-Guided Expansive Driving Scene Reconstruction via Diffusion Priors
  7. 2026Real2Sim: A Physics-driven and Editable Gaussian Splatting Framework for Autonomous Driving Scenes
  8. 2025Industrial-Grade Sensor Simulation via Gaussian Splatting: A Modular Framework for Scalable Editing and Full-Stack Validation