mlsanityData to 14 Oct 2026

GaussianAD: Gaussian-Centric End-to-End Autonomous Driving

arXiv 2412.10371 ↗First posted 13 Dec 2024cs.CV cs.AI cs.LGcited by 31

On the map: End-to-end autonomous driving → Generative simulation for driving

Abstract (arXiv)

Vision-based autonomous driving shows great potential due to its satisfactory performance and low costs. Most existing methods adopt dense representations (e.g., bird's eye view) or sparse representations (e.g., instance boxes) for decision-making, which suffer from the trade-off between comprehensiveness and efficiency. This paper explores a Gaussian-centric end-to-end autonomous driving (GaussianAD) framework and exploits 3D semantic Gaussians to extensively yet sparsely describe the scene. We initialize the scene with uniform 3D Gaussians and use surrounding-view images to progressively refine them to obtain the 3D Gaussian scene representation. We then use sparse convolutions to efficiently perform 3D perception (e.g., 3D detection, semantic map construction). We predict 3D flows for the Gaussians with dynamic semantics and plan the ego trajectory accordingly with an objective of future scene forecasting. Our GaussianAD can be trained in an end-to-end manner with optional perception labels when available. Extensive experiments on the widely used nuScenes dataset verify the effectiveness of our end-to-end GaussianAD on various tasks including motion planning, 3D occupancy prediction, and 4D occupancy forecasting. Code: https://github.com/wzzheng/GaussianAD.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Generative simulation for driving, one of 9 directions in End-to-end autonomous driving. Builds on 24 of its 66 references that are in the map. First posted on arXiv 13 Dec 2024.

Builds on

24 of 66 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024GPD-1: Generative Pre-training for Driving11 Dec

Other references · 23

  1. 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
  2. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  3. 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
  4. 2019MultiPath: Multiple Probabilistic Anchor Trajectory Hypotheses for Behavior Predictioncited by 420
  5. 2023Occ3D: A Large-Scale 3D Occupancy Prediction Benchmark for Autonomous Drivingcited by 332
  6. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 314
  7. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 245
  8. 2024SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representationcited by 241
  9. 2023VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Trackingcited by 222
  10. 2023OpenOccupancy: A Large Scale Benchmark for Surrounding Semantic Occupancy Perceptioncited by 221
  11. 2022BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Drivingcited by 191
  12. 2021Scene Transformer: A unified architecture for predicting multiple agent trajectoriescited by 165
11 more
  1. 2024GaussianFormer: Scene as Gaussians for Vision-Based 3D Semantic Occupancy Predictioncited by 157
  2. 2023GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Drivingcited by 141
  3. 2022ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queriescited by 117
  4. 2024GaussianFormer-2: Probabilistic Gaussian Superposition for Efficient 3D Occupancy Predictioncited by 86
  5. 2023FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Drivingcited by 67
  6. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 52
  7. 2024SparseAD: Sparse Query-Centric Paradigm for Efficient End-to-End Autonomous Drivingcited by 47
  8. 2022Perceive, Interact, Predict: Learning Dynamic and Static Clues for End-to-End Motion Predictioncited by 28
  9. 2023Rethinking Imitation-based Planner for Autonomous Drivingcited by 25
  10. 2023Hierarchical Generative Adversarial Imitation Learning with Mid-level Input Generation for Autonomous Driving on Urban Environmentscited by 12
  11. 2023CCIL: Context-conditioned imitation learning for urban drivingcited by 12

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 23; 2026: 8.

31 papers.

Sort:

By area:

  1. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  2. 20253D and 4D World Modeling: A Surveycited by 67
  3. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 33
  4. 2025GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flowcited by 24
  5. 2025DVGT: Driving Visual Geometry Transformercited by 21
  6. 2025QuadricFormer: Scene as Superquadrics for 3D Semantic Occupancy Predictioncited by 20
  7. 2025GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Drivingcited by 16
  8. 2025PRIX: Learning to Plan from Raw Pixels for End-to-End Autonomous Drivingcited by 16
  9. 2025InsightDrive: Insight Scene Representation for End-to-End Autonomous Drivingcited by 14
  10. 2025GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attentioncited by 14
  11. 2026DVGT-2: Vision-Geometry-Action Model for Autonomous Driving at Scalecited by 13
  12. 2025MindDrive: An All-in-One Framework Bridging World Models and Vision-Language Model for End-to-End Autonomous Drivingcited by 12
19 more
  1. 2025GS-Occ3D: Scaling Vision-only Occupancy Reconstruction with Gaussian Splattingcited by 11
  2. 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Drivingcited by 11
  3. 2026MindDriver: Introducing Progressive Multimodal Reasoning for Autonomous Drivingcited by 10
  4. 2025Learning-based 3D Reconstruction in Autonomous Driving: A Comprehensive Surveycited by 9
  5. 2026Vega: Learning to Drive with Natural Language Instructionscited by 6
  6. 2025SQS: Enhancing Sparse Perception Models via Query-based Splatting in Autonomous Drivingcited by 4
  7. 2025WPT: World-to-Policy Transfer via Online World Model Distillationcited by 4
  8. 2025SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Modelcited by 4
  9. 2025RadarGaussianDet3D: Gaussian Representation-based Real-time 3D Object Detection with 4D Automotive Radarscited by 3
  10. 2025Vision-Centric 4D Occupancy Forecasting and Planning via Implicit Residual World Modelscited by 3
  11. 2025ShelfGaussian: Shelf-Supervised Open-Vocabulary Gaussian-based 3D Scene Understandingcited by 3
  12. 2025UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Renderingcited by 2
  13. 2026DLWM: Dual Latent World Models enable Holistic Gaussian-centric Pre-training in Autonomous Drivingcited by 2
  14. 2025TGP: Two-modal occupancy prediction with 3D Gaussian and sparse points for 3D Environment Awarenesscited by 1
  15. 2025S2GO: Streaming Sparse Gaussian Occupancy Predictioncited by 1
  16. 2025SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Renderingcited by 0
  17. 2026GaussianMap: Learning Gaussian Representation for Multi-Sensor Online HD Map Constructioncited by 0
  18. 2026OPUS-V2: Bridging the Gap between Sparse Points and Dense Voxelscited by 0
  19. 2026SurgGMF: Fully Causal Gaussian Motion Forecasting for Anticipatory Surgical Scene Renderingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026USR-Drive: Unified Driving Scene Representation via Joint Denoising of 3D Gaussians and Boxes
  2. 2024GaussianWorld: Gaussian World Model for Streaming 3D Occupancy Prediction
  3. 2024GaussTR: Foundation Model-Aligned Gaussian Transformer for Self-Supervised 3D Spatial Understanding
  4. 2024UniGaussian: Driving Scene Reconstruction from Multiple Camera Models via Unified Gaussian Representations
  5. 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splatting
  6. 2024textitS3Gaussian: Self-Supervised Street Gaussians for Autonomous Driving
  7. 2025CoDa-4DGS: Dynamic Gaussian Splatting with Context and Deformation Awareness for Autonomous Driving
  8. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving