Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting
Not on the map. Why?
Abstract (arXiv)
Bird's-Eye-View (BEV) perception serves as a cornerstone for autonomous driving, offering a unified spatial representation that fuses surrounding-view images to enable reasoning for various downstream tasks, such as semantic segmentation, 3D object detection, and motion prediction. However, most existing BEV perception frameworks adopt an end-to-end training paradigm, where image features are directly transformed into the BEV space and optimized solely through downstream task supervision. This formulation treats the entire perception process as a black box, often lacking explicit 3D geometric understanding and interpretability, leading to suboptimal performance. In this paper, we claim that an explicit 3D representation matters for accurate BEV perception, and we propose Splat2BEV, a Gaussian Splatting-assisted framework for BEV tasks. Splat2BEV aims to learn BEV feature representations that are both semantically rich and geometrically precise. We first pre-train a Gaussian generator that explicitly reconstructs 3D scenes from multi-view inputs, enabling the generation of geometry-aligned feature representations. These representations are then projected into the BEV space to serve as inputs for downstream tasks. Extensive experiments on nuScenes and argoverse dataset demonstrate that Splat2BEV achieves state-of-the-art performance and validate the effectiveness of incorporating explicit 3D reconstruction into BEV perception.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 47 of its 53 references that are in the map. First posted on arXiv 19 Mar 2026.
Builds on
47 of 53 references.
References · 47
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2024Depth Anything V2cited by 1,422
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
- 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,218
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,044
- 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 850
- 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 732
35 more
- 2021BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-Viewcited by 713
- 2023pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstructioncited by 684
- 2022BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detectioncited by 572
- 2024MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Imagescited by 566
- 2022PETR: Position Embedding Transformation for Multi-View 3D Object Detectioncited by 552
- 2023Gaussian Splatting SLAMcited by 415
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
- 2023SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAMcited by 388
- 2023DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenescited by 360
- 2022PETRv2: A Unified Framework for 3D Perception from Multi-Camera Imagescited by 352
- 2024DepthSplat: Connecting Gaussian Splatting and Depthcited by 286
- 2024gsplat: An Open-Source Library for Gaussian Splattingcited by 285
- 2023GS-SLAM: Dense Visual SLAM with 3D Gaussian Splattingcited by 278
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
- 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
- 2024Street Gaussians: Modeling Dynamic Urban Scenes with Gaussian Splattingcited by 221
- 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
- 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 174
- 2022M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representationcited by 156
- 2024HUGS: Holistic Urban 3D Scene Understanding via Gaussian Splattingcited by 142
- 2023Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Camerascited by 141
- 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
- 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
- 2024SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Drivingcited by 69
- 2023DiffBEV: Conditional Diffusion Model for Bird's Eye View Perceptioncited by 30
- 2022LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentationcited by 29
- 2023PointBeV: A Sparse Approach to BeV Predictionscited by 23
- 2024GaussianBeV: 3D Gaussian Representation meets Perception Models for BeV Segmentationcited by 20
- 2025Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splattingcited by 20
- 2024DifFUSER: Diffusion Model for Robust Multi-Sensor Fusion in 3D Object Detection and BEV Segmentationcited by 18
- 2025BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidancecited by 17
- 2024Improving Bird's Eye View Semantic Segmentation by Task Decompositioncited by 13
- 2023Parametric Depth Based Feature Representation Learning for Object Detection and Segmentation in Bird's Eye Viewcited by 10
- 2023UAP-BEV: Uncertainty Aware Planning using Bird's Eye View generated from Surround Monocular Imagescited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.