# PointBeV: A Sparse Approach to BeV Predictions

arXiv: https://arxiv.org/abs/2312.00703 · HTML: https://mlsanity.com/paper/2312.00703/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 57 references that are in the map.
- First posted on arXiv 1 Dec 2023.

## Builds on (most cited first, 30 of 46)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Tri-Perspective View for Vision-Based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2302.07817/) (2023)
- [ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning](https://mlsanity.com/paper/2207.07601/) (2022)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving](https://mlsanity.com/paper/2205.09743/) (2022)
- [Time Will Tell: New Outlooks and A Baseline for Temporal Multi-View 3D Object Detection](https://mlsanity.com/paper/2210.02443/) (2022)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [M2BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation](https://mlsanity.com/paper/2204.05088/) (2022)
- [Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?](https://mlsanity.com/paper/2206.07959/) (2022)
- [ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries](https://mlsanity.com/paper/2208.01582/) (2022)
- [FusionAD: Multi-modality Fusion for Prediction and Planning Tasks of Autonomous Driving](https://mlsanity.com/paper/2308.01006/) (2023)
- [Exploring Recurrent Long-term Temporal Fusion for Multi-view 3D Perception](https://mlsanity.com/paper/2303.05970/) (2023)
- [Occupancy Flow Fields for Motion Forecasting in Autonomous Driving](https://mlsanity.com/paper/2203.03875/) (2022)
- [PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Prediction](https://mlsanity.com/paper/2308.16896/) (2023)
- [StretchBEV: Stretching Future Instance Prediction Spatially and Temporally](https://mlsanity.com/paper/2203.13641/) (2022)

## Built on by (most cited first, 23 of 23)

- [JAFAR: Jack up Any Feature at Any Resolution](https://mlsanity.com/paper/2506.11136/) (2025)
- [GaussianBeV: 3D Gaussian Representation meets Perception Models for BeV Segmentation](https://mlsanity.com/paper/2407.14108/) (2024)
- [Toward Real-world BEV Perception: Depth Uncertainty Estimation via Gaussian Splatting](https://mlsanity.com/paper/2504.01957/) (2025)
- [GaussRender: Learning 3D Occupancy with Gaussian Rendering](https://mlsanity.com/paper/2502.05040/) (2025)
- [AutoOcc: Automatic Open-Ended Semantic Occupancy Annotation via Vision-Language Guided Gaussian Splatting](https://mlsanity.com/paper/2502.04981/) (2025)
- [Robust Bird's Eye View Segmentation by Adapting DINOv2](https://mlsanity.com/paper/2409.10228/) (2024)
- [TempBEV: Improving Learned BEV Encoders with Combined Image and BEV Space Temporal Aggregation](https://mlsanity.com/paper/2404.11803/) (2024)
- [Dur360BEV: A Real-world 360-degree Single Camera Dataset and Benchmark for Bird-Eye View Mapping in Autonomous Driving](https://mlsanity.com/paper/2503.00675/) (2025)
- [OE-BevSeg: An Object Informed and Environment Aware Multimodal Framework for Bird's-eye-view Vehicle Semantic Segmentation](https://mlsanity.com/paper/2407.13137/) (2024)
- [Mapping like a Skeptic: Probabilistic BEV Projection for Online HD Mapping](https://mlsanity.com/paper/2508.21689/) (2025)
- [ShelfGaussian: Shelf-Supervised Open-Vocabulary Gaussian-based 3D Scene Understanding](https://mlsanity.com/paper/2512.03370/) (2025)
- [Predictive Uncertainty Quantification for Bird's Eye View Segmentation: A Benchmark and Novel Loss Function](https://mlsanity.com/paper/2405.20986/) (2024)
- [KD360-VoxelBEV: LiDAR and 360-degree Camera Cross Modality Knowledge Distillation for Bird's-Eye-View Segmentation](https://mlsanity.com/paper/2512.15311/) (2025)
- [Reconstruction Matters: Learning Geometry-Aligned BEV Representation through 3D Gaussian Splatting](https://mlsanity.com/paper/2603.19193/) (2026)
- [BEVPredFormer: Spatio-temporal Attention for BEV Instance Prediction in Autonomous Driving](https://mlsanity.com/paper/2604.02930/) (2026)
- [SUG-Occ: Explicit Semantics and Uncertainty Guided Sparse Learning for Efficient 3D Occupancy Prediction](https://mlsanity.com/paper/2601.11396/) (2026)
- [Faster Training, Fewer Labels: Self-Supervised Pretraining for Fine-Grained BEV Segmentation](https://mlsanity.com/paper/2602.18066/) (2026)
- [CycleBEV: Regularizing View Transformation Networks via View Cycle Consistency for Bird's-Eye-View Semantic Segmentation](https://mlsanity.com/paper/2602.23575/) (2026)
- [SToRe3D: Sparse Token Relevance in ViTs for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2605.14110/) (2026)
- [Open-Vocabulary BEV Segmentation with 3D-Aware Geometric Constraints](https://mlsanity.com/paper/2606.24353/) (2026)
- [TGRIP: A Text-Guided Approach to Vehicle Instance Prediction in Autonomous Driving](https://mlsanity.com/paper/2607.04812/) (2026)
- [FlashBEV: Fast and Memory-Efficient Exact BEV Transformation with IO-Awareness](https://mlsanity.com/paper/2607.10071/) (2026)
- [CoDS: Robust Collaborative Perception via Expert-driven Detection and BEV Segmentation](https://mlsanity.com/paper/2608.14085/) (2026)

## Abstract (arXiv)

Bird's-eye View (BeV) representations have emerged as the de-facto shared space in driving applications, offering a unified space for sensor data fusion and supporting various downstream tasks. However, conventional models use grids with fixed resolution and range and face computational inefficiencies due to the uniform allocation of resources across all cells. To address this, we propose PointBeV, a novel sparse BeV segmentation model operating on sparse BeV cells instead of dense grids. This approach offers precise control over memory usage, enabling the use of long temporal contexts and accommodating memory-constrained platforms. PointBeV employs an efficient two-pass strategy for training, enabling focused computation on regions of interest. At inference time, it can be used with various memory/performance trade-offs and flexibly adjusts to new specific use cases. PointBeV achieves state-of-the-art results on the nuScenes dataset for vehicle, pedestrian, and lane segmentation, showcasing superior performance in static and temporal settings despite being trained solely with sparse signals. We will release our code along with two new efficient modules used in the architecture: Sparse Feature Pulling, designed for the effective extraction of features from images to BeV, and Submanifold Attention, which enables efficient temporal modeling. Our code is available at https://github.com/valeoai/PointBeV.