# Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review

arXiv: https://arxiv.org/abs/2303.01212 · HTML: https://mlsanity.com/paper/2303.01212/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Placed in End-to-end system architecture, one of 9 directions in End-to-end autonomous driving.
- Builds on 185 of its 366 references that are in the map.
- First posted on arXiv 2 Mar 2023.

## Placed in

- End-to-end autonomous driving / [End-to-end system architecture](https://mlsanity.com/driving/end-to-end-system-architecture/)

## Builds on (most cited first, 30 of 185)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)

## Built on by (most cited first, 25 of 25)

- [DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving](https://mlsanity.com/paper/2312.09245/) (2023)
- [GaussianOcc: Fully Self-supervised and Efficient 3D Occupancy Estimation with Gaussian Splatting](https://mlsanity.com/paper/2408.11447/) (2024)
- [UniWorld: Autonomous Driving Pre-training via World Models](https://mlsanity.com/paper/2308.07234/) (2023)
- [MUVO: A Multimodal Generative World Model for Autonomous Driving with Geometric Representations](https://mlsanity.com/paper/2311.11762/) (2023)
- [GaussianFlowOcc: Sparse and Weakly Supervised Occupancy Estimation using Gaussian Splatting and Temporal Flow](https://mlsanity.com/paper/2502.17288/) (2025)
- [LiDAR-based 4D Occupancy Completion and Forecasting](https://mlsanity.com/paper/2310.11239/) (2023)
- [Self-supervised Learning for Pre-Training 3D Point Clouds: A Survey](https://mlsanity.com/paper/2305.04691/) (2023)
- [UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2305.18829/) (2023)
- [SOGDet: Semantic-Occupancy Guided Multi-view 3D Object Detection](https://mlsanity.com/paper/2308.13794/) (2023)
- [PanoSSC: Exploring Monocular Panoptic 3D Scene Reconstruction for Autonomous Driving](https://mlsanity.com/paper/2406.07037/) (2024)
- [EFFOcc: Learning Efficient Occupancy Networks from Minimal Labels for Autonomous Driving](https://mlsanity.com/paper/2406.07042/) (2024)
- [LightOcc: Lightweight Spatial Embedding for Efficient Vision-based 3D Occupancy Prediction](https://mlsanity.com/paper/2412.05976/) (2024)
- [LEGO-Motion: Learning-Enhanced Grids with Occupancy Instance Modeling for Class-Agnostic Motion Prediction](https://mlsanity.com/paper/2503.07367/) (2025)
- [ShelfOcc: Native 3D Supervision beyond LiDAR for Vision-Based Occupancy Estimation](https://mlsanity.com/paper/2511.15396/) (2025)
- [TEOcc: Radar-camera Multi-modal Occupancy Prediction via Temporal Enhancement](https://mlsanity.com/paper/2410.11228/) (2024)
- [Inverse++: Vision-Centric 3D Semantic Occupancy Prediction Assisted with 3D Object Detection](https://mlsanity.com/paper/2504.04732/) (2025)
- [Ego-centric Predictive Model Conditioned on Hand Trajectories](https://mlsanity.com/paper/2508.19852/) (2025)
- [Domain Specific Benchmarks for Evaluating Multimodal Large Language Models](https://mlsanity.com/paper/2506.12958/) (2025)
- [An Overview about Emerging Technologies of Autonomous Driving](https://mlsanity.com/paper/2306.13302/) (2023)
- [RVFL-X: A Novel Randomized Network Based on Complex Transformed Real-Valued Tabular Datasets](https://mlsanity.com/paper/2510.06278/) (2025)
- [QueryOcc: Query-based Self-Supervision for 3D Semantic Occupancy](https://mlsanity.com/paper/2511.17221/) (2025)
- [Velocity Driven Vision: Asynchronous Sensor Fusion Birds Eye View Models for Autonomous Vehicles](https://mlsanity.com/paper/2407.16636/) (2024)
- [A Synthetic Benchmark for Collaborative 3D Semantic Occupancy Prediction in V2X-Enabled Autonomous Driving](https://mlsanity.com/paper/2506.17004/) (2025)
- [BDC-Occ: Binarized Deep Convolution Unit For Binarized Occupancy Network](https://mlsanity.com/paper/2405.17037/) (2024)
- [Camera Perspective Transformation to Bird's Eye View via Spatial Transformer Model for Road Intersection Monitoring](https://mlsanity.com/paper/2408.05577/) (2024)

## Abstract (arXiv)

Grid-centric perception is a crucial field for mobile robot perception and navigation. Nonetheless, grid-centric perception is less prevalent than object-centric perception as autonomous vehicles need to accurately perceive highly dynamic, large-scale traffic scenarios and the complexity and computational costs of grid-centric perception are high. In recent years, the rapid development of deep learning techniques and hardware provides fresh insights into the evolution of grid-centric perception. The fundamental difference between grid-centric and object-centric pipeline lies in that grid-centric perception follows a geometry-first paradigm which is more robust to the open-world driving scenarios with endless long-tailed semantically-unknown obstacles. Recent researches demonstrate the great advantages of grid-centric perception, such as comprehensive fine-grained environmental representation, greater robustness to occlusion and irregular shaped objects, better ground estimation, and safer planning policies. There is also a growing trend that the capacity of occupancy networks are greatly expanded to 4D scene perception and prediction and latest techniques are highly related to new research topics such as 4D occupancy forecasting, generative AI and world models in the field of autonomous driving. Given the lack of current surveys for this rapidly expanding field, we present a hierarchically-structured review of grid-centric perception for autonomous vehicles. We organize previous and current knowledge of occupancy grid techniques along the main vein from 2D BEV grids to 3D occupancy to 4D occupancy forecasting. We additionally summarize label-efficient occupancy learning and the role of grid-centric perception in driving systems. Lastly, we present a summary of the current research trend and provide future outlooks.