# GitNet: Geometric Prior-based Transformation for Birds-Eye-View Segmentation

arXiv: https://arxiv.org/abs/2204.07733 · HTML: https://mlsanity.com/paper/2204.07733/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 16 Apr 2022.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Categorical Depth Distribution Network for Monocular 3D Object Detection](https://mlsanity.com/paper/2103.01100/) (2021)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [Orthographic Feature Transform for Monocular 3D Object Detection](https://mlsanity.com/paper/1811.08188/) (2018)
- [Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks](https://mlsanity.com/paper/2003.13402/) (2020)
- [Cross-view Semantic Segmentation for Sensing Surroundings](https://mlsanity.com/paper/1906.03560/) (2019)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Translating Images into Maps](https://mlsanity.com/paper/2110.00966/) (2021)
- [Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks](https://mlsanity.com/paper/1804.02176/) (2018)
- [A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye View](https://mlsanity.com/paper/2005.04078/) (2020)
- [MonoLayout: Amodal scene layout from a single image](https://mlsanity.com/paper/2002.08394/) (2020)
- [Learning to Look around Objects for Top-View Representations of Outdoor Scenes](https://mlsanity.com/paper/1803.10870/) (2018)
- [Generative Adversarial Frontal View to Bird View Synthesis](https://mlsanity.com/paper/1808.00327/) (2018)

## Built on by (most cited first, 24 of 24)

- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Street-View Image Generation from a Bird's-Eye View Layout](https://mlsanity.com/paper/2301.04634/) (2023)
- [CAPE: Camera View Position Embedding for Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.10209/) (2023)
- [DiffBEV: Conditional Diffusion Model for Bird's Eye View Perception](https://mlsanity.com/paper/2303.08333/) (2023)
- [LaRa: Latents and Rays for Multi-Camera Bird's-Eye-View Semantic Segmentation](https://mlsanity.com/paper/2206.13294/) (2022)
- [Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving](https://mlsanity.com/paper/2305.04205/) (2023)
- [Improving Bird's Eye View Semantic Segmentation by Task Decomposition](https://mlsanity.com/paper/2404.01925/) (2024)
- [F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving](https://mlsanity.com/paper/2303.03651/) (2023)
- [NeMO: Neural Map Growing System for Spatiotemporal Fusion in Bird's-Eye-View and BDD-Map Benchmark](https://mlsanity.com/paper/2306.04540/) (2023)
- [Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentation](https://mlsanity.com/paper/2308.14525/) (2023)
- [A Cross-Scale Hierarchical Transformer with Correspondence-Augmented Attention for inferring Bird's-Eye-View Semantic Segmentation](https://mlsanity.com/paper/2304.03650/) (2023)
- [Disentangling Instance and Scene Contexts for 3D Semantic Scene Completion](https://mlsanity.com/paper/2507.08555/) (2025)
- [U-ViLAR: Uncertainty-Aware Visual Localization for Autonomous Driving via Differentiable Association and Registration](https://mlsanity.com/paper/2507.04503/) (2025)
- [Monocular BEV Perception of Road Scenes via Front-to-Top View Projection](https://mlsanity.com/paper/2211.08144/) (2022)
- [Multi-camera Bird's Eye View Perception for Autonomous Driving](https://mlsanity.com/paper/2309.09080/) (2023)
- [Focus on BEV: Self-calibrated Cycle View Transformation for Monocular Birds-Eye-View Segmentation](https://mlsanity.com/paper/2410.15932/) (2024)
- [VQ-Map: Bird's-Eye-View Map Layout Estimation in Tokenized Discrete Space via Vector Quantization](https://mlsanity.com/paper/2411.01618/) (2024)
- [nuScenes Revisited: Progress and Challenges in Autonomous Driving](https://mlsanity.com/paper/2512.02448/) (2025)
- [DeepSTEP -- Deep Learning-Based Spatio-Temporal End-To-End Perception for Autonomous Vehicles](https://mlsanity.com/paper/2305.06820/) (2023)
- [TopView: Vectorising road users in a bird's eye view from uncalibrated street-level imagery with deep learning](https://mlsanity.com/paper/2412.16229/) (2024)
- [SimBEV: A Synthetic Multi-Task Multi-Sensor Driving Data Generation Tool and Dataset](https://mlsanity.com/paper/2502.01894/) (2025)
- [Zero-BEV: Zero-shot Projection of Any First-Person Modality to BEV Maps](https://mlsanity.com/paper/2402.13848/) (2024)
- [CVCP-Fusion: On Implicit Depth Estimation for 3D Bounding Box Prediction](https://mlsanity.com/paper/2410.11211/) (2024)

## Abstract (arXiv)

Birds-eye-view (BEV) semantic segmentation is critical for autonomous driving for its powerful spatial representation ability. It is challenging to estimate the BEV semantic maps from monocular images due to the spatial gap, since it is implicitly required to realize both the perspective-to-BEV transformation and segmentation. We present a novel two-stage Geometry Prior-based Transformation framework named GitNet, consisting of (i) the geometry-guided pre-alignment and (ii) ray-based transformer. In the first stage, we decouple the BEV segmentation into the perspective image segmentation and geometric prior-based mapping, with explicit supervision by projecting the BEV semantic labels onto the image plane to learn visibility-aware features and learnable geometry to translate into BEV space. Second, the pre-aligned coarse BEV features are further deformed by ray-based transformers to take visibility knowledge into account. GitNet achieves the leading performance on the challenging nuScenes and Argoverse Datasets.