mlsanityData to 7 Oct 2026

RendBEV: Semantic Novel View Synthesis for Self-Supervised Bird's Eye View Segmentation

arXiv 2502.14792 ↗First posted 20 Feb 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Bird's Eye View (BEV) semantic maps have recently garnered a lot of attention as a useful representation of the environment to tackle assisted and autonomous driving tasks. However, most of the existing work focuses on the fully supervised setting, training networks on large annotated datasets. In this work, we present RendBEV, a new method for the self-supervised training of BEV semantic segmentation networks, leveraging differentiable volumetric rendering to receive supervision from semantic perspective views computed by a 2D semantic segmentation model. Our method enables zero-shot BEV semantic segmentation, and already delivers competitive results in this challenging setting. When used as pretraining to then fine-tune on labeled BEV ground-truth, our method significantly boosts performance in low-annotation regimes, and sets a new state of the art when fine-tuning on all available labels.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

22 of 29 references.

References · 22

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  2. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  5. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  6. 2021KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3Dcited by 570
  7. 2019Panoptic-DeepLabcited by 362
  8. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  9. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  10. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  11. 2022Panoptic Lifting for 3D Scene Understanding with Neural Fieldscited by 171
  12. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
10 more
  1. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  2. 2021Translating Images into Mapscited by 115
  3. 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
  4. 2020A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye Viewcited by 71
  5. 2023Behind the Scenes: Density Fields for Single View Reconstructioncited by 63
  6. 2021Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Imagescited by 48
  7. 2023SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Imagescited by 35
  8. 2023Towards Domain Generalization for Multi-view 3D Object Detection in Bird-Eye-Viewcited by 27
  9. 2023S4C: Self-Supervised Semantic Scene Completion with Neural Fieldscited by 27
  10. 2024OccFeat: Self-supervised Occupancy Feature Prediction for Pretraining BEV Segmentation Networkscited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025MapRF: Weakly Supervised Online HD Map Construction via NeRF-Guided Self-Trainingcited by 2
  2. 2026Faster Training, Fewer Labels: Self-Supervised Pretraining for Fine-Grained BEV Segmentationcited by 0
  3. 2026CycleBEV: Regularizing View Transformation Networks via View Cycle Consistency for Bird's-Eye-View Semantic Segmentationcited by 0