mlsanityData to 7 Oct 2026

Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networks

arXiv 1804.02176 ↗First posted 6 Apr 2018cs.RO cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

In this work, we research and evaluate end-to-end learning of monocular semantic-metric occupancy grid mapping from weak binocular ground truth. The network learns to predict four classes, as well as a camera to bird's eye view mapping. At the core, it utilizes a variational encoder-decoder network that encodes the front-view visual information of the driving scene and subsequently decodes it into a 2-D top-view Cartesian coordinate system. The evaluations on Cityscapes show that the end-to-end learning of semantic-metric occupancy grids outperforms the deterministic mapping approach with flat-plane assumption by more than 12% mean IoU. Furthermore, we show that the variational sampling with a relatively small embedding vector brings robustness against vehicle dynamic perturbations, and generalizability for unseen KITTI data. Our network achieves real-time inference rates of approx. 35 Hz for an input image with a resolution of 256x512 pixels and an output map with 64x64 occupancy grid cells using a Titan V GPU.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 20; 2023: 29; 2024: 14; 2025: 4; 2026: 5.

84 papers.

Sort:

By area:

  1. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 365
  2. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  3. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  4. 2021FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Camerascited by 239
  5. 2022Vision-Centric BEV Perception: A Surveycited by 126
  6. 2021Translating Images into Mapscited by 115
  7. 2021Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Imagescited by 98
  8. 2023COTR: Compact Occupancy TRansformer for Vision-based 3D Occupancy Predictioncited by 94
  9. 2023PivotNet: Vectorized Pivot Learning for End-to-end HD Map Constructioncited by 84
  10. 2023Neural Map Prior for Autonomous Drivingcited by 68
  11. 2021Bird's-Eye-View Panoptic Segmentation Using Monocular Frontal View Imagescited by 48
  12. 2022Learning Ego 3D Representation as Ray Tracingcited by 43
38 more
  1. 2023Online Vectorized HD Map Construction using Geometrycited by 42
  2. 2023InstaGraM: Instance-level Graph Modeling for Vectorized HD Map Learningcited by 40
  3. 2022SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generationcited by 39
  4. 2023OccNeRF: Advancing 3D Occupancy Prediction in LiDAR-Free Environmentscited by 39
  5. 2022X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentationcited by 38
  6. 2023EA-LSS: Edge-aware Lift-splat-shot Framework for 3D BEV Object Detectioncited by 38
  7. 2023SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Imagescited by 35
  8. 2023DiffBEV: Conditional Diffusion Model for Bird's Eye View Perceptioncited by 30
  9. 2023TBP-Former: Learning Temporal Bird's-Eye-View Pyramid for Joint Perception and Prediction in Vision-Centric Autonomous Drivingcited by 30
  10. 2023RoadFormer: Duplex Transformer for RGB-Normal Semantic Road Scene Parsingcited by 27
  11. 2024BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentationcited by 27
  12. 2021Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillationcited by 25
  13. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  14. 2022GitNet: Geometric Prior-based Transformation for Birds-Eye-View Segmentationcited by 24
  15. 2022Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmarkcited by 24
  16. 2022Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimationcited by 23
  17. 2021Learning by Watchingcited by 22
  18. 2021MASS: Multi-Attentional Semantic Segmentation of LiDAR Data for Dense Top-View Understandingcited by 22
  19. 2025VLM-E2E: Enhancing End-to-End Autonomous Driving with Multimodal Driver Attention Fusioncited by 20
  20. 2023LiDAR2Map: In Defense of LiDAR-Based Semantic Map Construction Using Online Camera Distillationcited by 18
  21. 2023Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Drivingcited by 13
  22. 2024Improving Bird's Eye View Semantic Segmentation by Task Decompositioncited by 13
  23. 2023F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Drivingcited by 12
  24. 2024Autonomous Driving with Spiking Neural Networkscited by 12
  25. 2022HFT: Lifting Perspective Representations via Hybrid Feature Transformationcited by 11
  26. 2022JPerceiver: Joint Perception Network for Depth, Pose and Layout Estimation in Driving Scenescited by 11
  27. 2023NeMO: Neural Map Growing System for Spatiotemporal Fusion in Bird's-Eye-View and BDD-Map Benchmarkcited by 11
  28. 2022CrossDTR: Cross-view and Depth-guided Transformers for 3D Object Detectioncited by 10
  29. 2023Generating Evidential BEV Maps in Continuous Driving Spacecited by 10
  30. 2024DTCLMapper: Dual Temporal Consistent Learning for Vectorized HD Map Constructioncited by 10
  31. 2023Rethinking Amodal Video Segmentation from Learning Supervised Signals with Object-centric Representationcited by 9
  32. 2023BEV-CV: Birds-Eye-View Transform for Cross-View Geo-Localisationcited by 9
  33. 2024RoadFormer+: Delivering RGB-X Scene Parsing through Scale-Aware Information Decoupling and Advanced Heterogeneous Feature Fusioncited by 9
  34. 2023Semi-Supervised Learning for Visual Bird's Eye View Semantic Segmentationcited by 8
  35. 2022ViT-BEVSeg: A Hierarchical Transformer Network for Monocular Birds-Eye-View Segmentationcited by 7
  36. 2023A Cross-Scale Hierarchical Transformer with Correspondence-Augmented Attention for inferring Bird's-Eye-View Semantic Segmentationcited by 7
  37. 2024Improving Hierarchical Representations of Vectorized HD Maps with Perspective Cluescited by 6
  38. 2024Map It Anywhere (MIA): Empowering Bird's Eye View Mapping using Large-scale Public Datacited by 5

50 most cited of 84 All 84 (JSON)