mlsanityData to 7 Oct 2026

Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Images

arXiv 2110.01997 ↗First posted 5 Oct 2021cs.CVcited by 98

Not on the map. Why?

Abstract (arXiv)

Autonomous navigation requires structured representation of the road network and instance-wise identification of the other traffic agents. Since the traffic scene is defined on the ground plane, this corresponds to scene understanding in the bird's-eye-view (BEV). However, the onboard cameras of autonomous cars are customarily mounted horizontally for a better view of the surrounding, making this task very challenging. In this work, we study the problem of extracting a directed graph representing the local road network in BEV coordinates, from a single onboard camera image. Moreover, we show that the method can be extended to detect dynamic objects on the BEV plane. The semantics, locations, and orientations of the detected objects together with the road graph facilitates a comprehensive understanding of the scene. Such understanding becomes fundamental for the downstream tasks, such as path planning and navigation. We validate our approach against powerful baselines and show that our network achieves superior performance. We also demonstrate the effects of various design choices through ablation studies. Code: https://github.com/ybarancan/STSU

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Builds on 33 of its 43 references that are in the map. First posted on arXiv 5 Oct 2021.

Builds on

33 of 43 references.

References · 33

  1. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  2. 2020End-to-End Object Detection with Transformerscited by 7,384
  3. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  4. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  5. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  6. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  7. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  8. 2019Learning by Cheatingcited by 361
  9. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  10. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  11. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  12. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
21 more
  1. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  2. 2019Learning Lightweight Lane Detection CNNs by Self Attention Distillationcited by 162
  3. 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 119
  4. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  5. 2018Monocular Semantic Occupancy Grid Mapping with Convolutional Variational Encoder-Decoder Networkscited by 84
  6. 2020HDNET: Exploiting HD Maps for 3D Object Detectioncited by 78
  7. 2020Key Points Estimation and Point Instance Segmentation Approach for Lane Detectioncited by 77
  8. 2021Semantics for Robotic Mapping, Perception and Interaction: A Surveycited by 56
  9. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55
  10. 2020MonoLayout: Amodal scene layout from a single imagecited by 49
  11. 2020DAGMapper: Learning to Map by Discovering Lane Topologycited by 49
  12. 2020Hierarchical Recurrent Attention Networks for Structured Online Mapscited by 38
  13. 2020Convolutional Recurrent Network for Road Boundary Extractioncited by 33
  14. 2019Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localizationcited by 24
  15. 2019Leveraging Crowdsourced GPS Data for Road Extraction from Aerial Imagerycited by 23
  16. 2018Real-time Dynamic Object Detection for Autonomous Driving using Prior 3D-Mapscited by 17
  17. 2021Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Predictioncited by 12
  18. 2018Iterative Deep Learning for Road Topology Extractioncited by 9
  19. 2020End-to-End Deep Structured Models for Drawing Crosswalkscited by 9
  20. 2020Action Sequence Predictions of Vehicles in Urban Environments using Map and Social Contextcited by 8
  21. 2021TridentNet: A Conditional Generative Model for Dynamic Trajectory Generationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 17; 2023: 29; 2024: 25; 2025: 20; 2026: 5.

98 papers.

Sort:

By area:

  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2022MapTR: Structured Modeling and Learning for Online Vectorized HD Map Constructioncited by 311
  3. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  4. 2023MapTRv2: An End-to-End Framework for Online Vectorized HD Map Constructioncited by 192
  5. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  6. 2022PolarFormer: Multi-camera 3D Object Detection with Polar Transformercited by 147
  7. 2022PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmarkcited by 140
  8. 2022Vision-Centric BEV Perception: A Surveycited by 126
  9. 2022BEVSegFormer: Bird's Eye View Semantic Segmentation From Arbitrary Camera Rigscited by 123
  10. 2022Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipecited by 122
  11. 2023OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mappingcited by 83
  12. 2023End-to-End Vectorized HD-map Construction with Piecewise Bezier Curvecited by 80
38 more
  1. 2023LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Drivingcited by 57
  2. 2022Uncertainty-aware Vision-based Metric Cross-view Geolocalizationcited by 54
  3. 2023Graph-based Topology Reasoning for Driving Scenescited by 53
  4. 2024MGMap: Mask-Guided Learning for Online Vectorized HD Map Constructioncited by 52
  5. 2023Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Constructioncited by 50
  6. 2023Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix Itcited by 44
  7. 2023DistillBEV: Boosting Multi-Camera 3D Object Detection with Cross-Modal Knowledge Distillationcited by 41
  8. 2023TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoningcited by 41
  9. 2022CurveFormer: 3D Lane Detection by Curve Propagation with Curve Queries and Attentioncited by 40
  10. 2024T2SG: Traffic Topology Scene Graph for Topology Reasoning in Autonomous Drivingcited by 35
  11. 2021Topology Preserving Local Road Network Estimation from Single Onboard Camera Imagecited by 34
  12. 2024Producing and Leveraging Online Map Uncertainty in Trajectory Predictioncited by 32
  13. 2024TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving Scenescited by 29
  14. 2024Enhancing Vectorized Map Perception with Historical Rasterized Mapscited by 29
  15. 2022RNGDet: Road Network Graph Detection by Transformer in Aerial Imagescited by 26
  16. 2023Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Reviewcited by 25
  17. 2022RNGDet++: Road Network Graph Detection by Transformer with Instance Segmentation and Multi-scale Features Enhancementcited by 24
  18. 2022Vision-based Uneven BEV Representation Learning with Polar Rasterization and Surface Estimationcited by 23
  19. 2023Towards Motion Forecasting with Real-World Perception Inputs: Are End-to-End Approaches Competitive?cited by 22
  20. 2024Enhancing 3D Lane Detection and Topology Reasoning with 2D Lane Priorscited by 21
  21. 2023Learning and Aggregating Lane Graphs for Urban Automated Drivingcited by 20
  22. 2024RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulationcited by 20
  23. 2023InsMapper: Exploring Inner-instance Information for Vectorized HD Mappingcited by 19
  24. 2024RoadPainter: Points Are Ideal Navigators for Topology transformERcited by 17
  25. 2022CenterLineDet: CenterLine Graph Detection for Road Lanes with Vehicle-mounted Sensors by Transformer for HD Map Generationcited by 15
  26. 2021csBoundary: City-scale Road-boundary Detection in Aerial Images for High-definition Mapscited by 12
  27. 2025SMART: Advancing Scalable Map Priors for Driving Topology Reasoningcited by 12
  28. 2023NeMO: Neural Map Growing System for Spatiotemporal Fusion in Bird's-Eye-View and BDD-Map Benchmarkcited by 11
  29. 2024DisC-GS: Discontinuity-aware Gaussian Splattingcited by 11
  30. 2024Enhancing Online Road Network Perception and Reasoning with Standard Definition Mapscited by 11
  31. 2023OCBEV: Object-Centric BEV Transformer for Multi-View 3D Object Detectioncited by 10
  32. 2024DTCLMapper: Dual Temporal Consistent Learning for Vectorized HD Map Constructioncited by 10
  33. 2025SeqGrowGraph: Learning Lane Topology as a Chain of Graph Expansionscited by 10
  34. 2025SEPT: Standard-Definition Map Enhanced Scene Perception and Topology Reasoning for Autonomous Drivingcited by 9
  35. 2025Reusing Attention for One-stage Lane Topology Understandingcited by 9
  36. 2023DualCross: Cross-Modality Cross-Domain Adaptation for Monocular BEV Perceptioncited by 8
  37. 2024Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attentioncited by 8
  38. 2023Improving Online Lane Graph Extraction by Object-Lane Clusteringcited by 7

50 most cited of 98 All 98 (JSON)