mlsanityData to 7 Oct 2026

Online Lane Graph Extraction from Onboard Video

arXiv 2304.00930 ↗First posted 3 Apr 2023cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Autonomous driving requires a structured understanding of the surrounding road network to navigate. One of the most common and useful representation of such an understanding is done in the form of BEV lane graphs. In this work, we use the video stream from an onboard camera for online extraction of the surrounding's lane graph. Using video, instead of a single image, as input poses both benefits and challenges in terms of combining the information from different timesteps. We study the emerged challenges using three different approaches. The first approach is a post-processing step that is capable of merging single frame lane graph estimates into a unified lane graph. The second approach uses the spatialtemporal embeddings in the transformer to enable the network to discover the best temporal aggregation strategy. Finally, the third, and the proposed method, is an early temporal aggregation through explicit BEV projection and alignment of framewise features. A single model of this proposed simple, yet effective, method can process any number of images, including one, to produce accurate lane graphs. The experiments on the Nuscenes and Argoverse datasets show the validity of all the approaches while highlighting the superiority of the proposed method. The code will be made public.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 43 of its 52 references that are in the map. First posted on arXiv 3 Apr 2023.

Builds on

43 of 52 references.

References · 43

  1. 2020End-to-End Object Detection with Transformerscited by 7,384
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  3. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
  4. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,210
  5. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  6. 2019Argoverse: 3D Tracking and Forecasting with Rich Mapscited by 728
  7. 2018ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worstcited by 391
  8. 2019Learning by Cheatingcited by 361
  9. 2018Multimodal Trajectory Predictions for Autonomous Driving using Deep Convolutional Networkscited by 289
  10. 2021HDMapNet: An Online HD Map Construction and Evaluation Frameworkcited by 280
  11. 2022Cross-view Transformers for real-time Map-view Semantic Segmentationcited by 264
  12. 2020Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Predictioncited by 218
31 more
  1. 2021MP3: A Unified Model to Map, Perceive, Predict and Plancited by 211
  2. 2020Predicting Semantic Map Representations from Images using Pyramid Occupancy Networkscited by 175
  3. 2019Cross-view Semantic Segmentation for Sensing Surroundingscited by 173
  4. 2021IntentNet: Learning to Predict Intention from Raw Sensor Datacited by 158
  5. 2018Towards End-to-End Lane Detection: an Instance Segmentation Approachcited by 129
  6. 2018Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++cited by 119
  7. 2019Rules of the Road: Predicting Driving Behavior with a Convolutional Model of Semantic Interactionscited by 117
  8. 20183D-LaneNet: End-to-End 3D Multiple Lane Detectioncited by 106
  9. 2021SSTVOS: Sparse Spatiotemporal Transformers for Video Object Segmentationcited by 106
  10. 2020MotionNet: Joint Perception and Motion Prediction for Autonomous Driving Based on Bird's Eye View Mapscited by 100
  11. 2021Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Imagescited by 98
  12. 2020HDNET: Exploiting HD Maps for 3D Object Detectioncited by 78
  13. 2020Key Points Estimation and Point Instance Segmentation Approach for Lane Detectioncited by 77
  14. 2020FISHING Net: Future Inference of Semantic Heatmaps In Gridscited by 55
  15. 2020DAGMapper: Learning to Map by Discovering Lane Topologycited by 49
  16. 2021Long-Range Transformers for Dynamic Spatiotemporal Forecastingcited by 46
  17. 2020Hierarchical Recurrent Attention Networks for Structured Online Mapscited by 38
  18. 2019End-to-end Lane Detection through Differentiable Least-Squares Fittingcited by 36
  19. 2021Topology Preserving Local Road Network Estimation from Single Onboard Camera Imagecited by 34
  20. 2020Convolutional Recurrent Network for Road Boundary Extractioncited by 33
  21. 20203D-LaneNet+: Anchor Free Lane Detection using a Semi-Local Representationcited by 32
  22. 2020Understanding Bird's-Eye View of Road Semantics using an Onboard Cameracited by 29
  23. 2019Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localizationcited by 24
  24. 2019Leveraging Crowdsourced GPS Data for Road Extraction from Aerial Imagerycited by 23
  25. 2018Real-time Dynamic Object Detection for Autonomous Driving using Prior 3D-Mapscited by 17
  26. 2020Probabilistic Semantic Mapping for Urban Autonomous Driving Applicationscited by 15
  27. 2021Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Predictioncited by 12
  28. 2018Iterative Deep Learning for Road Topology Extractioncited by 9
  29. 2020End-to-End Deep Structured Models for Drawing Crosswalkscited by 9
  30. 2020Action Sequence Predictions of Vehicles in Urban Environments using Map and Social Contextcited by 8
  31. 2021TridentNet: A Conditional Generative Model for Dynamic Trajectory Generationcited by 5

Built on by

Not cited yet.