# LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving

arXiv: https://arxiv.org/abs/2312.16108 · HTML: https://mlsanity.com/paper/2312.16108/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 29 of its 35 references that are in the map.
- First posted on arXiv 26 Dec 2023.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [Learning Lane Graph Representations for Motion Forecasting](https://mlsanity.com/paper/2007.13732/) (2020)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [HDMapNet: An Online HD Map Construction and Evaluation Framework](https://mlsanity.com/paper/2107.06307/) (2021)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Construction](https://mlsanity.com/paper/2308.12570/) (2023)
- [PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark](https://mlsanity.com/paper/2203.11089/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Structured Bird's-Eye-View Traffic Scene Understanding from Onboard Images](https://mlsanity.com/paper/2110.01997/) (2021)
- [PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction](https://mlsanity.com/paper/2308.16477/) (2023)
- [Graph-based Topology Reasoning for Driving Scenes](https://mlsanity.com/paper/2304.05277/) (2023)
- [Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction](https://mlsanity.com/paper/2303.08815/) (2023)
- [Anchor3DLane: Learning to Regress 3D Anchors for Monocular 3D Lane Detection](https://mlsanity.com/paper/2301.02371/) (2023)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [Topology Preserving Local Road Network Estimation from Single Onboard Camera Image](https://mlsanity.com/paper/2112.10155/) (2021)
- [Open-sourced Data Ecosystem in Autonomous Driving: the Present and Future](https://mlsanity.com/paper/2312.03408/) (2023)
- [Sparse Dense Fusion for 3D Object Detection](https://mlsanity.com/paper/2304.04179/) (2023)

## Built on by (most cited first, 30 of 57)

- [DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.07656/) (2025)
- [Fully Sparse 3D Occupancy Prediction](https://mlsanity.com/paper/2312.17118/) (2023)
- [TopoLogic: An Interpretable Pipeline for Lane Topology Reasoning on Driving Scenes](https://mlsanity.com/paper/2405.14747/) (2024)
- [Enhancing Vectorized Map Perception with Historical Rasterized Maps](https://mlsanity.com/paper/2409.00620/) (2024)
- [SLEDGE: Synthesizing Driving Environments with Generative Models and Rule-Based Traffic](https://mlsanity.com/paper/2403.17933/) (2024)
- [Chameleon: Fast-slow Neuro-symbolic Lane Topology Extraction](https://mlsanity.com/paper/2503.07485/) (2025)
- [CountFormer: Multi-View Crowd Counting Transformer](https://mlsanity.com/paper/2407.02047/) (2024)
- [PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2407.17378/) (2024)
- [PriorMapNet: Enhancing Online Vectorized HD Map Construction with Priors](https://mlsanity.com/paper/2408.08802/) (2024)
- [MGMapNet: Multi-Granularity Representation Learning for End-to-End Vectorized HD Map Construction](https://mlsanity.com/paper/2410.07733/) (2024)
- [Topo2Seq: Enhanced Topology Reasoning via Topology Sequence Learning](https://mlsanity.com/paper/2502.08974/) (2025)
- [SEPT: Standard-Definition Map Enhanced Scene Perception and Topology Reasoning for Autonomous Driving](https://mlsanity.com/paper/2505.12246/) (2025)
- [TopoPoint: Enhance Topology Reasoning via Endpoint Detection in Autonomous Driving](https://mlsanity.com/paper/2505.17771/) (2025)
- [Anchor3DLane++: 3D Lane Detection via Sample-Adaptive Sparse 3D Anchor Regression](https://mlsanity.com/paper/2412.16889/) (2024)
- [InteractionMap: Improving Online Vectorized HDMap Construction with Interaction](https://mlsanity.com/paper/2503.21659/) (2025)
- [TopoMaskV2: Enhanced Instance-Mask-Based Formulation for the Road Topology Problem](https://mlsanity.com/paper/2409.11325/) (2024)
- [Delving into Mapping Uncertainty for Mapless Trajectory Prediction](https://mlsanity.com/paper/2507.18498/) (2025)
- [DHGS: Decoupled Hybrid Gaussian Splatting for Driving Scene](https://mlsanity.com/paper/2407.16600/) (2024)
- [Enhancing Lane Segment Perception and Topology Reasoning with Crowdsourcing Trajectory Priors](https://mlsanity.com/paper/2411.17161/) (2024)
- [TopoBDA: Towards Bezier Deformable Attention for Road Topology Understanding](https://mlsanity.com/paper/2412.18951/) (2024)
- [RTMap: Real-Time Recursive Mapping with Change Detection and Localization](https://mlsanity.com/paper/2507.00980/) (2025)
- [PseudoMapTrainer: Learning Online Mapping without HD Maps](https://mlsanity.com/paper/2508.18788/) (2025)
- [Spatial Retrieval Augmented Autonomous Driving](https://mlsanity.com/paper/2512.06865/) (2025)
- [EAN-MapNet: Efficient Vectorized HD Map Construction with Anchor Neighborhoods](https://mlsanity.com/paper/2402.18278/) (2024)
- [MapVision: CVPR 2024 Autonomous Grand Challenge Mapless Driving Tech Report](https://mlsanity.com/paper/2406.10125/) (2024)
- [Monocular Lane Detection Based on Deep Learning: A Survey](https://mlsanity.com/paper/2411.16316/) (2024)
- [Uni-PrevPredMap: Extending PrevPredMap to a Unified Framework of Prior-Informed Modeling for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2504.06647/) (2025)
- [Decoupling Scene Perception and Ego Status: A Multi-Context Fusion Approach for Enhanced Generalization in End-to-End Autonomous Driving](https://mlsanity.com/paper/2511.13079/) (2025)
- [ExelMap: Explainable Element-based HD-Map Change Detection and Update](https://mlsanity.com/paper/2409.10178/) (2024)
- [TopoStreamer: Temporal Lane Segment Topology Reasoning in Autonomous Driving](https://mlsanity.com/paper/2507.00709/) (2025)

## Abstract (arXiv)

A map, as crucial information for downstream applications of an autonomous driving system, is usually represented in lanelines or centerlines. However, existing literature on map learning primarily focuses on either detecting geometry-based lanelines or perceiving topology relationships of centerlines. Both of these methods ignore the intrinsic relationship of lanelines and centerlines, that lanelines bind centerlines. While simply predicting both types of lane in one model is mutually excluded in learning objective, we advocate lane segment as a new representation that seamlessly incorporates both geometry and topology information. Thus, we introduce LaneSegNet, the first end-to-end mapping network generating lane segments to obtain a complete representation of the road structure. Our algorithm features two key modifications. One is a lane attention module to capture pivotal region details within the long-range feature space. Another is an identical initialization strategy for reference points, which enhances the learning of positional priors for lane attention. On the OpenLane-V2 dataset, LaneSegNet outperforms previous counterparts by a substantial gain across three tasks, textiti.e., map element detection (+4.8 mAP), centerline perception (+6.9 DETl), and the newly defined one, lane segment perception (+5.6 mAP). Furthermore, it obtains a real-time inference speed of 14.7 FPS. Code is accessible at https://github.com/OpenDriveLab/LaneSegNet.