# End-to-end Lane Shape Prediction with Transformers

arXiv: https://arxiv.org/abs/2011.04233 · HTML: https://mlsanity.com/paper/2011.04233/

## Key facts

- Cited by 111 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 30 of 111)

- [Transformer in Transformer](https://mlsanity.com/paper/2103.00112/) (2021)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [Visual Saliency Transformer](https://mlsanity.com/paper/2104.12099/) (2021)
- [Transformer-Based Attention Networks for Continuous Pixel-Wise Prediction](https://mlsanity.com/paper/2103.12091/) (2021)
- [CondLaneNet: a Top-to-down Lane Detection Framework Based on Conditional Convolution](https://mlsanity.com/paper/2105.05003/) (2021)
- [Rethinking Efficient Lane Detection via Curve Modeling](https://mlsanity.com/paper/2203.02431/) (2022)
- [PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction](https://mlsanity.com/paper/2308.16477/) (2023)
- [P2T: Pyramid Pooling Transformer for Scene Understanding](https://mlsanity.com/paper/2106.12011/) (2021)
- [OadTR: Online Action Detection with Transformers](https://mlsanity.com/paper/2106.11149/) (2021)
- [CLRNet: Cross Layer Refinement Network for Lane Detection](https://mlsanity.com/paper/2203.10350/) (2022)
- [End-to-End Vectorized HD-map Construction with Piecewise Bezier Curve](https://mlsanity.com/paper/2306.09700/) (2023)
- [A Keypoint-based Global Association Network for Lane Detection](https://mlsanity.com/paper/2204.07335/) (2022)
- [TransCrowd: weakly-supervised crowd counting with transformers](https://mlsanity.com/paper/2104.09116/) (2021)
- [TAPE: Task-Agnostic Prior Embedding for Image Restoration](https://mlsanity.com/paper/2203.06074/) (2022)
- [Online Map Vectorization for Autonomous Driving: A Rasterization Perspective](https://mlsanity.com/paper/2306.10502/) (2023)
- [Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction](https://mlsanity.com/paper/2303.08815/) (2023)
- [Anchor3DLane: Learning to Regress 3D Anchors for Monocular 3D Lane Detection](https://mlsanity.com/paper/2301.02371/) (2023)
- [HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Construction](https://mlsanity.com/paper/2403.08639/) (2024)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [Visual Grounding with Transformers](https://mlsanity.com/paper/2105.04281/) (2021)
- [CurveFormer: 3D Lane Detection by Curve Propagation with Curve Queries and Attention](https://mlsanity.com/paper/2209.07989/) (2022)
- [Ultra Fast Deep Lane Detection with Hybrid Anchor Driven Ordinal Classification](https://mlsanity.com/paper/2206.07389/) (2022)
- [Learning to Predict 3D Lane Shape and Camera Pose from a Single Image via Geometry Constraints](https://mlsanity.com/paper/2112.15351/) (2021)
- [VST++: Efficient and Stronger Visual Saliency Transformer](https://mlsanity.com/paper/2310.11725/) (2023)
- [Using Sequences of Life-events to Predict Human Lives](https://mlsanity.com/paper/2306.03009/) (2023)
- [Towards Robust Physical-world Backdoor Attacks on Lane Detection](https://mlsanity.com/paper/2405.05553/) (2024)
- [Laneformer: Object-aware Row-Column Transformers for Lane Detection](https://mlsanity.com/paper/2203.09830/) (2022)
- [VIL-100: A New Dataset and A Baseline Model for Video Instance Lane Detection](https://mlsanity.com/paper/2108.08482/) (2021)
- [RCLane: Relay Chain Prediction for Lane Detection](https://mlsanity.com/paper/2207.09399/) (2022)

## Abstract (arXiv)

Lane detection, the process of identifying lane markings as approximated curves, is widely used for lane departure warning and adaptive cruise control in autonomous vehicles. The popular pipeline that solves it in two steps -- feature extraction plus post-processing, while useful, is too inefficient and flawed in learning the global context and lanes' long and thin structures. To tackle these issues, we propose an end-to-end method that directly outputs parameters of a lane shape model, using a network built with a transformer to learn richer structures and context. The lane shape model is formulated based on road structures and camera pose, providing physical interpretation for parameters of network output. The transformer models non-local interactions with a self-attention mechanism to capture slender structures and global context. The proposed method is validated on the TuSimple benchmark and shows state-of-the-art accuracy with the most lightweight model size and fastest speed. Additionally, our method shows excellent adaptability to a challenging self-collected lane detection dataset, showing its powerful deployment potential in real applications. Codes are available at https://github.com/liuruijin17/LSTR.