# Learning to Predict 3D Lane Shape and Camera Pose from a Single Image via Geometry Constraints

arXiv: https://arxiv.org/abs/2112.15351 · HTML: https://mlsanity.com/paper/2112.15351/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 31 Dec 2021.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Spatial As Deep: Spatial CNN for Traffic Scene Understanding](https://mlsanity.com/paper/1712.06080/) (2017)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [Geometry-Aware Learning of Maps for Camera Localization](https://mlsanity.com/paper/1712.03342/) (2017)
- [Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection](https://mlsanity.com/paper/2010.12035/) (2020)
- [Image-based localization using LSTMs for structured feature correlation](https://mlsanity.com/paper/1611.07890/) (2016)
- [Towards End-to-End Lane Detection: an Instance Segmentation Approach](https://mlsanity.com/paper/1802.05591/) (2018)
- [Ultra Fast Structure-aware Deep Lane Detection](https://mlsanity.com/paper/2004.11757/) (2020)
- [End-to-end Lane Shape Prediction with Transformers](https://mlsanity.com/paper/2011.04233/) (2020)
- [3D-LaneNet: End-to-End 3D Multiple Lane Detection](https://mlsanity.com/paper/1811.10203/) (2018)
- [PolyLaneNet: Lane Estimation via Deep Polynomial Regression](https://mlsanity.com/paper/2004.10924/) (2020)
- [CurveLane-NAS: Unifying Lane-Sensitive Architecture Search and Adaptive Point Blending](https://mlsanity.com/paper/2007.12147/) (2020)
- [Key Points Estimation and Point Instance Segmentation Approach for Lane Detection](https://mlsanity.com/paper/2002.06604/) (2020)
- [RESA: Recurrent Feature-Shift Aggregator for Lane Detection](https://mlsanity.com/paper/2008.13719/) (2020)
- [Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection](https://mlsanity.com/paper/2003.10656/) (2020)
- [Monocular 3D Object Detection: An Extrinsic Parameter Free Approach](https://mlsanity.com/paper/2106.15796/) (2021)
- [VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization](https://mlsanity.com/paper/1702.06521/) (2017)
- [End-to-End Lane Marker Detection via Row-wise Classification](https://mlsanity.com/paper/2005.08630/) (2020)
- [Hierarchical Recurrent Attention Networks for Structured Online Maps](https://mlsanity.com/paper/2012.12314/) (2020)
- [FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network](https://mlsanity.com/paper/1905.04354/) (2019)
- [Deep Multi-Sensor Lane Detection](https://mlsanity.com/paper/1905.01555/) (2019)
- [Semi-Local 3D Lane Detection and Uncertainty Estimation](https://mlsanity.com/paper/2003.05257/) (2020)
- [Robust Lane Detection via Expanded Self Attention](https://mlsanity.com/paper/2102.07037/) (2021)
- [Towards Lightweight Lane Detection by Optimizing Spatial Embedding](https://mlsanity.com/paper/2008.08311/) (2020)

## Built on by (most cited first, 30 of 30)

- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark](https://mlsanity.com/paper/2203.11089/) (2022)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction](https://mlsanity.com/paper/2303.08815/) (2023)
- [Anchor3DLane: Learning to Regress 3D Anchors for Monocular 3D Lane Detection](https://mlsanity.com/paper/2301.02371/) (2023)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)
- [Online Vectorized HD Map Construction using Geometry](https://mlsanity.com/paper/2312.03341/) (2023)
- [CurveFormer: 3D Lane Detection by Curve Propagation with Curve Queries and Attention](https://mlsanity.com/paper/2209.07989/) (2022)
- [LaneCPP: Continuous 3D Lane Detection using Physical Priors](https://mlsanity.com/paper/2406.08381/) (2024)
- [VMA: Divide-and-Conquer Vectorized Map Annotation System for Large-Scale Driving Scene](https://mlsanity.com/paper/2304.09807/) (2023)
- [CurveFormer++: 3D Lane Detection by Curve Propagation with Temporal Curve Queries and Attention](https://mlsanity.com/paper/2402.06423/) (2024)
- [WS-3D-Lane: Weakly Supervised 3D Lane Detection With 2D Lane Labels](https://mlsanity.com/paper/2209.11523/) (2022)
- [Monocular 3D lane detection for Autonomous Driving: Recent Achievements, Challenges, and Outlooks](https://mlsanity.com/paper/2404.06860/) (2024)
- [DV-3DLane: End-to-end Multi-modal 3D Lane Detection with Dual-view Representation](https://mlsanity.com/paper/2406.16072/) (2024)
- [M2-3DLaneNet: Exploring Multi-Modal 3D Lane Detection](https://mlsanity.com/paper/2209.05996/) (2022)
- [BEV-LaneDet: a Simple and Effective 3D Lane Detection Baseline](https://mlsanity.com/paper/2210.06006/) (2022)
- [BezierFormer: A Unified Architecture for 2D and 3D Lane Detection](https://mlsanity.com/paper/2404.16304/) (2024)
- [RepVF: A Unified Vector Fields Representation for Multi-task 3D Perception](https://mlsanity.com/paper/2407.10876/) (2024)
- [HeightLane: BEV Heightmap guided 3D Lane Detection](https://mlsanity.com/paper/2408.08270/) (2024)
- [Monocular Lane Detection Based on Deep Learning: A Survey](https://mlsanity.com/paper/2411.16316/) (2024)
- [SparseLaneSTP: Leveraging Spatio-Temporal Priors with Sparse Transformers for 3D Lane Detection](https://mlsanity.com/paper/2601.04968/) (2026)
- [PBFormer: Capturing Complex Scene Text Shape with Polynomial Band Transformer](https://mlsanity.com/paper/2308.15004/) (2023)
- [Geometry-Guided Representations for Coherent Lane and Traffic Topology Reasoning in Driving Scenes](https://mlsanity.com/paper/2506.13553/) (2025)
- [RATopo: Improving Lane Topology Reasoning via Redundancy Assignment](https://mlsanity.com/paper/2508.15272/) (2025)
- [Rethinking Lanes and Points in Complex Scenarios for Monocular 3D Lane Detection](https://mlsanity.com/paper/2503.06237/) (2025)
- [Depth3DLane: Monocular 3D Lane Detection via Depth Prior Distillation](https://mlsanity.com/paper/2504.18325/) (2025)
- [Unified Modeling of Lane and Lane Topology for Driving Scene Reasoning](https://mlsanity.com/paper/2605.08911/) (2026)
- [Fine-Grained Representation for Lane Topology Reasoning](https://mlsanity.com/paper/2511.12590/) (2025)
- [Monocular 3D Lane Detection via Structure Uncertainty-Aware Network with Curve-Point Queries](https://mlsanity.com/paper/2511.13055/) (2025)
- [ReManNet: A Riemannian Manifold Network for Monocular 3D Lane Detection](https://mlsanity.com/paper/2603.19776/) (2026)

## Abstract (arXiv)

Detecting 3D lanes from the camera is a rising problem for autonomous vehicles. In this task, the correct camera pose is the key to generating accurate lanes, which can transform an image from perspective-view to the top-view. With this transformation, we can get rid of the perspective effects so that 3D lanes would look similar and can accurately be fitted by low-order polynomials. However, mainstream 3D lane detectors rely on perfect camera poses provided by other sensors, which is expensive and encounters multi-sensor calibration issues. To overcome this problem, we propose to predict 3D lanes by estimating camera pose from a single image with a two-stage framework. The first stage aims at the camera pose task from perspective-view images. To improve pose estimation, we introduce an auxiliary 3D lane task and geometry constraints to benefit from multi-task learning, which enhances consistencies between 3D and 2D, as well as compatibility in the above two tasks. The second stage targets the 3D lane task. It uses previously estimated pose to generate top-view images containing distance-invariant lane appearances for predicting accurate 3D lanes. Experiments demonstrate that, without ground truth camera pose, our method outperforms the state-of-the-art perfect-camera-pose-based methods and has the fewest parameters and computations. Codes are available at https://github.com/liuruijin17/CLGo.