# PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark

arXiv: https://arxiv.org/abs/2203.11089 · HTML: https://mlsanity.com/paper/2203.11089/

## Key facts

- Cited by 140 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 77 references that are in the map.
- First posted on arXiv 21 Mar 2022.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation](https://mlsanity.com/paper/2005.04259/) (2020)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets](https://mlsanity.com/paper/2108.09640/) (2021)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [The ApolloScape Open Dataset for Autonomous Driving and its Application](https://mlsanity.com/paper/1803.06184/) (2018)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [Spatial As Deep: Spatial CNN for Traffic Scene Understanding](https://mlsanity.com/paper/1712.06080/) (2017)
- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Learning Lightweight Lane Detection CNNs by Self Attention Distillation](https://mlsanity.com/paper/1908.00821/) (2019)
- [Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection](https://mlsanity.com/paper/2010.12035/) (2020)
- [Towards End-to-End Lane Detection: an Instance Segmentation Approach](https://mlsanity.com/paper/1802.05591/) (2018)

## Built on by (most cited first, 30 of 140)

- [ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning](https://mlsanity.com/paper/2207.07601/) (2022)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving](https://mlsanity.com/paper/2311.17918/) (2023)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?](https://mlsanity.com/paper/2312.03031/) (2023)
- [Scene as Occupancy](https://mlsanity.com/paper/2306.02851/) (2023)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Driving](https://mlsanity.com/paper/2305.06242/) (2023)
- [Panacea: Panoramic and Controllable Video Generation for Autonomous Driving](https://mlsanity.com/paper/2311.16813/) (2023)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.07656/) (2025)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [Visual Point Cloud Forecasting enables Scalable Autonomous Driving](https://mlsanity.com/paper/2312.17655/) (2023)
- [FB-BEV: BEV Representation from Forward-Backward View Transformations](https://mlsanity.com/paper/2308.02236/) (2023)
- [PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction](https://mlsanity.com/paper/2308.16477/) (2023)
- [Cosmos-Drive-Dreams: Scalable Synthetic Driving Data Generation with World Foundation Models](https://mlsanity.com/paper/2506.09042/) (2025)
- [OpenLane-V2: A Topology Reasoning Benchmark for Unified 3D HD Mapping](https://mlsanity.com/paper/2304.10440/) (2023)
- [End-to-End Vectorized HD-map Construction with Piecewise Bezier Curve](https://mlsanity.com/paper/2306.09700/) (2023)
- [Fully Sparse 3D Occupancy Prediction](https://mlsanity.com/paper/2312.17118/) (2023)
- [LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving](https://mlsanity.com/paper/2312.16108/) (2023)
- [Graph-based Topology Reasoning for Driving Scenes](https://mlsanity.com/paper/2304.05277/) (2023)
- [Online Map Vectorization for Autonomous Driving: A Rasterization Perspective](https://mlsanity.com/paper/2306.10502/) (2023)
- [Lane Graph as Path: Continuity-preserving Path-wise Modeling for Online Lane Graph Construction](https://mlsanity.com/paper/2303.08815/) (2023)
- [Pure Vision Language Action (VLA) Models: A Comprehensive Survey](https://mlsanity.com/paper/2509.19012/) (2025)
- [Anchor3DLane: Learning to Regress 3D Anchors for Monocular 3D Lane Detection](https://mlsanity.com/paper/2301.02371/) (2023)
- [HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Construction](https://mlsanity.com/paper/2403.08639/) (2024)
- [Leveraging Enhanced Queries of Point Sets for Vectorized Map Construction](https://mlsanity.com/paper/2402.17430/) (2024)
- [Boosting 3-DoF Ground-to-Satellite Camera Localization Accuracy via Geometry-Guided Cross-View Transformer](https://mlsanity.com/paper/2307.08015/) (2023)
- [LATR: 3D Lane Detection from Monocular Images with Transformer](https://mlsanity.com/paper/2308.04583/) (2023)

## Abstract (arXiv)

Methods for 3D lane detection have been recently proposed to address the issue of inaccurate lane layouts in many autonomous driving scenarios (uphill/downhill, bump, etc.). Previous work struggled in complex cases due to their simple designs of the spatial transformation between front view and bird's eye view (BEV) and the lack of a realistic dataset. Towards these issues, we present PersFormer: an end-to-end monocular 3D lane detector with a novel Transformer-based spatial feature transformation module. Our model generates BEV features by attending to related front-view local regions with camera parameters as a reference. PersFormer adopts a unified 2D/3D anchor design and an auxiliary task to detect 2D/3D lanes simultaneously, enhancing the feature consistency and sharing the benefits of multi-task learning. Moreover, we release one of the first large-scale real-world 3D lane datasets: OpenLane, with high-quality annotation and scenario diversity. OpenLane contains 200,000 frames, over 880,000 instance-level lanes, 14 lane categories, along with scene tags and the closed-in-path object annotations to encourage the development of lane detection and more industrial-related autonomous driving methods. We show that PersFormer significantly outperforms competitive baselines in the 3D lane detection task on our new OpenLane dataset as well as Apollo 3D Lane Synthetic dataset, and is also on par with state-of-the-art algorithms in the 2D task on OpenLane. The project page is available at https://github.com/OpenPerceptionX/PersFormer_3DLane and OpenLane dataset is provided at https://github.com/OpenPerceptionX/OpenLane.