# HIMap: HybrId Representation Learning for End-to-end Vectorized HD Map Construction

arXiv: https://arxiv.org/abs/2403.08639 · HTML: https://mlsanity.com/paper/2403.08639/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 71 references that are in the map.
- First posted on arXiv 13 Mar 2024.

## Builds on (most cited first, 30 of 53)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model](https://mlsanity.com/paper/2310.01412/) (2023)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning](https://mlsanity.com/paper/2207.07601/) (2022)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [HDMapNet: An Online HD Map Construction and Evaluation Framework](https://mlsanity.com/paper/2107.06307/) (2021)
- [Cross-view Transformers for real-time Map-view Semantic Segmentation](https://mlsanity.com/paper/2205.02833/) (2022)
- [Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baseline](https://mlsanity.com/paper/2206.08129/) (2022)
- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [FIERY: Future Instance Prediction in Bird's-Eye View from Surround Monocular Cameras](https://mlsanity.com/paper/2104.10490/) (2021)
- [Multimodal Motion Prediction with Stacked Transformers](https://mlsanity.com/paper/2103.11624/) (2021)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [PnPNet: End-to-End Perception and Prediction with Tracking in the Loop](https://mlsanity.com/paper/2005.14711/) (2020)
- [PersFormer: 3D Lane Detection via Perspective Transformer and the OpenLane Benchmark](https://mlsanity.com/paper/2203.11089/) (2022)
- [Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection](https://mlsanity.com/paper/2010.12035/) (2020)
- [Ultra Fast Structure-aware Deep Lane Detection](https://mlsanity.com/paper/2004.11757/) (2020)
- [End-to-end Lane Shape Prediction with Transformers](https://mlsanity.com/paper/2011.04233/) (2020)
- [Multimodal End-to-End Autonomous Driving](https://mlsanity.com/paper/1906.03199/) (2019)

## Built on by (most cited first, 30 of 48)

- [Is Your HD Map Constructor Reliable under Sensor Corruptions?](https://mlsanity.com/paper/2406.12214/) (2024)
- [MapExpert: Online HD Map Construction with Simple and Efficient Sparse Map Element Expert](https://mlsanity.com/paper/2412.12704/) (2024)
- [MSC-Bench: Benchmarking and Analyzing Multi-Sensor Corruption for Driving Perception](https://mlsanity.com/paper/2501.01037/) (2025)
- [Trajectory Prediction for Autonomous Driving: Progress, Limitations, and Future Directions](https://mlsanity.com/paper/2503.03262/) (2025)
- [SafeMap: Robust HD Map Construction from Incomplete Observations](https://mlsanity.com/paper/2507.00861/) (2025)
- [MapDiffusion: Generative Diffusion for Vectorized Online HD Map Construction and Uncertainty Estimation in Autonomous Driving](https://mlsanity.com/paper/2507.21423/) (2025)
- [PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2407.17378/) (2024)
- [Unveiling the Hidden: Online Vectorized HD Map Construction with Clip-Level Token Interaction and Propagation](https://mlsanity.com/paper/2411.11002/) (2024)
- [PriorDrive: Enhancing Online HD Mapping with Unified Vector Priors](https://mlsanity.com/paper/2409.05352/) (2024)
- [SeqGrowGraph: Learning Lane Topology as a Chain of Graph Expansions](https://mlsanity.com/paper/2507.04822/) (2025)
- [PriorMapNet: Enhancing Online Vectorized HD Map Construction with Priors](https://mlsanity.com/paper/2408.08802/) (2024)
- [MGMapNet: Multi-Granularity Representation Learning for End-to-End Vectorized HD Map Construction](https://mlsanity.com/paper/2410.07733/) (2024)
- [What Really Matters for Robust Multi-Sensor HD Map Construction?](https://mlsanity.com/paper/2507.01484/) (2025)
- [InteractionMap: Improving Online Vectorized HDMap Construction with Interaction](https://mlsanity.com/paper/2503.21659/) (2025)
- [SDTagNet: Leveraging Text-Annotated Navigation Maps for Online HD Map Construction](https://mlsanity.com/paper/2506.08997/) (2025)
- [TopoMaskV2: Enhanced Instance-Mask-Based Formulation for the Road Topology Problem](https://mlsanity.com/paper/2409.11325/) (2024)
- [Improving Hierarchical Representations of Vectorized HD Maps with Perspective Clues](https://mlsanity.com/paper/2404.11155/) (2024)
- [AugMapNet: Improving Spatial Latent Structure via BEV Grid Augmentation for Enhanced Vectorized Online HD Map Construction](https://mlsanity.com/paper/2503.13430/) (2025)
- [Delving into Mapping Uncertainty for Mapless Trajectory Prediction](https://mlsanity.com/paper/2507.18498/) (2025)
- [DAMap: Distance-aware MapNet for High Quality HD Map Construction](https://mlsanity.com/paper/2510.22675/) (2025)
- [TopoBDA: Towards Bezier Deformable Attention for Road Topology Understanding](https://mlsanity.com/paper/2412.18951/) (2024)
- [PseudoMapTrainer: Learning Online Mapping without HD Maps](https://mlsanity.com/paper/2508.18788/) (2025)
- [Monocular Lane Detection Based on Deep Learning: A Survey](https://mlsanity.com/paper/2411.16316/) (2024)
- [Uni-PrevPredMap: Extending PrevPredMap to a Unified Framework of Prior-Informed Modeling for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2504.06647/) (2025)
- [SparseMeXT Unlocking the Potential of Sparse Representations for HD Map Construction](https://mlsanity.com/paper/2505.08808/) (2025)
- [M3TR: A Generalist Model for Real-World HD Map Completion](https://mlsanity.com/paper/2411.10316/) (2024)
- [FastMap: Fast Queries Initialization Based Vectorized HD Map Reconstruction Framework](https://mlsanity.com/paper/2503.05492/) (2025)
- [Uncertainty-Instructed Structure Injection for Generalizable HD Map Construction](https://mlsanity.com/paper/2503.23109/) (2025)
- [Geometry-Guided Representations for Coherent Lane and Traffic Topology Reasoning in Driving Scenes](https://mlsanity.com/paper/2506.13553/) (2025)
- [Coherent Online Road Topology Estimation and Reasoning with Standard-Definition Maps](https://mlsanity.com/paper/2507.01397/) (2025)

## Abstract (arXiv)

Vectorized High-Definition (HD) map construction requires predictions of the category and point coordinates of map elements (e.g. road boundary, lane divider, pedestrian crossing, etc.). State-of-the-art methods are mainly based on point-level representation learning for regressing accurate point coordinates. However, this pipeline has limitations in obtaining element-level information and handling element-level failures, e.g. erroneous element shape or entanglement between elements. To tackle the above issues, we propose a simple yet effective HybrId framework named HIMap to sufficiently learn and interact both point-level and element-level information. Concretely, we introduce a hybrid representation called HIQuery to represent all map elements, and propose a point-element interactor to interactively extract and encode the hybrid information of elements, e.g. point position and element shape, into the HIQuery. Additionally, we present a point-element consistency constraint to enhance the consistency between the point-level and element-level information. Finally, the output point-element integrated HIQuery can be directly converted into map elements' class, point coordinates, and mask. We conduct extensive experiments and consistently outperform previous methods on both nuScenes and Argoverse2 datasets. Notably, our method achieves 77.8 mAP on the nuScenes dataset, remarkably superior to previous SOTAs by 8.3 mAP at least.