# MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping

arXiv: https://arxiv.org/abs/2403.15951 · HTML: https://mlsanity.com/paper/2403.15951/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 49 references that are in the map.
- First posted on arXiv 23 Mar 2024.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision](https://mlsanity.com/paper/2211.10439/) (2022)
- [HDMapNet: An Online HD Map Construction and Evaluation Framework](https://mlsanity.com/paper/2107.06307/) (2021)
- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [LIO-SAM: Tightly-coupled Lidar Inertial Odometry via Smoothing and Mapping](https://mlsanity.com/paper/2007.00258/) (2020)
- [MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors](https://mlsanity.com/paper/2211.09791/) (2022)
- [StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Construction](https://mlsanity.com/paper/2308.12570/) (2023)
- [Vision-Centric BEV Perception: A Survey](https://mlsanity.com/paper/2208.02797/) (2022)
- [Delving into the Devils of Bird's-eye-view Perception: A Review, Evaluation and Recipe](https://mlsanity.com/paper/2209.05324/) (2022)
- [ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries](https://mlsanity.com/paper/2208.01582/) (2022)
- [MeMOT: Multi-Object Tracking with Memory](https://mlsanity.com/paper/2203.16761/) (2022)
- [Sparse4D v2: Recurrent Temporal Fusion with Sparse Model](https://mlsanity.com/paper/2305.14018/) (2023)
- [MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Tracking](https://mlsanity.com/paper/2307.15700/) (2023)
- [PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction](https://mlsanity.com/paper/2308.16477/) (2023)

## Built on by (most cited first, 30 of 68)

- [Is Your HD Map Constructor Reliable under Sensor Corruptions?](https://mlsanity.com/paper/2406.12214/) (2024)
- [MapExpert: Online HD Map Construction with Simple and Efficient Sparse Map Element Expert](https://mlsanity.com/paper/2412.12704/) (2024)
- [GlobalMapNet: An Online Framework for Vectorized Global HD Map Construction](https://mlsanity.com/paper/2409.10063/) (2024)
- [PrevPredMap: Exploring Temporal Modeling with Previous Predictions for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2407.17378/) (2024)
- [TopoSD: Topology-Enhanced Lane Segment Perception with SDMap Prior](https://mlsanity.com/paper/2411.14751/) (2024)
- [DTCLMapper: Dual Temporal Consistent Learning for Vectorized HD Map Construction](https://mlsanity.com/paper/2405.05518/) (2024)
- [MemFusionMap: Working Memory Fusion for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2409.18737/) (2024)
- [HisTrackMap: Global Vectorized High-Definition Map Construction via History Map Tracking](https://mlsanity.com/paper/2503.07168/) (2025)
- [InteractionMap: Improving Online Vectorized HDMap Construction with Interaction](https://mlsanity.com/paper/2503.21659/) (2025)
- [SDTagNet: Leveraging Text-Annotated Navigation Maps for Online HD Map Construction](https://mlsanity.com/paper/2506.08997/) (2025)
- [TopoMaskV2: Enhanced Instance-Mask-Based Formulation for the Road Topology Problem](https://mlsanity.com/paper/2409.11325/) (2024)
- [Stability Under Scrutiny: Benchmarking Representation Paradigms for Online HD Mapping](https://mlsanity.com/paper/2510.10660/) (2025)
- [Exploring Semi-Supervised Learning for Online Mapping](https://mlsanity.com/paper/2410.10279/) (2024)
- [Persistent Autoregressive Mapping with Traffic Rules for Autonomous Driving](https://mlsanity.com/paper/2509.22756/) (2025)
- [TopoBDA: Towards Bezier Deformable Attention for Road Topology Understanding](https://mlsanity.com/paper/2412.18951/) (2024)
- [Divide and Merge: Motion and Semantic Learning in End-to-End Autonomous Driving](https://mlsanity.com/paper/2502.07631/) (2025)
- [RTMap: Real-Time Recursive Mapping with Change Detection and Localization](https://mlsanity.com/paper/2507.00980/) (2025)
- [PseudoMapTrainer: Learning Online Mapping without HD Maps](https://mlsanity.com/paper/2508.18788/) (2025)
- [Spatial Retrieval Augmented Autonomous Driving](https://mlsanity.com/paper/2512.06865/) (2025)
- [GenMapping: Unleashing the Potential of Inverse Perspective Mapping for Robust Online HD Map Construction](https://mlsanity.com/paper/2409.08688/) (2024)
- [Monocular Lane Detection Based on Deep Learning: A Survey](https://mlsanity.com/paper/2411.16316/) (2024)
- [Uni-PrevPredMap: Extending PrevPredMap to a Unified Framework of Prior-Informed Modeling for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2504.06647/) (2025)
- [UniMapGen: A Generative Framework for Large-Scale Map Construction from Multi-modal Data](https://mlsanity.com/paper/2509.22262/) (2025)
- [HeightMapNet: Explicit Height Modeling for End-to-End HD Map Learning](https://mlsanity.com/paper/2411.01408/) (2024)
- [M3TR: A Generalist Model for Real-World HD Map Completion](https://mlsanity.com/paper/2411.10316/) (2024)
- [FastMap: Fast Queries Initialization Based Vectorized HD Map Reconstruction Framework](https://mlsanity.com/paper/2503.05492/) (2025)
- [Uncertainty-Instructed Structure Injection for Generalizable HD Map Construction](https://mlsanity.com/paper/2503.23109/) (2025)
- [TopoStreamer: Temporal Lane Segment Topology Reasoning in Autonomous Driving](https://mlsanity.com/paper/2507.00709/) (2025)
- [Coherent Online Road Topology Estimation and Reasoning with Standard-Definition Maps](https://mlsanity.com/paper/2507.01397/) (2025)
- [Mapping like a Skeptic: Probabilistic BEV Projection for Online HD Mapping](https://mlsanity.com/paper/2508.21689/) (2025)

## Abstract (arXiv)

This paper presents a vector HD-mapping algorithm that formulates the mapping as a tracking task and uses a history of memory latents to ensure consistent reconstructions over time. Our method, MapTracker, accumulates a sensor stream into memory buffers of two latent representations: 1) Raster latents in the bird's-eye-view (BEV) space and 2) Vector latents over the road elements (i.e., pedestrian-crossings, lane-dividers, and road-boundaries). The approach borrows the query propagation paradigm from the tracking literature that explicitly associates tracked road elements from the previous frame to the current, while fusing a subset of memory latents selected with distance strides to further enhance temporal consistency. A vector latent is decoded to reconstruct the geometry of a road element. The paper further makes benchmark contributions by 1) Improving processing code for existing datasets to produce consistent ground truth with temporal alignments and 2) Augmenting existing mAP metrics with consistency checks. MapTracker significantly outperforms existing methods on both nuScenes and Agroverse2 datasets by over 8% and 19% on the conventional and the new consistency-aware metrics, respectively. The code and models are available on our project page: https://map-tracker.github.io.