# From Fragments to Global Maps: Learning Vectorized Map Aggregation with Large Language Models

arXiv: https://arxiv.org/abs/2610.02513 · HTML: https://mlsanity.com/paper/2610.02513/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 25 references that are in the map.
- First posted on arXiv 1 Oct 2026.

## Builds on (most cited first, 22 of 22)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [MapTR: Structured Modeling and Learning for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2208.14437/) (2022)
- [HDMapNet: An Online HD Map Construction and Evaluation Framework](https://mlsanity.com/paper/2107.06307/) (2021)
- [VectorMapNet: End-to-end Vectorized HD Map Learning](https://mlsanity.com/paper/2206.08920/) (2022)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Construction](https://mlsanity.com/paper/2308.12570/) (2023)
- [Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs](https://mlsanity.com/paper/2402.14903/) (2024)
- [PivotNet: Vectorized Pivot Learning for End-to-end HD Map Construction](https://mlsanity.com/paper/2308.16477/) (2023)
- [End-to-End Vectorized HD-map Construction with Piecewise Bezier Curve](https://mlsanity.com/paper/2306.09700/) (2023)
- [MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping](https://mlsanity.com/paper/2403.15951/) (2024)
- [GlobalMapNet: An Online Framework for Vectorized Global HD Map Construction](https://mlsanity.com/paper/2409.10063/) (2024)
- [VMA: Divide-and-Conquer Vectorized Map Annotation System for Large-Scale Driving Scene](https://mlsanity.com/paper/2304.09807/) (2023)
- [Road Mapping and Localization using Sparse Semantic Visual Features](https://mlsanity.com/paper/2108.05047/) (2021)
- [Online Monocular Lane Mapping Using Catmull-Rom Spline](https://mlsanity.com/paper/2307.11653/) (2023)
- [Efficient numeracy in language models through single-token number embeddings](https://mlsanity.com/paper/2510.06824/) (2025)
- [Persistent Autoregressive Mapping with Traffic Rules for Autonomous Driving](https://mlsanity.com/paper/2509.22756/) (2025)
- [Neural HD Map Generation from Multiple Vectorized Tiles Locally Produced by Autonomous Vehicles](https://mlsanity.com/paper/2409.03445/) (2024)
- [UniMapGen: A Generative Framework for Large-Scale Map Construction from Multi-modal Data](https://mlsanity.com/paper/2509.22262/) (2025)
- [MapAgent: An Industrial-Grade Agentic Framework for City-scale Lane-level Map Generation](https://mlsanity.com/paper/2606.04513/) (2026)

## Abstract (arXiv)

Large-scale vectorized HD maps provide structured road information that is essential for perception, localization, and planning in autonomous driving. Constructing such maps requires aggregating noisy, fragmented, and overlapping local predictions collected along a vehicle trajectory into a coherent global map. Existing aggregation methods typically rely on hand-crafted rules for fragment association and refinement. However, a fixed set of thresholds cannot effectively handle variations in road structures and prediction errors, often requiring detector-specific tuning or manual adjustment. To address this limitation, we propose MapMergeLLM, a data-driven framework that formulates vectorized map aggregation as conditional sequence generation with a large language model. Given serialized local vectorized maps, our model directly predicts the aggregated global map polylines. To reduce dependence on any particular upstream detector, we train the model on synthetic local maps generated from clean vector maps using corruptions that simulate representative prediction errors. We further introduce a coordinate tokenizer with geometry-aware pretraining to precisely represent map coordinates. In addition, we propose a line-level association loss that explicitly supervises correspondences between local observations of the same map element. Experiments on Argoverse2 and nuScenes using multiple recent upstream detectors demonstrate that MapMergeLLM substantially outperforms heuristic and optimization-based aggregation baselines without detector-specific retraining.