# Leveraging Vision-Based Point Cloud Map Priors for Camera-Based 3D Object Detection and Online Vectorized HD Mapping

arXiv: https://arxiv.org/abs/2609.26325 · HTML: https://mlsanity.com/paper/2609.26325/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 37 of its 39 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Builds on (most cited first, 30 of 37)

- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation](https://mlsanity.com/paper/2205.13542/) (2022)
- [Argoverse 2: Next Generation Datasets for Self-Driving Perception and Forecasting](https://mlsanity.com/paper/2301.00493/) (2023)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [π3: Permutation-Equivariant Visual Geometry Learning](https://mlsanity.com/paper/2507.13347/) (2025)
- [Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.11926/) (2023)
- [SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation](https://mlsanity.com/paper/2405.19620/) (2024)
- [MapTRv2: An End-to-End Framework for Online Vectorized HD Map Construction](https://mlsanity.com/paper/2308.05736/) (2023)
- [An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection](https://mlsanity.com/paper/1904.09730/) (2019)
- [StreamMapNet: Streaming Mapping Network for Vectorized Online HD Map Construction](https://mlsanity.com/paper/2308.12570/) (2023)
- [Far3D: Expanding the Horizon for Surround-view 3D Object Detection](https://mlsanity.com/paper/2308.09616/) (2023)
- [Sparse4D v3: Advancing End-to-End 3D Detection and Tracking](https://mlsanity.com/paper/2311.11722/) (2023)
- [MapTracker: Tracking with Strided Memory Fusion for Consistent Vector HD Mapping](https://mlsanity.com/paper/2403.15951/) (2024)
- [Robust Autonomy Emerges from Self-Play](https://mlsanity.com/paper/2502.03349/) (2025)
- [Stream Query Denoising for Vectorized HD Map Construction](https://mlsanity.com/paper/2401.09112/) (2024)
- [BEVNeXt: Reviving Dense BEV Frameworks for 3D Object Detection](https://mlsanity.com/paper/2312.01696/) (2023)
- [PreSight: Enhancing Autonomous Vehicle Perception with City-Scale NeRF Priors](https://mlsanity.com/paper/2403.09079/) (2024)
- [Hindsight is 20/20: Leveraging Past Traversals to Aid 3D Perception](https://mlsanity.com/paper/2203.11405/) (2022)
- [Learning and Aggregating Lane Graphs for Urban Automated Driving](https://mlsanity.com/paper/2302.06175/) (2023)
- [AmodalSynthDrive: A Synthetic Amodal Perception Dataset for Autonomous Driving](https://mlsanity.com/paper/2309.06547/) (2023)
- [Panoptic Out-of-Distribution Segmentation](https://mlsanity.com/paper/2310.11797/) (2023)
- [Syn-Mediverse: A Multimodal Synthetic Dataset for Intelligent Scene Understanding of Healthcare Facilities](https://mlsanity.com/paper/2308.03193/) (2023)
- [Progressive Multi-Modal Fusion for Robust 3D Object Detection](https://mlsanity.com/paper/2410.07475/) (2024)
- [A Point-Based Approach to Efficient LiDAR Multi-Task Perception](https://mlsanity.com/paper/2404.12798/) (2024)
- [Label-Efficient LiDAR Semantic Segmentation with 2D-3D Vision Transformer Adapters](https://mlsanity.com/paper/2503.03299/) (2025)
- [Bridging Perspectives: Foundation Model Guided BEV Maps for 3D Object Detection and Tracking](https://mlsanity.com/paper/2510.10287/) (2025)
- [Amodal Optical Flow](https://mlsanity.com/paper/2311.07761/) (2023)
- [A Good Foundation is Worth Many Labels: Label-Efficient Panoptic Segmentation](https://mlsanity.com/paper/2405.19035/) (2024)

## Abstract (arXiv)

Camera-based 3D object detection and online vectorized HD mapping provide compact scene representations for autonomous driving, but both depend on accurate metric geometry and remain limited by depth ambiguity. Over long-term deployment, observations from repeated traversals can be accumulated into persistent point cloud priors that provide geometric context beyond the current observations. Existing explicit point cloud prior approaches, however, rely on LiDAR-based map construction and therefore require expensive 3D ranging sensors. We propose a framework that constructs a static point cloud prior map from previous camera traversals using Pi3X and augments each point with DINOv3 features. At runtime, a local prior patch is retrieved using global localization, encoded with a sparse voxel backbone, and fused in bird's-eye view (BEV) with lifted multi-view camera features. Task-specific sparse transformer heads then predict 3D objects and vectorized map elements from the fused representation. On Argoverse 2, the vision-based prior improves a strong baseline from 0.287 to 0.299 CDS and from 0.669 to 0.750 vectorized mapping mAP. Ablations show that semantic DINOv3 features are particularly important for vectorized mapping. These results demonstrate that vision-built geometric-semantic priors provide an effective form of long-term scene memory for camera-based perception, improving both tasks without LiDAR for prior-map construction or online inference.