# SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding

arXiv: https://arxiv.org/abs/2306.05407 · HTML: https://mlsanity.com/paper/2306.05407/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 84 of its 123 references that are in the map.
- First posted on arXiv 8 Jun 2023.

## Builds on (most cited first, 30 of 84)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [SuperPoint: Self-Supervised Interest Point Detection and Description](https://mlsanity.com/paper/1712.07629/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D](https://mlsanity.com/paper/2008.05711/) (2020)
- [Scaling Vision Transformers](https://mlsanity.com/paper/2106.04560/) (2021)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [What Makes for Good Views for Contrastive Learning?](https://mlsanity.com/paper/2005.10243/) (2020)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [IBRNet: Learning Multi-View Image-Based Rendering](https://mlsanity.com/paper/2102.13090/) (2021)
- [NICE-SLAM: Neural Implicit Scalable Encoding for SLAM](https://mlsanity.com/paper/2112.12130/) (2021)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Quantifying the Carbon Emissions of Machine Learning](https://mlsanity.com/paper/1910.09700/) (2019)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [MonoScene: Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2112.00726/) (2021)
- [DISK: Learning local features with policy gradient](https://mlsanity.com/paper/2006.13566/) (2020)
- [LIFT: Learned Invariant Feature Transform](https://mlsanity.com/paper/1603.09114/) (2016)

## Built on by (most cited first, 30 of 38)

- [Cross-view image geo-localization with Panorama-BEV Co-Retrieval Network](https://mlsanity.com/paper/2408.05475/) (2024)
- [AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesis](https://mlsanity.com/paper/2504.13157/) (2025)
- [FG2: Fine-Grained Cross-View Localization by Fine-Grained Feature Matching](https://mlsanity.com/paper/2503.18725/) (2025)
- [General Place Recognition Survey: Towards Real-World Autonomy](https://mlsanity.com/paper/2405.04812/) (2024)
- [MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Maps](https://mlsanity.com/paper/2407.08561/) (2024)
- [Adapting Fine-Grained Cross-View Localization to Areas without Fine Ground Truth](https://mlsanity.com/paper/2406.00474/) (2024)
- [U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalization](https://mlsanity.com/paper/2310.13766/) (2023)
- [LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignment](https://mlsanity.com/paper/2410.12269/) (2024)
- [Statewide Visual Geolocalization in the Wild](https://mlsanity.com/paper/2409.16763/) (2024)
- [C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimation](https://mlsanity.com/paper/2312.08060/) (2023)
- [Agents Are All You Need for LLM Unlearning](https://mlsanity.com/paper/2502.00406/) (2025)
- [Scaling Image Geo-Localization to Continent Level](https://mlsanity.com/paper/2510.26795/) (2025)
- [BevSplat: Resolving Height Ambiguity via Feature-Based Gaussian Primitives for Weakly-Supervised Cross-View Localization](https://mlsanity.com/paper/2502.09080/) (2025)
- [GeoDistill: Geometry-Guided Self-Distillation for Weakly Supervised Cross-View Localization](https://mlsanity.com/paper/2507.10935/) (2025)
- [Loc2: Interpretable Cross-View Localization via Depth-Lifted Local Feature Matching](https://mlsanity.com/paper/2509.09792/) (2025)
- [Map It Anywhere (MIA): Empowering Bird's Eye View Mapping using Large-scale Public Data](https://mlsanity.com/paper/2407.08726/) (2024)
- [RING#: PR-by-PE Global Localization with Roto-translation Equivariant Gram Learning](https://mlsanity.com/paper/2409.00206/) (2024)
- [LoD-Loc v2: Aerial Visual Localization over Low Level-of-Detail City Models using Explicit Silhouette Alignment](https://mlsanity.com/paper/2507.00659/) (2025)
- [When Cars meet Drones: Hyperbolic Federated Learning for Source-Free Domain Adaptation in Adverse Weather](https://mlsanity.com/paper/2403.13762/) (2024)
- [BEVLoc: Cross-View Localization and Matching via Birds-Eye-View Synthesis](https://mlsanity.com/paper/2410.06410/) (2024)
- [SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completion](https://mlsanity.com/paper/2503.16825/) (2025)
- [AGL-NET: Aerial-Ground Cross-Modal Global Localization with Varying Scales](https://mlsanity.com/paper/2404.03187/) (2024)
- [Maps from Motion (MfM): Generating 2D Semantic Maps from Sparse Multi-view Images](https://mlsanity.com/paper/2411.12620/) (2024)
- [Beyond Memorization: A Multi-Modal Ordinal Regression Benchmark to Expose Popularity Bias in Vision-Language Models](https://mlsanity.com/paper/2512.21337/) (2025)
- [BEVPose: Unveiling Scene Semantics through Pose-Guided Multi-Modal BEV Alignment](https://mlsanity.com/paper/2410.20969/) (2024)
- [SF-Loc: A Visual Mapping and Geo-Localization System based on Sparse Visual Structure Frames](https://mlsanity.com/paper/2412.01500/) (2024)
- [HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Maps](https://mlsanity.com/paper/2601.02730/) (2026)
- [LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignment](https://mlsanity.com/paper/2603.19609/) (2026)
- [Seeing Across Skies and Streets: Feedforward 3D Reconstruction from Satellite, Drone, and Ground Images](https://mlsanity.com/paper/2605.07978/) (2026)
- [Fusing Satellite Imagery and Planimetric Maps for Cross-View Localization](https://mlsanity.com/paper/2606.10166/) (2026)

## Abstract (arXiv)

Semantic 2D maps are commonly used by humans and machines for navigation purposes, whether it's walking or driving. However, these maps have limitations: they lack detail, often contain inaccuracies, and are difficult to create and maintain, especially in an automated fashion. Can we use raw imagery to automatically create better maps that can be easily interpreted by both humans and machines? We introduce SNAP, a deep network that learns rich neural 2D maps from ground-level and overhead images. We train our model to align neural maps estimated from different inputs, supervised only with camera poses over tens of millions of StreetView images. SNAP can resolve the location of challenging image queries beyond the reach of traditional methods, outperforming the state of the art in localization by a large margin. Moreover, our neural maps encode not only geometry and appearance but also high-level semantics, discovered without explicit supervision. This enables effective pre-training for data-efficient semantic scene understanding, with the potential to unlock cost-efficient creation of more detailed maps.