mlsanityData to 7 Oct 2026

SNAP: Self-Supervised Neural Maps for Visual Positioning and Semantic Understanding

arXiv 2306.05407 ↗First posted 8 Jun 2023cs.CVcited by 38

Not on the map. Why?

Abstract (arXiv)

Semantic 2D maps are commonly used by humans and machines for navigation purposes, whether it's walking or driving. However, these maps have limitations: they lack detail, often contain inaccuracies, and are difficult to create and maintain, especially in an automated fashion. Can we use raw imagery to automatically create better maps that can be easily interpreted by both humans and machines? We introduce SNAP, a deep network that learns rich neural 2D maps from ground-level and overhead images. We train our model to align neural maps estimated from different inputs, supervised only with camera poses over tens of millions of StreetView images. SNAP can resolve the location of challenging image queries beyond the reach of traditional methods, outperforming the state of the art in localization by a large margin. Moreover, our neural maps encode not only geometry and appearance but also high-level semantics, discovered without explicit supervision. This enables effective pre-training for data-efficient semantic scene understanding, with the potential to unlock cost-efficient creation of more detailed maps.

Key facts. Cited by 38 papers in the map (data through 7 Oct 2026). Builds on 84 of its 123 references that are in the map. First posted on arXiv 8 Jun 2023.

Builds on

84 of 123 references.

References · 84

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  8. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  11. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  12. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
38 more
  1. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  2. 2020Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3Dcited by 1,040
  3. 2021Scaling Vision Transformerscited by 993
  4. 2021LoFTR: Detector-Free Local Feature Matching with Transformerscited by 974
  5. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  6. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  7. 2019Big Transfer (BiT): General Visual Representation Learningcited by 766
  8. 2020Convolutional Occupancy Networkscited by 760
  9. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  10. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  11. 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 698
  12. 2021NICE-SLAM: Neural Implicit Scalable Encoding for SLAMcited by 586
  13. 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 556
  14. 2019Quantifying the Carbon Emissions of Machine Learningcited by 516
  15. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  16. 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 357
  17. 2020DISK: Learning local features with policy gradientcited by 310
  18. 2016LIFT: Learned Invariant Feature Transformcited by 305
  19. 2017Sampling Matters in Deep Embedding Learningcited by 285
  20. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  21. 2020Image Matching across Wide Baselines: From Paper to Practicecited by 224
  22. 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 222
  23. 2017Working hard to know your neighbor's margins: Local descriptor learning losscited by 212
  24. 2019R2D2: Repeatable and Reliable Detector and Descriptorcited by 206
  25. 2020Atlas: End-to-End 3D Scene Reconstruction from Posed Imagescited by 206
  26. 2018Orthographic Feature Transform for Monocular 3D Object Detectioncited by 201
  27. 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
  28. 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 199
  29. 2021COTR: Correspondence Transformer for Matching Across Imagescited by 182
  30. 2017Learning to Find Good Correspondencescited by 176
  31. 2019MAGSAC++, a fast, reliable and accurate robust estimatorcited by 160
  32. 2019Learning Two-View Correspondences and Geometry Using Order-Aware Networkcited by 148
  33. 2020Unsupervised Learning of Dense Visual Representationscited by 147
  34. 2019SOSNet: Second Order Similarity Regularization for Local Descriptor Learningcited by 131
  35. 2019Neural-Guided RANSAC: Learning Where to Sample Model Hypothesescited by 129
  36. 2020ASLFeat: Learning Local Features of Accurate Shape and Localizationcited by 129
  37. 2022Simple-BEV: What Really Matters for Multi-Sensor BEV Perception?cited by 128
  38. 2022MatchFormer: Interleaving Attention in Transformers for Feature Matchingcited by 126

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 14; 2025: 10; 2026: 12.

38 papers.

Sort:

By area:

  1. 2024Cross-view image geo-localization with Panorama-BEV Co-Retrieval Networkcited by 40
  2. 2025AerialMegaDepth: Learning Aerial-Ground Reconstruction and View Synthesiscited by 39
  3. 2025FG2: Fine-Grained Cross-View Localization by Fine-Grained Feature Matchingcited by 27
  4. 2024General Place Recognition Survey: Towards Real-World Autonomycited by 23
  5. 2024MapLocNet: Coarse-to-Fine Feature Registration for Visual Re-Localization in Navigation Mapscited by 18
  6. 2024Adapting Fine-Grained Cross-View Localization to Areas without Fine Ground Truthcited by 14
  7. 2023U-BEV: Height-aware Bird's-Eye-View Segmentation and Neural Map-based Relocalizationcited by 13
  8. 2024Statewide Visual Geolocalization in the Wildcited by 12
  9. 2024LoD-Loc: Aerial Visual Localization using LoD 3D Map with Neural Wireframe Alignmentcited by 12
  10. 2023C-BEV: Contrastive Bird's Eye View Training for Cross-View Image Retrieval and 3-DoF Pose Estimationcited by 10
  11. 2025Scaling Image Geo-Localization to Continent Levelcited by 10
  12. 2025Agents Are All You Need for LLM Unlearningcited by 9
26 more
  1. 2025GeoDistill: Geometry-Guided Self-Distillation for Weakly Supervised Cross-View Localizationcited by 8
  2. 2025Loc2: Interpretable Cross-View Localization via Depth-Lifted Local Feature Matchingcited by 8
  3. 2025BevSplat: Resolving Height Ambiguity via Feature-Based Gaussian Primitives for Weakly-Supervised Cross-View Localizationcited by 7
  4. 2024Map It Anywhere (MIA): Empowering Bird's Eye View Mapping using Large-scale Public Datacited by 5
  5. 2024RING#: PR-by-PE Global Localization with Roto-translation Equivariant Gram Learningcited by 5
  6. 2025LoD-Loc v2: Aerial Visual Localization over Low Level-of-Detail City Models using Explicit Silhouette Alignmentcited by 5
  7. 2024When Cars meet Drones: Hyperbolic Federated Learning for Source-Free Domain Adaptation in Adverse Weathercited by 4
  8. 2024BEVLoc: Cross-View Localization and Matching via Birds-Eye-View Synthesiscited by 4
  9. 2025SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completioncited by 3
  10. 2024AGL-NET: Aerial-Ground Cross-Modal Global Localization with Varying Scalescited by 2
  11. 2024Maps from Motion (MfM): Generating 2D Semantic Maps from Sparse Multi-view Imagescited by 1
  12. 2024BEVPose: Unveiling Scene Semantics through Pose-Guided Multi-Modal BEV Alignmentcited by 0
  13. 2024SF-Loc: A Visual Mapping and Geo-Localization System based on Sparse Visual Structure Framescited by 0
  14. 2025Beyond Memorization: A Multi-Modal Ordinal Regression Benchmark to Expose Popularity Bias in Vision-Language Modelscited by 0
  15. 2026HOLO: Homography-Guided Pose Estimator Network for Fine-Grained Visual Localization on SD Mapscited by 0
  16. 2026LoD-Loc v3: Generalized Aerial Localization in Dense Cities using Instance Silhouette Alignmentcited by 0
  17. 2026Seeing Across Skies and Streets: Feedforward 3D Reconstruction from Satellite, Drone, and Ground Imagescited by 0
  18. 2026Fusing Satellite Imagery and Planimetric Maps for Cross-View Localizationcited by 0
  19. 2026Globally Localizing Lunar Rover in Pixels via Graph Alignmentcited by 0
  20. 2026Cross-View Yaw Estimation in Location Uncertainty with Line-Aligning Yaw Scoringcited by 0
  21. 2026More Than Where You Are: Learning Semantics, Structure, and Geometry from Cross-View Localizationcited by 0
  22. 2026Spotter: Efficient Urban Visual Localization via Geo-Referenced Facade Landmarks in GPS-Degraded Environmentscited by 0
  23. 2026OpenCVL: An Open, Diverse, and Large-Scale Dataset for Fine-Grained Cross-View Localizationcited by 0
  24. 2026AutoCompass: Accurate Visual Localization on Public Maps by Learning from Weak Labelscited by 0
  25. 2026Multimodal Floorplan Encoding: Learning Dense Modality-Invariant Representationscited by 0
  26. 2026How to Reduce Localization Ambiguity? Geometry-Semantic Constrained BEV Representation Learning for Satellite-Ground Localizationcited by 0