# SceneGraphLoc: Cross-Modal Coarse Visual Localization on 3D Scene Graphs

arXiv: https://arxiv.org/abs/2404.00469 · HTML: https://mlsanity.com/paper/2404.00469/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 86 of its 139 references that are in the map.
- First posted on arXiv 30 Mar 2024.

## Builds on (most cited first, 30 of 86)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [NetVLAD: CNN architecture for weakly supervised place recognition](https://mlsanity.com/paper/1511.07247/) (2015)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [From Coarse to Fine: Robust Hierarchical Localization at Large Scale](https://mlsanity.com/paper/1812.03506/) (2018)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [OpenMask3D: Open-Vocabulary 3D Instance Segmentation](https://mlsanity.com/paper/2306.13631/) (2023)
- [CosyPose: Consistent multi-view multi-object 6D pose estimation](https://mlsanity.com/paper/2008.08465/) (2020)
- [3D Scene Graph: A Structure for Unified Semantics, 3D Space, and Camera](https://mlsanity.com/paper/1910.02527/) (2019)
- [DSAC - Differentiable RANSAC for Camera Localization](https://mlsanity.com/paper/1611.05705/) (2016)
- [Patch-NetVLAD: Multi-Scale Fusion of Locally-Global Descriptors for Place Recognition](https://mlsanity.com/paper/2103.01486/) (2021)
- [Geometric Loss Functions for Camera Pose Regression with Deep Learning](https://mlsanity.com/paper/1704.00390/) (2017)
- [Texture Fields: Learning Texture Representations in Function Space](https://mlsanity.com/paper/1905.07259/) (2019)
- [Learning 3D Semantic Scene Graphs from 3D Indoor Reconstructions](https://mlsanity.com/paper/2004.03967/) (2020)
- [Atlas: End-to-End 3D Scene Reconstruction from Posed Images](https://mlsanity.com/paper/2003.10432/) (2020)
- [Back to the Feature: Learning Robust Camera Localization from Pixels to Pose](https://mlsanity.com/paper/2103.09213/) (2021)
- [Embracing Single Stride 3D Object Detector with Sparse Transformer](https://mlsanity.com/paper/2112.06375/) (2021)
- [RIO: 3D Object Instance Re-Localization in Changing Indoor Environments](https://mlsanity.com/paper/1908.06109/) (2019)
- [Wide-Area Image Geolocalization with Aerial Reference Imagery](https://mlsanity.com/paper/1510.03743/) (2015)
- [Hydra: A Real-time Spatial Perception System for 3D Scene Graph Construction and Optimization](https://mlsanity.com/paper/2201.13360/) (2022)

## Built on by (most cited first, 29 of 29)

- [Open-Vocabulary Functional 3D Scene Graphs for Real-World Indoor Spaces](https://mlsanity.com/paper/2503.19199/) (2025)
- [SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretraining](https://mlsanity.com/paper/2503.18052/) (2025)
- [EgoNight: Towards Egocentric Vision Understanding at Night with a Challenging Benchmark](https://mlsanity.com/paper/2510.06218/) (2025)
- [ViSTA-SLAM: Visual SLAM with Symmetric Two-view Association](https://mlsanity.com/paper/2509.01584/) (2025)
- [V2-SAM: Marrying SAM2 with Multi-Prompt Experts for Cross-View Object Correspondence](https://mlsanity.com/paper/2511.20886/) (2025)
- [CrossOver: 3D Scene Cross-Modal Alignment](https://mlsanity.com/paper/2502.15011/) (2025)
- [Compile Scene Graphs with Reinforcement Learning](https://mlsanity.com/paper/2504.13617/) (2025)
- [Articulate3D: Holistic Understanding of 3D Scenes as Universal Scene Description](https://mlsanity.com/paper/2412.01398/) (2024)
- [Reducing Unimodal Bias in Multi-Modal Semantic Segmentation with Multi-Scale Functional Entropy Regularization](https://mlsanity.com/paper/2505.06635/) (2025)
- [UnLoc: Leveraging Depth Uncertainties for Floorplan Localization](https://mlsanity.com/paper/2509.11301/) (2025)
- [GOTPR: General Outdoor Text-based Place Recognition Using Scene Graph Retrieval with OpenStreetMap](https://mlsanity.com/paper/2501.08575/) (2025)
- [Learning 3D Scene Analogies with Neural Contextual Scene Maps](https://mlsanity.com/paper/2503.15897/) (2025)
- [3D Scene Graphs: Open Challenges and Future Directions](https://mlsanity.com/paper/2606.19383/) (2026)
- [OpenCity3D: What do Vision-Language Models know about Urban Environments?](https://mlsanity.com/paper/2503.16776/) (2025)
- [From Data to Modeling: Fully Open-vocabulary Scene Graph Generation](https://mlsanity.com/paper/2505.20106/) (2025)
- [Masked Clustering Prediction for Unsupervised Point Cloud Pre-training](https://mlsanity.com/paper/2508.08910/) (2025)
- [SGAligner++: Cross-Modal Language-Aided 3D Scene Graph Alignment](https://mlsanity.com/paper/2509.20401/) (2025)
- [The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models](https://mlsanity.com/paper/2601.08876/) (2026)
- [Any Image Restoration via Efficient Spatial-Frequency Degradation Adaptation](https://mlsanity.com/paper/2504.14249/) (2025)
- [Spatiotemporal Knowledge Graphs as Persistent Scene Memory for Embodied Question Answering](https://mlsanity.com/paper/2510.01483/) (2025)
- [ReefNet: A Large-Scale Dataset and Benchmark for Fine-Grained Coral Reef Recognition](https://mlsanity.com/paper/2510.16822/) (2025)
- [HouseLayout3D: A Benchmark and Training-Free Baseline for 3D Layout Estimation in the Wild](https://mlsanity.com/paper/2512.02450/) (2025)
- [Text2Graph VPR: A Text-to-Graph Expert System for Explainable Place Recognition in Changing Environments](https://mlsanity.com/paper/2512.18613/) (2025)
- [PlanaReLoc: Camera Relocalization in 3D Planar Primitives via Region-Based Structure Matching](https://mlsanity.com/paper/2603.20818/) (2026)
- [OpenSGA: Efficient 3D Scene Graph Alignment in the Open World](https://mlsanity.com/paper/2605.10484/) (2026)
- [SG2Loc: Sequential Visual Localization on 3D Scene Graphs](https://mlsanity.com/paper/2606.11880/) (2026)
- [Computing Monetary Risk Measures in Linear Time](https://mlsanity.com/paper/2607.05078/) (2026)
- [CR-Refiner: An Object-Centric Optimal Transport Reranker for Edit-Conditioned 3D Scene Retrieval](https://mlsanity.com/paper/2607.19115/) (2026)
- [VideoReloc: Long-Term Indoor Video Relocalization against a Kilobyte-Scale Semantic Scene Graph](https://mlsanity.com/paper/2609.21804/) (2026)

## Abstract (arXiv)

We introduce a novel problem, i.e., the localization of an input image within a multi-modal reference map represented by a database of 3D scene graphs. These graphs comprise multiple modalities, including object-level point clouds, images, attributes, and relationships between objects, offering a lightweight and efficient alternative to conventional methods that rely on extensive image databases. Given the available modalities, the proposed method SceneGraphLoc learns a fixed-sized embedding for each node (i.e., representing an object instance) in the scene graph, enabling effective matching with the objects visible in the input query image. This strategy significantly outperforms other cross-modal methods, even without incorporating images into the map embeddings. When images are leveraged, SceneGraphLoc achieves performance close to that of state-of-the-art techniques depending on large image databases, while requiring three orders-of-magnitude less storage and operating orders-of-magnitude faster. The code will be made public.