# Open-Set Semantic Uncertainty Aware Metric-Semantic Graph Matching

arXiv: https://arxiv.org/abs/2409.11555 · HTML: https://mlsanity.com/paper/2409.11555/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 17 Sep 2024.

## Builds on (most cited first, 19 of 19)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Neural Graph Matching Network: Learning Lawler's Quadratic Assignment Problem with Extension to Hypergraph and Multiple-graph Matching](https://mlsanity.com/paper/1911.11308/) (2019)
- [Semantic Graph Based Place Recognition for 3D Point Clouds](https://mlsanity.com/paper/2008.11459/) (2020)
- [Sketching Curvature for Efficient Out-of-Distribution Detection for Deep Neural Networks](https://mlsanity.com/paper/2102.12567/) (2021)
- [CLIPPER: A Graph-Theoretic Framework for Robust Data Association](https://mlsanity.com/paper/2011.10202/) (2020)
- [URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates](https://mlsanity.com/paper/2307.03810/) (2023)
- [Probabilistic Data Association via Mixture Models for Robust Semantic SLAM](https://mlsanity.com/paper/1909.11213/) (2019)
- [Pretrained Visual Uncertainties](https://mlsanity.com/paper/2402.16569/) (2024)
- [SemanticLoop: loop closure with 3D semantic graph matching](https://mlsanity.com/paper/2211.11977/) (2022)
- [GOReloc: Graph-based Object-Level Relocalization for Visual SLAM](https://mlsanity.com/paper/2408.07917/) (2024)
- [Multi S-Graphs: An Efficient Distributed Semantic-Relational Collaborative SLAM](https://mlsanity.com/paper/2401.05152/) (2024)
- [SOS-Match: Segmentation for Open-Set Robust Correspondence Search and Robot Localization in Unstructured Environments](https://mlsanity.com/paper/2401.04791/) (2024)
- [Efficient Constellation-Based Map-Merging for Semantic SLAM](https://mlsanity.com/paper/1809.09646/) (2018)
- [SemanticTopoLoop: Semantic Loop Closure With 3D Topological Graph Based on Quadric-Level Object Map](https://mlsanity.com/paper/2311.02831/) (2023)
- [Opti-Acoustic Semantic SLAM with Unknown Objects in Underwater Environments](https://mlsanity.com/paper/2403.12837/) (2024)
- [LOSS-SLAM: Lightweight Open-Set Semantic Simultaneous Localization and Mapping](https://mlsanity.com/paper/2404.04377/) (2024)
- [Loop Closure Detection Based on Object-level Spatial Layout and Semantic Consistency](https://mlsanity.com/paper/2304.05146/) (2023)

## Built on by (most cited first, 1 of 1)

- [LunarLoc: Segment-Based Global Localization on the Moon](https://mlsanity.com/paper/2506.16940/) (2025)

## Abstract (arXiv)

Underwater object-level mapping requires incorporating visual foundation models to handle the uncommon and often previously unseen object classes encountered in marine scenarios. In this work, a metric of semantic uncertainty for open-set object detections produced by visual foundation models is calculated and then incorporated into an object-level uncertainty tracking framework. Object-level uncertainties and geometric relationships between objects are used to enable robust object-level loop closure detection for unknown object classes. The above loop closure detection problem is formulated as a graph-matching problem. While graph matching, in general, is NP-Complete, a solver for an equivalent formulation of the proposed graph matching problem as a graph editing problem is tested on multiple challenging underwater scenes. Results for this solver as well as three other solvers demonstrate that the proposed methods are feasible for real-time use in marine environments for the robust, open-set, multi-object, semantic-uncertainty-aware loop closure detection. Further experimental results on the KITTI dataset demonstrate that the method generalizes to large-scale terrestrial scenes.