# TOPO-Bench: An Open-Source Topological Mapping Evaluation Framework with Quantifiable Perceptual Aliasing

arXiv: https://arxiv.org/abs/2510.04100 · HTML: https://mlsanity.com/paper/2510.04100/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 32 references that are in the map.
- First posted on arXiv 5 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [Habitat: A Platform for Embodied AI Research](https://mlsanity.com/paper/1904.01201/) (2019)
- [NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration](https://mlsanity.com/paper/2310.07896/) (2023)
- [Semi-parametric Topological Memory for Navigation](https://mlsanity.com/paper/1803.00653/) (2018)
- [RELLIS-3D Dataset: Data, Benchmarks and Analysis](https://mlsanity.com/paper/2011.12954/) (2020)
- [EigenPlaces: Training Viewpoint Robust Models for Visual Place Recognition](https://mlsanity.com/paper/2308.10832/) (2023)
- [BoQ: A Place is Worth a Bag of Learnable Queries](https://mlsanity.com/paper/2405.07364/) (2024)
- [MegaLoc: One Retrieval to Place Them All](https://mlsanity.com/paper/2502.17237/) (2025)
- [Are We Ready for Service Robots? The OpenLORIS-Scene Datasets for Lifelong SLAM](https://mlsanity.com/paper/1911.05603/) (2019)
- [Scaling Local Control to Large-Scale Topological Navigation](https://mlsanity.com/paper/1909.12329/) (2019)
- [Probabilistic Visual Place Recognition for Hierarchical Localization](https://mlsanity.com/paper/2105.03091/) (2021)
- [Topomap: Topological Mapping and Navigation Based on Visual SLAM Maps](https://mlsanity.com/paper/1709.05533/) (2017)
- [PlaceNav: Topological Navigation through Place Recognition](https://mlsanity.com/paper/2309.17260/) (2023)
- [GeNIE: A Generalizable Navigation System for In-the-Wild Environments](https://mlsanity.com/paper/2506.17960/) (2025)
- [Sparse 3D Topological Graphs for Micro-Aerial Vehicle Planning](https://mlsanity.com/paper/1803.04345/) (2018)
- [Probable Object Location (POLo) Score Estimation for Efficient Object Goal Navigation](https://mlsanity.com/paper/2311.07992/) (2023)
- [ROVER: A Multi-Season Dataset for Visual SLAM](https://mlsanity.com/paper/2412.02506/) (2024)

## Abstract (arXiv)

Topological mapping offers a compact and robust representation for navigation, but progress in the field is hindered by the lack of standardized evaluation metrics, datasets, and protocols. Existing systems are assessed using different environments and criteria, preventing fair and reproducible comparisons. Moreover, a key challenge - perceptual aliasing - remains under-quantified, despite its strong influence on system performance. We address these gaps by (1) formalizing topological consistency as the fundamental property of topological maps and showing that localization accuracy provides an efficient and interpretable surrogate metric, and (2) proposing the first quantitative measure of dataset ambiguity to enable fair comparisons across environments. To support this protocol, we curate a diverse benchmark dataset with calibrated ambiguity levels, implement and release deep-learned baseline systems, and evaluate them alongside classical methods. Our experiments and analysis yield new insights into the limitations of current approaches under perceptual aliasing. All datasets, baselines, and evaluation tools are fully open-sourced to foster consistent and reproducible research in topological mapping.