# Optimizing SLAM Evaluation Footprint Through Dynamic Range Coverage Analysis of Datasets

arXiv: https://arxiv.org/abs/2209.06316 · HTML: https://mlsanity.com/paper/2209.06316/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 41 references that are in the map.
- First posted on arXiv 13 Sep 2022.

## Builds on (most cited first, 9 of 9)

- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [Statistical Aspects of Wasserstein Distances](https://mlsanity.com/paper/1806.05500/) (2018)
- [The TUM VI Benchmark for Evaluating Visual-Inertial Odometry](https://mlsanity.com/paper/1804.06120/) (2018)
- [Assessing and Remedying Coverage for a Given Dataset](https://mlsanity.com/paper/1810.06742/) (2018)
- [Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems](https://mlsanity.com/paper/2109.12343/) (2021)
- [Simultaneous Localization and Mapping Related Datasets: A Comprehensive Survey](https://mlsanity.com/paper/2102.04036/) (2021)
- [Coverage Testing of Deep Learning Models using Dataset Characterization](https://mlsanity.com/paper/1911.07309/) (2019)
- [Are We Ready for Robust and Resilient SLAM? A Framework For Quantitative Characterization of SLAM Datasets](https://mlsanity.com/paper/2202.11312/) (2022)

## Abstract (arXiv)

Simultaneous Localization and Mapping (SLAM) is considered an ever-evolving problem due to its usage in many applications. Evaluation of SLAM is done typically using publicly available datasets which are increasing in number and the level of difficulty. Each dataset provides a certain level of dynamic range coverage that is a key aspect of measuring the robustness and resilience of SLAM. In this paper, we provide a systematic analysis of the dynamic range coverage of datasets based on a number of characterization metrics, and our analysis shows a huge level of redundancy within and between datasets. Subsequently, we propose a dynamic programming (DP) algorithm for eliminating the redundancy in the evaluation process of SLAM by selecting a subset of sequences that matches a single or multiple dynamic range coverage objectives. It is shown that, with the help of dataset characterization and DP selection algorithm, a reduction in the evaluation effort can be achieved while maintaining the same level of coverage. We also study how the evaluation process of a real-world SLAM system can be optimized utilizing the method proposed.