# Lost in Tracking Translation: A Comprehensive Analysis of Visual SLAM in Human-Centered XR and IoT Ecosystems

arXiv: https://arxiv.org/abs/2411.07146 · HTML: https://mlsanity.com/paper/2411.07146/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 35 of its 153 references that are in the map.
- First posted on arXiv 11 Nov 2024.

## Builds on (most cited first, 30 of 35)

- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Unsupervised Learning of Depth and Ego-Motion from Video](https://mlsanity.com/paper/1704.07813/) (2017)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks](https://mlsanity.com/paper/1709.08429/) (2017)
- [CodeSLAM - Learning a Compact, Optimisable Representation for Dense Visual SLAM](https://mlsanity.com/paper/1804.00874/) (2018)
- [TartanVO: A Generalizable Learning-based VO](https://mlsanity.com/paper/2011.00359/) (2020)
- [The TUM VI Benchmark for Evaluating Visual-Inertial Odometry](https://mlsanity.com/paper/1804.06120/) (2018)
- [DeepTAM: Deep Tracking and Mapping](https://mlsanity.com/paper/1808.01900/) (2018)
- [Visual Odometry Revisited: What Should Be Learnt?](https://mlsanity.com/paper/1909.09803/) (2019)
- [Self-Supervised Deep Visual Odometry with Online Adaptation](https://mlsanity.com/paper/2005.06136/) (2020)
- [A Survey on Deep Learning for Localization and Mapping: Towards the Age of Spatial Machine Intelligence](https://mlsanity.com/paper/2006.12567/) (2020)
- [HoloLens 2 Research Mode as a Tool for Computer Vision Research](https://mlsanity.com/paper/2008.11239/) (2020)
- [The Revisiting Problem in Simultaneous Localization and Mapping: A Survey on Visual Loop Closure Detection](https://mlsanity.com/paper/2204.12831/) (2022)
- [gradSLAM: Automagically differentiable SLAM](https://mlsanity.com/paper/1910.10672/) (2019)
- [Online Temporal Calibration for Monocular Visual-Inertial Systems](https://mlsanity.com/paper/1808.00692/) (2018)
- [Direct Sparse Mapping](https://mlsanity.com/paper/1904.06577/) (2019)
- [ADVIO: An authentic dataset for visual-inertial odometry](https://mlsanity.com/paper/1807.09828/) (2018)
- [OxIOD: The Dataset for Deep Inertial Odometry](https://mlsanity.com/paper/1809.07491/) (2018)
- [RGB-D SLAM with Structural Regularities](https://mlsanity.com/paper/2010.07997/) (2020)
- [Measuring robustness of Visual SLAM](https://mlsanity.com/paper/1910.04755/) (2019)
- [Keyframe-based monocular SLAM: design, survey, and future directions](https://mlsanity.com/paper/1607.00470/) (2016)
- [DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction](https://mlsanity.com/paper/1908.03918/) (2019)
- [StructVIO : Visual-inertial Odometry with Structural Regularity of Man-made Environments](https://mlsanity.com/paper/1810.06796/) (2018)
- [Self-Supervised 3D Keypoint Learning for Ego-motion Estimation](https://mlsanity.com/paper/1912.03426/) (2019)
- [Approaches, Challenges, and Applications for Deep Visual Odometry: Toward to Complicated and Emerging Areas](https://mlsanity.com/paper/2009.02672/) (2020)
- [LF-VIO: A Visual-Inertial-Odometry Framework for Large Field-of-View Cameras with Negative Plane](https://mlsanity.com/paper/2202.12613/) (2022)
- [Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM](https://mlsanity.com/paper/1410.2167/) (2014)
- [Simultaneous Localization and Mapping Related Datasets: A Comprehensive Survey](https://mlsanity.com/paper/2102.04036/) (2021)
- [Evaluation of RGB-D SLAM in Large Indoor Environments](https://mlsanity.com/paper/2212.05980/) (2022)

## Abstract (arXiv)

Advancements in tracking algorithms have empowered nascent applications across various domains, from steering autonomous vehicles to guiding robots to enhancing augmented reality experiences for users. However, these algorithms are application-specific and do not work across applications with different types of motion; even a tracking algorithm designed for a given application does not work in scenarios deviating from highly standard conditions. For example, a tracking algorithm designed for robot navigation inside a building will not work for tracking the same robot in an outdoor environment. To demonstrate this problem, we evaluate the performance of the state-of-the-art tracking methods across various applications and scenarios. To inform our analysis, we first categorize algorithmic, environmental, and locomotion-related challenges faced by tracking algorithms. We quantitatively evaluate the performance using multiple tracking algorithms and representative datasets for a wide range of Internet of Things (IoT) and Extended Reality (XR) applications, including autonomous vehicles, drones, and humans. Our analysis shows that no tracking algorithm works across different applications and scenarios within applications. Ultimately, using the insights generated from our analysis, we discuss multiple approaches to improving the tracking performance using input data characterization, leveraging intermediate information, and output evaluation.