# Customizable Perturbation Synthesis for Robust SLAM Benchmarking

arXiv: https://arxiv.org/abs/2402.08125 · HTML: https://mlsanity.com/paper/2402.08125/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 81 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [ProcTHOR: Large-Scale Embodied AI Using Procedural Generation](https://mlsanity.com/paper/2206.06994/) (2022)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [NeRF-SLAM: Real-Time Dense Monocular SLAM with Neural Radiance Fields](https://mlsanity.com/paper/2210.13641/) (2022)
- [TartanVO: A Generalizable Learning-based VO](https://mlsanity.com/paper/2011.00359/) (2020)
- [SubT-MRS Dataset: Pushing SLAM Towards All-weather Environments](https://mlsanity.com/paper/2307.07607/) (2023)
- [Multi-Camera LiDAR Inertial Extension to the Newer College Dataset](https://mlsanity.com/paper/2112.08854/) (2021)
- [Resilient and Distributed Multi-Robot Visual SLAM: Datasets, Experiments, and Lessons Learned](https://mlsanity.com/paper/2304.04362/) (2023)
- [D2SLAM: Decentralized and Distributed Collaborative Visual-inertial SLAM System for Aerial Swarm](https://mlsanity.com/paper/2211.01538/) (2022)
- [Task-Aware Risk Estimation of Perception Failures for Autonomous Vehicles](https://mlsanity.com/paper/2305.01870/) (2023)

## Built on by (most cited first, 6 of 6)

- [How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey](https://mlsanity.com/paper/2402.13255/) (2024)
- [Is Your LiDAR Placement Optimized for 3D Scene Understanding?](https://mlsanity.com/paper/2403.17009/) (2024)
- [RecGS: Removing Water Caustic with Recurrent Gaussian Splatting](https://mlsanity.com/paper/2407.10318/) (2024)
- [R2-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations](https://mlsanity.com/paper/2403.04924/) (2024)
- [From NeRFs to Gaussian Splats, and Back](https://mlsanity.com/paper/2405.09717/) (2024)
- [Does Dynamic-Point Filtering Help When Texture Is Scarce? A Controlled Study of ORB-SLAM2 Front-Ends in Synthetic Indoor Scenes](https://mlsanity.com/paper/2610.10564/) (2026)

## Abstract (arXiv)

Robustness is a crucial factor for the successful deployment of robots in unstructured environments, particularly in the domain of Simultaneous Localization and Mapping (SLAM). Simulation-based benchmarks have emerged as a highly scalable approach for robustness evaluation compared to real-world data collection. However, crafting a challenging and controllable noisy world with diverse perturbations remains relatively under-explored. To this end, we propose a novel, customizable pipeline for noisy data synthesis, aimed at assessing the resilience of multi-modal SLAM models against various perturbations. This pipeline incorporates customizable hardware setups, software components, and perturbed environments. In particular, we introduce comprehensive perturbation taxonomy along with a perturbation composition toolbox, allowing the transformation of clean simulations into challenging noisy environments. Utilizing the pipeline, we instantiate the Robust-SLAM benchmark, which includes diverse perturbation types, to evaluate the risk tolerance of existing advanced multi-modal SLAM models. Our extensive analysis uncovers the susceptibilities of existing SLAM models to real-world disturbance, despite their demonstrated accuracy in standard benchmarks. Our perturbation synthesis toolbox, SLAM robustness evaluation pipeline, and Robust-SLAM benchmark will be made publicly available at https://github.com/Xiaohao-Xu/SLAM-under-Perturbation/.