# Self-Improving SLAM in Dynamic Environments: Learning When to Mask

arXiv: https://arxiv.org/abs/2210.08350 · HTML: https://mlsanity.com/paper/2210.08350/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 15 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM](https://mlsanity.com/paper/1812.07976/) (2018)
- [Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments](https://mlsanity.com/paper/1711.06623/) (2017)
- [DOT: Dynamic Object Tracking for Visual SLAM](https://mlsanity.com/paper/2010.00052/) (2020)
- [Asynchronous Multi-View SLAM](https://mlsanity.com/paper/2101.06562/) (2021)
- [Learning to Segment Dynamic Objects using SLAM Outliers](https://mlsanity.com/paper/2011.06259/) (2020)
- [A Front-End for Dense Monocular SLAM using a Learned Outlier Mask Prior](https://mlsanity.com/paper/2104.00562/) (2021)

## Built on by (most cited first, 2 of 2)

- [Elevating Visual Question Answering through Implicitly Learned Reasoning Pathways in LVLMs](https://mlsanity.com/paper/2503.14674/) (2025)
- [Efficient and Robust Camera-independent Multiview 3D Geometric Reconstruction from Noisy Monocular Depth Estimation and Multiple Point Matching](https://mlsanity.com/paper/2609.05972/) (2026)

## Abstract (arXiv)

Visual SLAM - Simultaneous Localization and Mapping - in dynamic environments typically relies on identifying and masking image features on moving objects to prevent them from negatively affecting performance. Current approaches are suboptimal: they either fail to mask objects when needed or, on the contrary, mask objects needlessly. Thus, we propose a novel SLAM that learns when masking objects improves its performance in dynamic scenarios. Given a method to segment objects and a SLAM, we give the latter the ability of Temporal Masking, i.e., to infer when certain classes of objects should be masked to maximize any given SLAM metric. We do not make any priors on motion: our method learns to mask moving objects by itself. To prevent high annotations costs, we created an automatic annotation method for self-supervised training. We constructed a new dataset, named ConsInv, which includes challenging real-world dynamic sequences respectively indoors and outdoors. Our method reaches the state of the art on the TUM RGB-D dataset and outperforms it on KITTI and ConsInv datasets.