# Visual-Inertial Multi-Instance Dynamic SLAM with Object-level Relocalisation

arXiv: https://arxiv.org/abs/2208.04274 · HTML: https://mlsanity.com/paper/2208.04274/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 8 Aug 2022.

## Builds on (most cited first, 19 of 19)

- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [ORB-SLAM2: an Open-Source SLAM System for Monocular, Stereo and RGB-D Cameras](https://mlsanity.com/paper/1610.06475/) (2016)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping](https://mlsanity.com/paper/1910.02490/) (2019)
- [Rearrangement: A Challenge for Embodied AI](https://mlsanity.com/paper/2011.01975/) (2020)
- [DynaSLAM: Tracking, Mapping and Inpainting in Dynamic Scenes](https://mlsanity.com/paper/1806.05620/) (2018)
- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM](https://mlsanity.com/paper/1812.07976/) (2018)
- [Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects](https://mlsanity.com/paper/1706.06629/) (2017)
- [DynaSLAM II: Tightly-Coupled Multi-Object Tracking and SLAM](https://mlsanity.com/paper/2010.07820/) (2020)
- [OKVIS2: Realtime Scalable Visual-Inertial SLAM with Loop Closure](https://mlsanity.com/paper/2202.09199/) (2022)
- [EM-Fusion: Dynamic Object-Level SLAM with Probabilistic Data Association](https://mlsanity.com/paper/1904.11781/) (2019)
- [Driven to Distraction: Self-Supervised Distractor Learning for Robust Monocular Visual Odometry in Urban Environments](https://mlsanity.com/paper/1711.06623/) (2017)
- [ClusterVO: Clustering Moving Instances and Estimating Visual Odometry for Self and Surroundings](https://mlsanity.com/paper/2003.12980/) (2020)
- [A Primer on the Differential Calculus of 3D Orientations](https://mlsanity.com/paper/1606.05285/) (2016)
- [Visual-Inertial Object Detection and Mapping](https://mlsanity.com/paper/1806.08498/) (2018)
- [OrcVIO: Object residual constrained Visual-Inertial Odometry](https://mlsanity.com/paper/2007.15107/) (2020)
- [Dense RGB-D-Inertial SLAM with Map Deformations](https://mlsanity.com/paper/2207.10940/) (2022)
- [DetectFusion: Detecting and Segmenting Both Known and Unknown Dynamic Objects in Real-time SLAM](https://mlsanity.com/paper/1907.09127/) (2019)

## Built on by (most cited first, 2 of 2)

- [SeMLaPS: Real-time Semantic Mapping with Latent Prior Networks and Quasi-Planar Segmentation](https://mlsanity.com/paper/2306.16585/) (2023)
- [An Earth Rover dataset recorded at the ICRA@40 party](https://mlsanity.com/paper/2407.05735/) (2024)

## Abstract (arXiv)

In this paper, we present a tightly-coupled visual-inertial object-level multi-instance dynamic SLAM system. Even in extremely dynamic scenes, it can robustly optimise for the camera pose, velocity, IMU biases and build a dense 3D reconstruction object-level map of the environment. Our system can robustly track and reconstruct the geometries of arbitrary objects, their semantics and motion by incrementally fusing associated colour, depth, semantic, and foreground object probabilities into each object model thanks to its robust sensor and object tracking. In addition, when an object is lost or moved outside the camera field of view, our system can reliably recover its pose upon re-observation. We demonstrate the robustness and accuracy of our method by quantitatively and qualitatively testing it in real-world data sequences.