# Scene-level Tracking and Reconstruction without Object Priors

arXiv: https://arxiv.org/abs/2210.03815 · HTML: https://mlsanity.com/paper/2210.03815/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 24 references that are in the map.
- First posted on arXiv 7 Oct 2022.

## Builds on (most cited first, 7 of 7)

- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM](https://mlsanity.com/paper/1812.07976/) (2018)
- [Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects](https://mlsanity.com/paper/1706.06629/) (2017)
- [SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction](https://mlsanity.com/paper/1904.13073/) (2019)
- [Topology-Aware Non-Rigid Point Cloud Registration](https://mlsanity.com/paper/1811.07014/) (2018)
- [GeoFusion: Geometric Consistency informed Scene Estimation in Dense Clutter](https://mlsanity.com/paper/2003.12610/) (2020)
- [Topology-Change-Aware Volumetric Fusion for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2007.06853/) (2020)

## Built on by (most cited first, 4 of 4)

- [Context-Aware Entity Grounding with Open-Vocabulary 3D Scene Graphs](https://mlsanity.com/paper/2309.15940/) (2023)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Mono-STAR: Mono-camera Scene-level Tracking and Reconstruction](https://mlsanity.com/paper/2301.13244/) (2023)
- [Motion Blender Gaussian Splatting for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2503.09040/) (2025)

## Abstract (arXiv)

We present the first real-time system capable of tracking and reconstructing, individually, every visible object in a given scene, without any form of prior on the rigidness of the objects, texture existence, or object category. In contrast with previous methods such as Co-Fusion and MaskFusion that first segment the scene into individual objects and then process each object independently, the proposed method dynamically segments the non-rigid scene as part of the tracking and reconstruction process. When new measurements indicate topology change, reconstructed models are updated in real-time to reflect that change. Our proposed system can provide the live geometry and deformation of all visible objects in a novel scene in real-time, which makes it possible to be integrated seamlessly into numerous existing robotics applications that rely on object models for grasping and manipulation. The capabilities of the proposed system are demonstrated in challenging scenes that contain multiple rigid and non-rigid objects.