# TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

arXiv: https://arxiv.org/abs/2105.07468 · HTML: https://mlsanity.com/paper/2105.07468/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 25 references that are in the map.
- First posted on arXiv 16 May 2021.

## Builds on (most cited first, 17 of 17)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning](https://mlsanity.com/paper/1611.03631/) (2016)
- [SemanticFusion: Dense 3D Semantic Mapping with Convolutional Neural Networks](https://mlsanity.com/paper/1609.05130/) (2016)
- [Fusion++: Volumetric Object-Level SLAM](https://mlsanity.com/paper/1808.08378/) (2018)
- [MaskFusion: Real-Time Recognition, Tracking and Reconstruction of Multiple Moving Objects](https://mlsanity.com/paper/1804.09194/) (2018)
- [Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery](https://mlsanity.com/paper/1903.00268/) (2019)
- [VolumeDeform: Real-time Volumetric Non-rigid Reconstruction](https://mlsanity.com/paper/1603.08161/) (2016)
- [MID-Fusion: Octree-based Object-Level Multi-Instance Dynamic SLAM](https://mlsanity.com/paper/1812.07976/) (2018)
- [Co-Fusion: Real-time Segmentation, Tracking and Fusion of Multiple Objects](https://mlsanity.com/paper/1706.06629/) (2017)
- [Meaningful Maps With Object-Oriented Semantic Mapping](https://mlsanity.com/paper/1609.07849/) (2016)
- [EM-Fusion: Dynamic Object-Level SLAM with Probabilistic Data Association](https://mlsanity.com/paper/1904.11781/) (2019)
- [Robust Dense Mapping for Large-Scale Dynamic Environments](https://mlsanity.com/paper/1905.02781/) (2019)
- [Real-time Progressive 3D Semantic Segmentation for Indoor Scene](https://mlsanity.com/paper/1804.00257/) (2018)
- [Incremental Object Database: Building 3D Models from Multiple Partial Observations](https://mlsanity.com/paper/1808.00760/) (2018)
- [Live Reconstruction of Large-Scale Dynamic Outdoor Worlds](https://mlsanity.com/paper/1903.06708/) (2019)

## Built on by (most cited first, 4 of 4)

- [Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimation](https://mlsanity.com/paper/2512.23705/) (2025)
- [Neural Implicit Vision-Language Feature Fields](https://mlsanity.com/paper/2303.10962/) (2023)
- [Object-level 3D Semantic Mapping using a Network of Smart Edge Sensors](https://mlsanity.com/paper/2211.11354/) (2022)
- [TivNe-SLAM: Dynamic Mapping and Tracking via Time-Varying Neural Radiance Fields](https://mlsanity.com/paper/2310.18917/) (2023)

## Abstract (arXiv)

The ability to simultaneously track and reconstruct multiple objects moving in the scene is of the utmost importance for robotic tasks such as autonomous navigation and interaction. Virtually all of the previous attempts to map multiple dynamic objects have evolved to store individual objects in separate reconstruction volumes and track the relative pose between them. While simple and intuitive, such formulation does not scale well with respect to the number of objects in the scene and introduces the need for an explicit occlusion handling strategy. In contrast, we propose a map representation that allows maintaining a single volume for the entire scene and all the objects therein. To this end, we introduce a novel multi-object TSDF formulation that can encode multiple object surfaces at any given location in the map. In a multiple dynamic object tracking and reconstruction scenario, our representation allows maintaining accurate reconstruction of surfaces even while they become temporarily occluded by other objects moving in their proximity. We evaluate the proposed TSDF++ formulation on a public synthetic dataset and demonstrate its ability to preserve reconstructions of occluded surfaces when compared to the standard TSDF map representation.