# Rendering the Directional TSDF for Tracking and Multi-Sensor Registration with Point-To-Plane Scale ICP

arXiv: https://arxiv.org/abs/2301.12796 · HTML: https://mlsanity.com/paper/2301.12796/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 55 references that are in the map.
- First posted on arXiv 30 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [iMAP: Implicit Mapping and Positioning in Real-Time](https://mlsanity.com/paper/2103.12352/) (2021)
- [Neural RGB-D Surface Reconstruction](https://mlsanity.com/paper/2104.04532/) (2021)
- [TEASER: Fast and Certifiable Point Cloud Registration](https://mlsanity.com/paper/2001.07715/) (2020)
- [ReFusion: 3D Reconstruction in Dynamic Environments for RGB-D Cameras Exploiting Residuals](https://mlsanity.com/paper/1905.02082/) (2019)
- [Voxblox: Incremental 3D Euclidean Signed Distance Fields for On-Board MAV Planning](https://mlsanity.com/paper/1611.03631/) (2016)
- [Hierarchical Deep Stereo Matching on High-resolution Images](https://mlsanity.com/paper/1912.06704/) (2019)
- [BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integration](https://mlsanity.com/paper/1604.01093/) (2016)
- [InfiniTAM v3: A Framework for Large-Scale 3D Reconstruction with Loop Closure](https://mlsanity.com/paper/1708.00783/) (2017)
- [PSDF Fusion: Probabilistic Signed Distance Function for On-the-fly 3D Data Fusion and Scene Reconstruction](https://mlsanity.com/paper/1807.11034/) (2018)
- [Redesigning SLAM for Arbitrary Multi-Camera Systems](https://mlsanity.com/paper/2003.02014/) (2020)
- [An Efficient Volumetric Mesh Representation for Real-time Scene Reconstruction using Spatial Hashing](https://mlsanity.com/paper/1803.03949/) (2018)
- [Freetures: Localization in Signed Distance Function Maps](https://mlsanity.com/paper/2010.09378/) (2020)
- [stdgpu: Efficient STL-like Data Structures on the GPU](https://mlsanity.com/paper/1908.05936/) (2019)
- [Directional TSDF: Modeling Surface Orientation for Coherent Meshes](https://mlsanity.com/paper/1908.05146/) (2019)

## Abstract (arXiv)

Dense real-time tracking and mapping from RGB-D images is an important tool for many robotic applications, such as navigation and manipulation. The recently presented Directional Truncated Signed Distance Function (DTSDF) is an augmentation of the regular TSDF that shows potential for more coherent maps and improved tracking performance. In this work, we present methods for rendering depth- and color images from the DTSDF, making it a true drop-in replacement for the regular TSDF in established trackers. We evaluate the algorithm on well-established datasets and observe that our method improves tracking performance and increases re-usability of mapped scenes. Furthermore, we add color integration which notably improves color-correctness at adjacent surfaces. Our novel formulation of combined ICP with frame-to-keyframe photometric error minimization further improves tracking results. Lastly, we introduce Sim3 point-to-plane ICP for refining pose priors in a multi-sensor scenario with different scale factors.