# Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

arXiv: https://arxiv.org/abs/2006.10178 · HTML: https://mlsanity.com/paper/2006.10178/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 6 of 6)

- [Mastering Diverse Domains through World Models](https://mlsanity.com/paper/2301.04104/) (2023)
- [AgentLens: Visual Analysis for Agent Behaviors in LLM-based Autonomous Systems](https://mlsanity.com/paper/2402.08995/) (2024)
- [Information is Power: Intrinsic Control via Information Capture](https://mlsanity.com/paper/2112.03899/) (2021)
- [The Role of Predictive Uncertainty and Diversity in Embodied AI and Robot Learning](https://mlsanity.com/paper/2405.03164/) (2024)
- [Tracking and Planning with Spatial World Models](https://mlsanity.com/paper/2201.10335/) (2022)
- [PRISM: Probabilistic Real-Time Inference in Spatial World Models](https://mlsanity.com/paper/2212.02988/) (2022)

## Abstract (arXiv)

We solve the problem of 6-DoF localisation and 3D dense reconstruction in spatial environments as approximate Bayesian inference in a deep state-space model. Our approach leverages both learning and domain knowledge from multiple-view geometry and rigid-body dynamics. This results in an expressive predictive model of the world, often missing in current state-of-the-art visual SLAM solutions. The combination of variational inference, neural networks and a differentiable raycaster ensures that our model is amenable to end-to-end gradient-based optimisation. We evaluate our approach on realistic unmanned aerial vehicle flight data, nearing the performance of state-of-the-art visual-inertial odometry systems. We demonstrate the applicability of the model to generative prediction and planning.