# From Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping

arXiv: https://arxiv.org/abs/1812.11866 · HTML: https://mlsanity.com/paper/1812.11866/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Dec 2018.

## Built on by (most cited first, 6 of 6)

- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Towards Optimal Correlational Object Search](https://mlsanity.com/paper/2110.09991/) (2021)
- [Mobi-π: Mobilizing Your Robot Learning Policy](https://mlsanity.com/paper/2505.23692/) (2025)
- [QueSTMaps: Queryable Semantic Topological Maps for 3D Scene Understanding](https://mlsanity.com/paper/2404.06442/) (2024)
- [CoFi: Coarse-to-Fine ICP for LiDAR Localization in an Efficient Long-lasting Point Cloud Map](https://mlsanity.com/paper/2110.10194/) (2021)

## Abstract (arXiv)

We introduce TopoNets, end-to-end probabilistic deep networks for modeling semantic maps with structure reflecting the topology of large-scale environments. TopoNets build a unified deep network spanning multiple levels of abstraction and spatial scales, from pixels representing geometry of local places to high-level descriptions of semantics of buildings. To this end, TopoNets leverage complex spatial relations expressed in terms of arbitrary, dynamic graphs. We demonstrate how TopoNets can be used to perform end-to-end semantic mapping from partial sensory observations and noisy topological relations discovered by a robot exploring large-scale office spaces. Thanks to their probabilistic nature and generative properties, TopoNets extend the problem of semantic mapping beyond classification. We show that TopoNets successfully perform uncertain reasoning about yet unexplored space and detect novel and incongruent environment configurations unknown to the robot. Our implementation of TopoNets achieves real-time, tractable and exact inference, which makes these new deep models a promising, practical solution to mobile robot spatial understanding at scale.