# Semi-Supervised Semantic Mapping through Label Propagation with Semantic Texture Meshes

arXiv: https://arxiv.org/abs/1906.07029 · HTML: https://mlsanity.com/paper/1906.07029/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2019.

## Built on by (most cited first, 9 of 9)

- [Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs](https://mlsanity.com/paper/2101.06894/) (2021)
- [Leveraging Large (Visual) Language Models for Robot 3D Scene Understanding](https://mlsanity.com/paper/2209.05629/) (2022)
- [Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles with Label Propagation for Cross-Domain Adaptation](https://mlsanity.com/paper/2210.09739/) (2022)
- [Pix2Map: Cross-modal Retrieval for Inferring Street Maps from Images](https://mlsanity.com/paper/2301.04224/) (2023)
- [Self-supervised Pre-training for Semantic Segmentation in an Indoor Scene](https://mlsanity.com/paper/2210.01884/) (2022)
- [MCDS-VSS: Moving Camera Dynamic Scene Video Semantic Segmentation by Filtering with Self-Supervised Geometry and Motion](https://mlsanity.com/paper/2405.19921/) (2024)
- [Real-Time Multi-Modal Semantic Fusion on Unmanned Aerial Vehicles](https://mlsanity.com/paper/2108.06608/) (2021)
- [Improving Semantic Image Segmentation via Label Fusion in Semantically Textured Meshes](https://mlsanity.com/paper/2111.11103/) (2021)
- [Under-Canopy Terrain Reconstruction in Dense Forests Using RGB Imaging and Neural 3D Reconstruction](https://mlsanity.com/paper/2601.22861/) (2026)

## Abstract (arXiv)

Scene understanding is an important capability for robots acting in unstructured environments. While most SLAM approaches provide a geometrical representation of the scene, a semantic map is necessary for more complex interactions with the surroundings. Current methods treat the semantic map as part of the geometry which limits scalability and accuracy. We propose to represent the semantic map as a geometrical mesh and a semantic texture coupled at independent resolution. The key idea is that in many environments the geometry can be greatly simplified without loosing fidelity, while semantic information can be stored at a higher resolution, independent of the mesh. We construct a mesh from depth sensors to represent the scene geometry and fuse information into the semantic texture from segmentations of individual RGB views of the scene. Making the semantics persistent in a global mesh enables us to enforce temporal and spatial consistency of the individual view predictions. For this, we propose an efficient method of establishing consensus between individual segmentations by iteratively retraining semantic segmentation with the information stored within the map and using the retrained segmentation to re-fuse the semantics. We demonstrate the accuracy and scalability of our approach by reconstructing semantic maps of scenes from NYUv2 and a scene spanning large buildings.