# Semantic Scene Segmentation for Robotics

arXiv: https://arxiv.org/abs/2401.07589 · HTML: https://mlsanity.com/paper/2401.07589/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2024.

## Built on by (most cited first, 16 of 16)

- [CAFuser: Condition-Aware Multimodal Fusion for Robust Semantic Perception of Driving Scenes](https://mlsanity.com/paper/2410.10791/) (2024)
- [DITTO: Demonstration Imitation by Trajectory Transformation](https://mlsanity.com/paper/2403.15203/) (2024)
- [BEVCar: Camera-Radar Fusion for BEV Map and Object Segmentation](https://mlsanity.com/paper/2403.11761/) (2024)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [A Point-Based Approach to Efficient LiDAR Multi-Task Perception](https://mlsanity.com/paper/2404.12798/) (2024)
- [Perception Matters: Enhancing Embodied AI with Uncertainty-Aware Semantic Segmentation](https://mlsanity.com/paper/2408.02297/) (2024)
- [VideoSAM: Open-World Video Segmentation](https://mlsanity.com/paper/2410.08781/) (2024)
- [Imagine2touch: Predictive Tactile Sensing for Robotic Manipulation using Efficient Low-Dimensional Signals](https://mlsanity.com/paper/2405.01192/) (2024)
- [DiSCO-3D : Discovering and segmenting Sub-Concepts from Open-vocabulary queries in NeRF](https://mlsanity.com/paper/2507.14596/) (2025)
- [STAIR: Semantic-Targeted Active Implicit Reconstruction](https://mlsanity.com/paper/2403.11233/) (2024)
- [LetsMap: Unsupervised Representation Learning for Semantic BEV Mapping](https://mlsanity.com/paper/2405.18852/) (2024)
- [Panoptic-Depth Forecasting](https://mlsanity.com/paper/2409.12008/) (2024)
- [PseudoTouch: Efficiently Imaging the Surface Feel of Objects for Robotic Manipulation](https://mlsanity.com/paper/2403.15107/) (2024)
- [FIN: Fast Inference Network for Map Segmentation](https://mlsanity.com/paper/2510.00651/) (2025)
- [Multi-Objective Optimization for Synthetic-to-Real Style Transfer](https://mlsanity.com/paper/2602.03625/) (2026)
- [Contextrast++: Robust Multi-Scale Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2608.22679/) (2026)

## Abstract (arXiv)

Comprehensive scene understanding is a critical enabler of robot autonomy. Semantic segmentation is one of the key scene understanding tasks which is pivotal for several robotics applications including autonomous driving, domestic service robotics, last mile delivery, amongst many others. Semantic segmentation is a dense prediction task that aims to provide a scene representation in which each pixel of an image is assigned a semantic class label. Therefore, semantic segmentation considers the full scene context, incorporating the object category, location, and shape of all the scene elements, including the background. Numerous algorithms have been proposed for semantic segmentation over the years. However, the recent advances in deep learning combined with the boost in the computational capacity and the availability of large-scale labeled datasets have led to significant advances in semantic segmentation. In this chapter, we introduce the task of semantic segmentation and present the deep learning techniques that have been proposed to address this task over the years. We first define the task of semantic segmentation and contrast it with other closely related scene understanding problems. We detail different algorithms and architectures for semantic segmentation and the commonly employed loss functions. Furthermore, we present an overview of datasets, benchmarks, and metrics that are used in semantic segmentation. We conclude the chapter with a discussion of challenges and opportunities for further research in this area.