# Uncertainty-Aware Self-Supervised Learning of Spatial Perception Tasks

arXiv: https://arxiv.org/abs/2103.12007 · HTML: https://mlsanity.com/paper/2103.12007/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Mar 2021.

## Built on by (most cited first, 8 of 8)

- [Physics perception in sloshing scenes with guaranteed thermodynamic consistency](https://mlsanity.com/paper/2106.13301/) (2021)
- [FUSSL: Fuzzy Uncertain Self Supervised Learning](https://mlsanity.com/paper/2210.15818/) (2022)
- [On-device Self-supervised Learning of Visual Perception Tasks aboard Hardware-limited Nano-quadrotors](https://mlsanity.com/paper/2403.04071/) (2024)
- [Training on the Fly: On-device Self-supervised Learning aboard Nano-drones within 20 mW](https://mlsanity.com/paper/2408.03168/) (2024)
- [Confidence-Gated Robot Autonomy: When Does Uncertainty Actually Help?](https://mlsanity.com/paper/2605.18045/) (2026)
- [Imitation Learning-based Direct Visual Servoing using the Large Projection Formulation](https://mlsanity.com/paper/2406.09120/) (2024)
- [GUESS: Generative Uncertainty Ensemble for Self Supervision](https://mlsanity.com/paper/2412.02896/) (2024)
- [Sixth-Sense: Self-Supervised Learning of Spatial Awareness of Humans from a Planar Lidar](https://mlsanity.com/paper/2502.21029/) (2025)

## Abstract (arXiv)

We propose a general self-supervised learning approach for spatial perception tasks, such as estimating the pose of an object relative to the robot, from onboard sensor readings. The model is learned from training episodes, by relying on: a continuous state estimate, possibly inaccurate and affected by odometry drift; and a detector, that sporadically provides supervision about the target pose. We demonstrate the general approach in three different concrete scenarios: a simulated robot arm that visually estimates the pose of an object of interest; a small differential drive robot using 7 infrared sensors to localize a nearby wall; an omnidirectional mobile robot that localizes itself in an environment from camera images. Quantitative results show that the approach works well in all three scenarios, and that explicitly accounting for uncertainty yields statistically significant performance improvements.