# Applying Deep Learning for cockpit segmentation in the context of mixed reality

arXiv: https://arxiv.org/abs/2606.06520 · HTML: https://mlsanity.com/paper/2606.06520/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 12 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 6 of 6)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection](https://mlsanity.com/paper/2101.08418/) (2021)
- [Enhanced Self-Perception in Mixed Reality: Egocentric Arm Segmentation and Database with Automatic Labelling](https://mlsanity.com/paper/2003.12352/) (2020)
- [Using CNNs For Users Segmentation In Video See-Through Augmented Virtuality](https://mlsanity.com/paper/2001.00487/) (2020)

## Abstract (arXiv)

Computer vision is an area that has been growing continuously. With the advance of technologies with a first-person view, new development opportunities have emerged inside the area. Mixed reality promotes virtual environments with objects from the physical world shown in real time. For that, it's necessary to be concerned with the immersion of the user in this simulated environment, increasingly seeking to bring it closer to a possible desired reality. This paper proposes the development of image processing in order to perform the segmentation of images to identify what is foreground and background in order to facilitate the union of virtual and real images. Thus, the present work obtain real images of the user using the off-highway truck simulator CAT793F, through a camera, to be able to perform the segmentation of such images with artificial intelligence techniques.The convolutional neural network architectures "U-net" and "DeepLabV3+" are applied to perform image segmentation. As a result, metrics with around 90% accuracy were presented and and the best model was determined.