# Floors are Flat: Leveraging Semantics for Real-Time Surface Normal Prediction

arXiv: https://arxiv.org/abs/1906.06792 · HTML: https://mlsanity.com/paper/1906.06792/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2019.

## Built on by (most cited first, 6 of 6)

- [Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation](https://mlsanity.com/paper/2109.09881/) (2021)
- [Learning Indoor Inverse Rendering with 3D Spatially-Varying Lighting](https://mlsanity.com/paper/2109.06061/) (2021)
- [Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging](https://mlsanity.com/paper/2303.07126/) (2023)
- [PolyMaX: General Dense Prediction with Mask Transformer](https://mlsanity.com/paper/2311.05770/) (2023)
- [When Multi-Task Learning Meets Partial Supervision: A Computer Vision Review](https://mlsanity.com/paper/2307.14382/) (2023)
- [DeepJoin: Learning a Joint Occupancy, Signed Distance, and Normal Field Function for Shape Repair](https://mlsanity.com/paper/2211.12400/) (2022)

## Abstract (arXiv)

We propose 4 insights that help to significantly improve the performance of deep learning models that predict surface normals and semantic labels from a single RGB image. These insights are: (1) denoise the "ground truth" surface normals in the training set to ensure consistency with the semantic labels; (2) concurrently train on a mix of real and synthetic data, instead of pretraining on synthetic and finetuning on real; (3) jointly predict normals and semantics using a shared model, but only backpropagate errors on pixels that have valid training labels; (4) slim down the model and use grayscale instead of color inputs. Despite the simplicity of these steps, we demonstrate consistently improved results on several datasets, using a model that runs at 12 fps on a standard mobile phone.