# SceneNet: Understanding Real World Indoor Scenes With Synthetic Data

arXiv: https://arxiv.org/abs/1511.07041 · HTML: https://mlsanity.com/paper/1511.07041/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2015.

## Built on by (most cited first, 29 of 29)

- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation](https://mlsanity.com/paper/2306.01923/) (2023)
- [Bidirectional Projection Network for Cross Dimension Scene Understanding](https://mlsanity.com/paper/2103.14326/) (2021)
- [POCO: Point Convolution for Surface Reconstruction](https://mlsanity.com/paper/2201.01831/) (2022)
- [MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?](https://mlsanity.com/paper/2108.09518/) (2021)
- [Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priors](https://mlsanity.com/paper/2204.10603/) (2022)
- [Surface Reconstruction from Point Clouds by Learning Predictive Context Priors](https://mlsanity.com/paper/2204.11015/) (2022)
- [LRM-Zero: Training Large Reconstruction Models with Synthesized Data](https://mlsanity.com/paper/2406.09371/) (2024)
- [Neural-Sim: Learning to Generate Training Data with NeRF](https://mlsanity.com/paper/2207.11368/) (2022)
- [A Survey on Human-aware Robot Navigation](https://mlsanity.com/paper/2106.11650/) (2021)
- [Learning Part Segmentation through Unsupervised Domain Adaptation from Synthetic Vehicles](https://mlsanity.com/paper/2103.14098/) (2021)
- [DODA: Data-oriented Sim-to-Real Domain Adaptation for 3D Semantic Segmentation](https://mlsanity.com/paper/2204.01599/) (2022)
- [MaAST: Map Attention with Semantic Transformersfor Efficient Visual Navigation](https://mlsanity.com/paper/2103.11374/) (2021)
- [SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation](https://mlsanity.com/paper/2205.13490/) (2022)
- [CasFusionNet: A Cascaded Network for Point Cloud Semantic Scene Completion by Dense Feature Fusion](https://mlsanity.com/paper/2211.13702/) (2022)
- [GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds](https://mlsanity.com/paper/2308.13175/) (2023)
- [FurniScene: A Large-scale 3D Room Dataset with Intricate Furnishing Scenes](https://mlsanity.com/paper/2401.03470/) (2024)
- [Synthetic Data for Multi-Parameter Camera-Based Physiological Sensing](https://mlsanity.com/paper/2110.04902/) (2021)
- [Synthetic 3D Data Generation Pipeline for Geometric Deep Learning in Architecture](https://mlsanity.com/paper/2104.12564/) (2021)
- [M3DLayout: A Multi-Source Dataset of 3D Indoor Layouts and Structured Descriptions for 3D Generation](https://mlsanity.com/paper/2509.23728/) (2025)
- [Quasi-Balanced Self-Training on Noise-Aware Synthesis of Object Point Clouds for Closing Domain Gap](https://mlsanity.com/paper/2203.03833/) (2022)
- [A Procedural World Generation Framework for Systematic Evaluation of Continual Learning](https://mlsanity.com/paper/2106.02585/) (2021)
- [Robust-DefReg: A Robust Deformable Point Cloud Registration Method based on Graph Convolutional Neural Networks](https://mlsanity.com/paper/2306.04701/) (2023)
- [DCL: Differential Contrastive Learning for Geometry-Aware Depth Synthesis](https://mlsanity.com/paper/2107.13087/) (2021)
- [Scan2Part: Fine-grained and Hierarchical Part-level Understanding of Real-World 3D Scans](https://mlsanity.com/paper/2206.02366/) (2022)
- [KLIEP-based Density Ratio Estimation for Semantically Consistent Synthetic to Real Images Adaptation in Urban Traffic Scenes](https://mlsanity.com/paper/2105.12549/) (2021)
- [Meta-simulation for the Automated Design of Synthetic Overhead Imagery](https://mlsanity.com/paper/2209.08685/) (2022)
- [Towards Semi-supervised Dual-modal Semantic Segmentation](https://mlsanity.com/paper/2409.13325/) (2024)
- [SynBench: A Synthetic Benchmark for Non-rigid 3D Point Cloud Registration](https://mlsanity.com/paper/2409.14474/) (2024)

## Abstract (arXiv)

Scene understanding is a prerequisite to many high level tasks for any automated intelligent machine operating in real world environments. Recent attempts with supervised learning have shown promise in this direction but also highlighted the need for enormous quantity of supervised data --- performance increases in proportion to the amount of data used. However, this quickly becomes prohibitive when considering the manual labour needed to collect such data. In this work, we focus our attention on depth based semantic per-pixel labelling as a scene understanding problem and show the potential of computer graphics to generate virtually unlimited labelled data from synthetic 3D scenes. By carefully synthesizing training data with appropriate noise models we show comparable performance to state-of-the-art RGBD systems on NYUv2 dataset despite using only depth data as input and set a benchmark on depth-based segmentation on SUN RGB-D dataset. Additionally, we offer a route to generating synthesized frame or video data, and understanding of different factors influencing performance gains.