# Data Augmented 3D Semantic Scene Completion with 2D Segmentation Priors

arXiv: https://arxiv.org/abs/2111.13309 · HTML: https://mlsanity.com/paper/2111.13309/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 39 references that are in the map.
- First posted on arXiv 26 Nov 2021.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decay](https://mlsanity.com/paper/1803.09820/) (2018)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [Anisotropic Convolutional Networks for 3D Semantic Scene Completion](https://mlsanity.com/paper/2004.02122/) (2020)
- [3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior](https://mlsanity.com/paper/2003.14052/) (2020)
- [Depth-aware CNN for RGB-D Segmentation](https://mlsanity.com/paper/1803.06791/) (2018)
- [Efficient Semantic Scene Completion Network with Spatial Group Convolution](https://mlsanity.com/paper/1907.05091/) (2019)
- [RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion](https://mlsanity.com/paper/1903.00620/) (2019)
- [Semantic Scene Completion via Integrating Instances and Scene in-the-Loop](https://mlsanity.com/paper/2104.03640/) (2021)
- [Two Stream 3D Semantic Scene Completion](https://mlsanity.com/paper/1804.03550/) (2018)
- [Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completion](https://mlsanity.com/paper/1908.00382/) (2019)
- [ForkNet: Multi-branch Volumetric Semantic Completion from a Single Depth Image](https://mlsanity.com/paper/1909.01106/) (2019)
- [Attention-based Multi-modal Fusion Network for Semantic Scene Completion](https://mlsanity.com/paper/2003.13910/) (2020)
- [View-volume Network for Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1806.05361/) (2018)
- [EdgeNet: Semantic Scene Completion from a Single RGB-D Image](https://mlsanity.com/paper/1908.02893/) (2019)
- [Blockout: Dynamic Model Selection for Hierarchical Deep Networks](https://mlsanity.com/paper/1512.05246/) (2015)
- [Predicting Complete 3D Models of Indoor Scenes](https://mlsanity.com/paper/1504.02437/) (2015)
- [Semantic Scene Completion Combining Colour and Depth: preliminary experiments](https://mlsanity.com/paper/1802.04735/) (2018)

## Built on by (most cited first, 9 of 9)

- [Semantic Scene Completion with Cleaner Self](https://mlsanity.com/paper/2303.09977/) (2023)
- [CasFusionNet: A Cascaded Network for Point Cloud Semantic Scene Completion by Dense Feature Fusion](https://mlsanity.com/paper/2211.13702/) (2022)
- [Skip Mamba Diffusion for Monocular 3D Semantic Scene Completion](https://mlsanity.com/paper/2501.07260/) (2025)
- [SSR-2D: Semantic 3D Scene Reconstruction from 2D Images](https://mlsanity.com/paper/2302.03640/) (2023)
- [Semantic Complete Scene Forecasting from a 4D Dynamic Point Cloud Sequence](https://mlsanity.com/paper/2312.08054/) (2023)
- [CVSformer: Cross-View Synthesis Transformer for Semantic Scene Completion](https://mlsanity.com/paper/2307.07938/) (2023)
- [Unleashing Network Potentials for Semantic Scene Completion](https://mlsanity.com/paper/2403.07560/) (2024)
- [RWKV-PCSSC: Exploring RWKV Model for Point Cloud Semantic Scene Completion](https://mlsanity.com/paper/2511.09878/) (2025)
- [Semantic Scene Completion with Multi-Feature Data Balancing Network](https://mlsanity.com/paper/2412.01431/) (2024)

## Abstract (arXiv)

Semantic scene completion (SSC) is a challenging Computer Vision task with many practical applications, from robotics to assistive computing. Its goal is to infer the 3D geometry in a field of view of a scene and the semantic labels of voxels, including occluded regions. In this work, we present SPAwN, a novel lightweight multimodal 3D deep CNN that seamlessly fuses structural data from the depth component of RGB-D images with semantic priors from a bimodal 2D segmentation network. A crucial difficulty in this field is the lack of fully labeled real-world 3D datasets which are large enough to train the current data-hungry deep 3D CNNs. In 2D computer vision tasks, many data augmentation strategies have been proposed to improve the generalization ability of CNNs. However those approaches cannot be directly applied to the RGB-D input and output volume of SSC solutions. In this paper, we introduce the use of a 3D data augmentation strategy that can be applied to multimodal SSC networks. We validate our contributions with a comprehensive and reproducible ablation study. Our solution consistently surpasses previous works with a similar level of complexity.