# Semantic Scene Completion with Multi-Feature Data Balancing Network

arXiv: https://arxiv.org/abs/2412.01431 · HTML: https://mlsanity.com/paper/2412.01431/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 31 references that are in the map.
- First posted on arXiv 2 Dec 2024.

## Builds on (most cited first, 19 of 19)

- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Anisotropic Convolutional Networks for 3D Semantic Scene Completion](https://mlsanity.com/paper/2004.02122/) (2020)
- [3D Semantic Scene Completion: a Survey](https://mlsanity.com/paper/2103.07466/) (2021)
- [RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion](https://mlsanity.com/paper/1903.00620/) (2019)
- [Semantic Scene Completion via Integrating Instances and Scene in-the-Loop](https://mlsanity.com/paper/2104.03640/) (2021)
- [NeuralFusion: Online Depth Fusion in Latent Space](https://mlsanity.com/paper/2011.14791/) (2020)
- [SSCNav: Confidence-Aware Semantic Scene Completion for Visual Semantic Navigation](https://mlsanity.com/paper/2012.04512/) (2020)
- [Cascaded Context Pyramid for Full-Resolution 3D Semantic Scene Completion](https://mlsanity.com/paper/1908.00382/) (2019)
- [Review on Panoramic Imaging and Its Applications in Scene Understanding](https://mlsanity.com/paper/2205.05570/) (2022)
- [Depth Based Semantic Scene Completion with Position Importance Aware Loss](https://mlsanity.com/paper/2001.10709/) (2020)
- [Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective](https://mlsanity.com/paper/2112.12925/) (2021)
- [EdgeNet: Semantic Scene Completion from a Single RGB-D Image](https://mlsanity.com/paper/1908.02893/) (2019)
- [Semantic Scene Completion with Cleaner Self](https://mlsanity.com/paper/2303.09977/) (2023)
- [Data Augmented 3D Semantic Scene Completion with 2D Segmentation Priors](https://mlsanity.com/paper/2111.13309/) (2021)
- [Understanding the Challenges When 3D Semantic Segmentation Faces Class Imbalanced and OOD Data](https://mlsanity.com/paper/2203.00214/) (2022)
- [Unleashing Network Potentials for Semantic Scene Completion](https://mlsanity.com/paper/2403.07560/) (2024)

## Abstract (arXiv)

Semantic Scene Completion (SSC) is a critical task in computer vision, that utilized in applications such as virtual reality (VR). SSC aims to construct detailed 3D models from partial views by transforming a single 2D image into a 3D representation, assigning each voxel a semantic label. The main challenge lies in completing 3D volumes with limited information, compounded by data imbalance, inter-class ambiguity, and intra-class diversity in indoor scenes. To address this, we propose the Multi-Feature Data Balancing Network (MDBNet), a dual-head model for RGB and depth data (F-TSDF) inputs. Our hybrid encoder-decoder architecture with identity transformation in a pre-activation residual module (ITRM) effectively manages diverse signals within F-TSDF. We evaluate RGB feature fusion strategies and use a combined loss function cross entropy for 2D RGB features and weighted cross-entropy for 3D SSC predictions. MDBNet results surpass comparable state-of-the-art (SOTA) methods on NYU datasets, demonstrating the effectiveness of our approach.