# View-volume Network for Semantic Scene Completion from a Single Depth Image

arXiv: https://arxiv.org/abs/1806.05361 · HTML: https://mlsanity.com/paper/1806.05361/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2018.

## Built on by (most cited first, 26 of 26)

- [Symphonize 3D Semantic Scene Completion with Contextual Instance Queries](https://mlsanity.com/paper/2306.15670/) (2023)
- [3D Semantic Scene Completion: a Survey](https://mlsanity.com/paper/2103.07466/) (2021)
- [AdaPoinTr: Diverse Point Cloud Completion with Adaptive Geometry-Aware Transformers](https://mlsanity.com/paper/2301.04545/) (2023)
- [Semantic Scene Completion via Integrating Instances and Scene in-the-Loop](https://mlsanity.com/paper/2104.03640/) (2021)
- [MotionSC: Data Set and Network for Real-Time Semantic Mapping in Dynamic Environments](https://mlsanity.com/paper/2203.07060/) (2022)
- [Indoor Scene Generation from a Collection of Semantic-Segmented Depth Images](https://mlsanity.com/paper/2108.09022/) (2021)
- [Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective](https://mlsanity.com/paper/2112.12925/) (2021)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [Learning Local Displacements for Point Cloud Completion](https://mlsanity.com/paper/2203.16600/) (2022)
- [Semantic Scene Completion with Cleaner Self](https://mlsanity.com/paper/2303.09977/) (2023)
- [CasFusionNet: A Cascaded Network for Point Cloud Semantic Scene Completion by Dense Feature Fusion](https://mlsanity.com/paper/2211.13702/) (2022)
- [IMENet: Joint 3D Semantic Scene Completion and 2D Semantic Segmentation through Iterative Mutual Enhancement](https://mlsanity.com/paper/2106.15413/) (2021)
- [Towards Efficient Point Cloud Graph Neural Networks Through Architectural Simplification](https://mlsanity.com/paper/2108.06317/) (2021)
- [Data Augmented 3D Semantic Scene Completion with 2D Segmentation Priors](https://mlsanity.com/paper/2111.13309/) (2021)
- [PointSea: Point Cloud Completion via Self-structure Augmentation](https://mlsanity.com/paper/2502.17053/) (2025)
- [SSR-2D: Semantic 3D Scene Reconstruction from 2D Images](https://mlsanity.com/paper/2302.03640/) (2023)
- [Point Cloud Scene Completion with Joint Color and Semantic Estimation from Single RGB-D Image](https://mlsanity.com/paper/2210.05891/) (2022)
- [VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion](https://mlsanity.com/paper/2503.06219/) (2025)
- [CVSformer: Cross-View Synthesis Transformer for Semantic Scene Completion](https://mlsanity.com/paper/2307.07938/) (2023)
- [Unleashing Network Potentials for Semantic Scene Completion](https://mlsanity.com/paper/2403.07560/) (2024)
- [Real-time 3D Semantic Scene Completion Via Feature Aggregation and Conditioned Prediction](https://mlsanity.com/paper/2303.10967/) (2023)
- [Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance](https://mlsanity.com/paper/2502.14520/) (2025)
- [Vision-based 3D Semantic Scene Completion via Capture Dynamic Representations](https://mlsanity.com/paper/2503.06222/) (2025)
- [MGPC: Multimodal Network for Generalizable Point Cloud Completion With Modality Dropout and Progressive Decoding](https://mlsanity.com/paper/2601.03660/) (2026)
- [Unsigned Distance Field as an Accurate 3D Scene Representation for Neural Scene Completion](https://mlsanity.com/paper/2203.09167/) (2022)
- [A Federated Parameter Aggregation Method for Node Classification Tasks with Different Graph Network Structures](https://mlsanity.com/paper/2403.16004/) (2024)

## Abstract (arXiv)

We introduce a View-Volume convolutional neural network (VVNet) for inferring the occupancy and semantic labels of a volumetric 3D scene from a single depth image. The VVNet concatenates a 2D view CNN and a 3D volume CNN with a differentiable projection layer. Given a single RGBD image, our method extracts the detailed geometric features from the input depth image with a 2D view CNN and then projects the features into a 3D volume according to the input depth map via a projection layer. After that, we learn the 3D context information of the scene with a 3D volume CNN for computing the result volumetric occupancy and semantic labels. With combined 2D and 3D representations, the VVNet efficiently reduces the computational cost, enables feature extraction from multi-channel high resolution inputs, and thus significantly improves the result accuracy. We validate our method and demonstrate its efficiency and effectiveness on both synthetic SUNCG and real NYU dataset.