# Not All Voxels Are Equal: Semantic Scene Completion from the Point-Voxel Perspective

arXiv: https://arxiv.org/abs/2112.12925 · HTML: https://mlsanity.com/paper/2112.12925/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 55 references that are in the map.
- First posted on arXiv 24 Dec 2021.

## Builds on (most cited first, 30 of 39)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds](https://mlsanity.com/paper/1911.11236/) (2019)
- [DeepGCNs: Can GCNs Go as Deep as CNNs?](https://mlsanity.com/paper/1904.03751/) (2019)
- [Semantic Scene Completion from a Single Depth Image](https://mlsanity.com/paper/1611.08974/) (2016)
- [Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapes](https://mlsanity.com/paper/2101.10994/) (2021)
- [Large-scale Point Cloud Semantic Segmentation with Superpoint Graphs](https://mlsanity.com/paper/1711.09869/) (2017)
- [Point-Voxel CNN for Efficient 3D Deep Learning](https://mlsanity.com/paper/1907.03739/) (2019)
- [SqueezeSeg: Convolutional Neural Nets with Recurrent CRF for Real-Time Road-Object Segmentation from 3D LiDAR Point Cloud](https://mlsanity.com/paper/1710.07368/) (2017)
- [PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds](https://mlsanity.com/paper/2103.14635/) (2021)
- [FrankMocap: A Monocular 3D Whole-Body Pose Estimation System via Regression and Integration](https://mlsanity.com/paper/2108.06428/) (2021)
- [Pointwise Convolutional Neural Networks](https://mlsanity.com/paper/1712.05245/) (2017)
- [Anisotropic Convolutional Networks for 3D Semantic Scene Completion](https://mlsanity.com/paper/2004.02122/) (2020)
- [3D Sketch-aware Semantic Scene Completion via Semi-supervised Structure Prior](https://mlsanity.com/paper/2003.14052/) (2020)
- [Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation](https://mlsanity.com/paper/2007.09183/) (2020)
- [Scene-aware Generative Network for Human Motion Synthesis](https://mlsanity.com/paper/2105.14804/) (2021)
- [A-CNN: Annularly Convolutional Neural Networks on Point Clouds](https://mlsanity.com/paper/1904.08017/) (2019)
- [Efficient Semantic Scene Completion Network with Spatial Group Convolution](https://mlsanity.com/paper/1907.05091/) (2019)
- [RGBD Based Dimensional Decomposition Residual Network for 3D Semantic Scene Completion](https://mlsanity.com/paper/1903.00620/) (2019)
- [FPConv: Learning Local Flattening for Point Convolution](https://mlsanity.com/paper/2002.10701/) (2020)
- [Two Stream 3D Semantic Scene Completion](https://mlsanity.com/paper/1804.03550/) (2018)
- [Fully-Convolutional Point Networks for Large-Scale Point Clouds](https://mlsanity.com/paper/1808.06840/) (2018)

## Built on by (most cited first, 22 of 22)

- [Compressible-composable NeRF via Rank-residual Decomposition](https://mlsanity.com/paper/2205.14870/) (2022)
- [OccDepth: A Depth-Aware Method for 3D Semantic Scene Completion](https://mlsanity.com/paper/2302.13540/) (2023)
- [Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation](https://mlsanity.com/paper/2404.11958/) (2024)
- [SSC-RS: Elevate LiDAR Semantic Scene Completion with Representation Separation and BEV Fusion](https://mlsanity.com/paper/2306.15349/) (2023)
- [Uncovering and Quantifying Social Biases in Code Generation](https://mlsanity.com/paper/2305.15377/) (2023)
- [Interactive Segment Anything NeRF with Feature Imitation](https://mlsanity.com/paper/2305.16233/) (2023)
- [Conditional DETR V2: Efficient Detection Transformer with Box Queries](https://mlsanity.com/paper/2207.08914/) (2022)
- [Point Scene Understanding via Disentangled Instance Mesh Reconstruction](https://mlsanity.com/paper/2203.16832/) (2022)
- [PointSSC: A Cooperative Vehicle-Infrastructure Point Cloud Benchmark for Semantic Scene Completion](https://mlsanity.com/paper/2309.12708/) (2023)
- [Hierarchical Dynamic Image Harmonization](https://mlsanity.com/paper/2211.08639/) (2022)
- [Semantic Scene Completion with Cleaner Self](https://mlsanity.com/paper/2303.09977/) (2023)
- [CasFusionNet: A Cascaded Network for Point Cloud Semantic Scene Completion by Dense Feature Fusion](https://mlsanity.com/paper/2211.13702/) (2022)
- [ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning](https://mlsanity.com/paper/2409.18026/) (2024)
- [Uncovering and Categorizing Social Biases in Text-to-SQL](https://mlsanity.com/paper/2305.16253/) (2023)
- [Multi-View Pedestrian Occupancy Prediction with a Novel Synthetic Dataset](https://mlsanity.com/paper/2412.13569/) (2024)
- [Parallel Sentence-Level Explanation Generation for Real-World Low-Resource Scenarios](https://mlsanity.com/paper/2302.10707/) (2023)
- [Unleashing Network Potentials for Semantic Scene Completion](https://mlsanity.com/paper/2403.07560/) (2024)
- [Real-time 3D Semantic Scene Completion Via Feature Aggregation and Conditioned Prediction](https://mlsanity.com/paper/2303.10967/) (2023)
- [Improving Long Text Understanding with Knowledge Distilled from Summarization Model](https://mlsanity.com/paper/2405.04955/) (2024)
- [Global-Aware Monocular Semantic Scene Completion with State Space Models](https://mlsanity.com/paper/2503.06569/) (2025)
- [Towards Balanced RGB-TSDF Fusion for Consistent Semantic Scene Completion by 3D RGB Feature Completion and a Classwise Entropy Loss Function](https://mlsanity.com/paper/2403.16888/) (2024)
- [Semantic Scene Completion with Multi-Feature Data Balancing Network](https://mlsanity.com/paper/2412.01431/) (2024)

## Abstract (arXiv)

We revisit Semantic Scene Completion (SSC), a useful task to predict the semantic and occupancy representation of 3D scenes, in this paper. A number of methods for this task are always based on voxelized scene representations for keeping local scene structure. However, due to the existence of visible empty voxels, these methods always suffer from heavy computation redundancy when the network goes deeper, and thus limit the completion quality. To address this dilemma, we propose our novel point-voxel aggregation network for this task. Firstly, we transfer the voxelized scenes to point clouds by removing these visible empty voxels and adopt a deep point stream to capture semantic information from the scene efficiently. Meanwhile, a light-weight voxel stream containing only two 3D convolution layers preserves local structures of the voxelized scenes. Furthermore, we design an anisotropic voxel aggregation operator to fuse the structure details from the voxel stream into the point stream, and a semantic-aware propagation module to enhance the up-sampling process in the point stream by semantic labels. We demonstrate that our model surpasses state-of-the-arts on two benchmarks by a large margin, with only depth images as the input.