# Pointly-supervised 3D Scene Parsing with Viewpoint Bottleneck

arXiv: https://arxiv.org/abs/2109.08553 · HTML: https://mlsanity.com/paper/2109.08553/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 17 Sep 2021.

## Builds on (most cited first, 27 of 27)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [Unsupervised Visual Representation Learning by Context Prediction](https://mlsanity.com/paper/1505.05192/) (2015)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel Prediction](https://mlsanity.com/paper/1611.09842/) (2016)
- [Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts](https://mlsanity.com/paper/2012.09165/) (2020)
- [Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds](https://mlsanity.com/paper/2109.00179/) (2021)
- [One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation](https://mlsanity.com/paper/2104.02246/) (2021)
- [STC: A Simple to Complex Framework for Weakly-supervised Semantic Segmentation](https://mlsanity.com/paper/1509.03150/) (2015)
- [Self-Supervised Learning with Kernel Dependence Maximization](https://mlsanity.com/paper/2106.08320/) (2021)
- [Multi-Path Region Mining For Weakly Supervised 3D Semantic Segmentation on Point Clouds](https://mlsanity.com/paper/2003.13035/) (2020)
- [Contrastive Multimodal Fusion with TupleInfoNCE](https://mlsanity.com/paper/2107.02575/) (2021)
- [P4Contrast: Contrastive Learning with Pairs of Point-Pixel Pairs for RGB-D Scene Understanding](https://mlsanity.com/paper/2012.13089/) (2020)
- [Weakly Supervised Scene Parsing with Point-based Distance Metric Learning](https://mlsanity.com/paper/1811.02233/) (2018)
- [Weakly Supervised Semantic Segmentation in 3D Graph-Structured Point Clouds of Wild Scenes](https://mlsanity.com/paper/2004.12498/) (2020)

## Built on by (most cited first, 7 of 7)

- [Dual Adaptive Transformations for Weakly Supervised Point Cloud Segmentation](https://mlsanity.com/paper/2207.09084/) (2022)
- [Data Efficient 3D Learner via Knowledge Transferred from 2D Model](https://mlsanity.com/paper/2203.08479/) (2022)
- [Language-guided Semantic Style Transfer of 3D Indoor Scenes](https://mlsanity.com/paper/2208.07870/) (2022)
- [Reliability-Adaptive Consistency Regularization for Weakly-Supervised Point Cloud Segmentation](https://mlsanity.com/paper/2303.05164/) (2023)
- [Eff-3DPSeg: 3D organ-level plant shoot segmentation using annotation-efficient point clouds](https://mlsanity.com/paper/2212.10263/) (2022)
- [A Review and A Robust Framework of Data-Efficient 3D Scene Parsing with Traditional/Learned 3D Descriptors](https://mlsanity.com/paper/2312.01262/) (2023)
- [Integrating SAM Supervision for 3D Weakly Supervised Point Cloud Segmentation](https://mlsanity.com/paper/2508.19909/) (2025)

## Abstract (arXiv)

Semantic understanding of 3D point clouds is important for various robotics applications. Given that point-wise semantic annotation is expensive, in this paper, we address the challenge of learning models with extremely sparse labels. The core problem is how to leverage numerous unlabeled points. To this end, we propose a self-supervised 3D representation learning framework named viewpoint bottleneck. It optimizes a mutual-information based objective, which is applied on point clouds under different viewpoints. A principled analysis shows that viewpoint bottleneck leads to an elegant surrogate loss function that is suitable for large-scale point cloud data. Compared with former arts based upon contrastive learning, viewpoint bottleneck operates on the feature dimension instead of the sample dimension. This paradigm shift has several advantages: It is easy to implement and tune, does not need negative samples and performs better on our goal down-streaming task. We evaluate our method on the public benchmark ScanNet, under the pointly-supervised setting. We achieve the best quantitative results among comparable solutions. Meanwhile we provide an extensive qualitative inspection on various challenging scenes. They demonstrate that our models can produce fairly good scene parsing results for robotics applications. Our code, data and models will be made public.