# 3D Spatial Recognition without Spatially Labeled 3D

arXiv: https://arxiv.org/abs/2105.06461 · HTML: https://mlsanity.com/paper/2105.06461/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 65 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 30 of 49)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [LVIS: A Dataset for Large Vocabulary Instance Segmentation](https://mlsanity.com/paper/1908.03195/) (2019)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [3D Semantic Segmentation with Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1711.10275/) (2017)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Deep Hough Voting for 3D Object Detection in Point Clouds](https://mlsanity.com/paper/1904.09664/) (2019)
- [Frustum PointNets for 3D Object Detection from RGB-D Data](https://mlsanity.com/paper/1711.08488/) (2017)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [Cross-stitch Networks for Multi-task Learning](https://mlsanity.com/paper/1604.03539/) (2016)
- [Joint 2D-3D-Semantic Data for Indoor Scene Understanding](https://mlsanity.com/paper/1702.01105/) (2017)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [3D-SIS: 3D Semantic Instance Segmentation of RGB-D Scans](https://mlsanity.com/paper/1812.07003/) (2018)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [UberNet: Training a `Universal' Convolutional Neural Network for Low-, Mid-, and High-Level Vision using Diverse Datasets and Limited Memory](https://mlsanity.com/paper/1609.02132/) (2016)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [Weakly Supervised Deep Detection Networks](https://mlsanity.com/paper/1511.02853/) (2015)
- [Learning a Predictable and Generative Vector Representation for Objects](https://mlsanity.com/paper/1603.08637/) (2016)
- [GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloud](https://mlsanity.com/paper/1812.03320/) (2018)
- [Revisiting Dilated Convolution: A Simple Approach for Weakly- and Semi- Supervised Semantic Segmentation](https://mlsanity.com/paper/1805.04574/) (2018)
- [H3DNet: 3D Object Detection Using Hybrid Geometric Primitives](https://mlsanity.com/paper/2006.05682/) (2020)

## Built on by (most cited first, 30 of 31)

- [3D Object Detection from Images for Autonomous Driving: A Survey](https://mlsanity.com/paper/2202.02980/) (2022)
- [LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds](https://mlsanity.com/paper/2210.08064/) (2022)
- [GrowSP: Unsupervised Semantic Segmentation of 3D Point Clouds](https://mlsanity.com/paper/2305.16404/) (2023)
- [VIBUS: Data-efficient 3D Scene Parsing with VIewpoint Bottleneck and Uncertainty-Spectrum Modeling](https://mlsanity.com/paper/2210.11472/) (2022)
- [A Survey of Label-Efficient Deep Learning for 3D Point Clouds](https://mlsanity.com/paper/2305.19812/) (2023)
- [Back to Reality: Weakly-supervised 3D Object Detection with Shape-guided Label Enhancement](https://mlsanity.com/paper/2203.05238/) (2022)
- [LiDAL: Inter-frame Uncertainty Based Active Learning for 3D LiDAR Semantic Segmentation](https://mlsanity.com/paper/2211.05997/) (2022)
- [2D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervision](https://mlsanity.com/paper/2310.12817/) (2023)
- [Point-GCC: Universal Self-supervised 3D Scene Pre-training via Geometry-Color Contrast](https://mlsanity.com/paper/2305.19623/) (2023)
- [CPCM: Contextual Point Cloud Modeling for Weakly-supervised Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2307.10316/) (2023)
- [3D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidance](https://mlsanity.com/paper/2407.09826/) (2024)
- [Box2Seg: Learning Semantics of 3D Point Clouds with Box-Level Supervision](https://mlsanity.com/paper/2201.02963/) (2022)
- [Class-agnostic Reconstruction of Dynamic Objects from Videos](https://mlsanity.com/paper/2112.02091/) (2021)
- [Total Variation Optimization Layers for Computer Vision](https://mlsanity.com/paper/2204.03643/) (2022)
- [You Never Get a Second Chance To Make a Good First Impression: Seeding Active Learning for 3D Semantic Segmentation](https://mlsanity.com/paper/2304.11762/) (2023)
- [PlanarNeRF: Online Learning of Planar Primitives with Neural Radiance Fields](https://mlsanity.com/paper/2401.00871/) (2023)
- [LiDAR Remote Sensing Meets Weak Supervision: Concepts, Methods, and Perspectives](https://mlsanity.com/paper/2503.18384/) (2025)
- [SL3D: Self-supervised-Self-labeled 3D Recognition](https://mlsanity.com/paper/2210.16810/) (2022)
- [P-SLCR: Unsupervised Point Cloud Semantic Segmentation via Prototypes Structure Learning and Consistent Reasoning](https://mlsanity.com/paper/2603.06321/) (2026)
- [PointGS: Semantic-Consistent Unsupervised 3D Point Cloud Segmentation with 3D Gaussian Splatting](https://mlsanity.com/paper/2605.11520/) (2026)
- [Hypergraph Convolutional Network based Weakly Supervised Point Cloud Semantic Segmentation with Scene-Level Annotations](https://mlsanity.com/paper/2211.01174/) (2022)
- [Cross-modal and Cross-domain Knowledge Transfer for Label-free 3D Segmentation](https://mlsanity.com/paper/2309.10649/) (2023)
- [Multi-modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation](https://mlsanity.com/paper/2312.16578/) (2023)
- [Bayesian Self-Training for Semi-Supervised 3D Segmentation](https://mlsanity.com/paper/2409.08102/) (2024)
- [High-quality Pseudo-labeling for Point Cloud Segmentation with Scene-level Annotation](https://mlsanity.com/paper/2506.23227/) (2025)
- [3D Weakly Supervised Semantic Segmentation via Class-Aware and Geometry-Guided Pseudo-Label Refinement](https://mlsanity.com/paper/2510.17875/) (2025)
- [Box2Mask: Weakly Supervised 3D Semantic Instance Segmentation Using Bounding Boxes](https://mlsanity.com/paper/2206.01203/) (2022)
- [You Only Need One Thing One Click: Self-Training for Weakly Supervised 3D Scene Understanding](https://mlsanity.com/paper/2303.14727/) (2023)
- [Weakly Supervised Point Cloud Segmentation via Conservative Propagation of Scene-level Labels](https://mlsanity.com/paper/2312.06799/) (2023)
- [Smart Feature is What You Need](https://mlsanity.com/paper/2406.15805/) (2024)

## Abstract (arXiv)

We introduce WyPR, a Weakly-supervised framework for Point cloud Recognition, requiring only scene-level class tags as supervision. WyPR jointly addresses three core 3D recognition tasks: point-level semantic segmentation, 3D proposal generation, and 3D object detection, coupling their predictions through self and cross-task consistency losses. We show that in conjunction with standard multiple-instance learning objectives, WyPR can detect and segment objects in point cloud data without access to any spatial labels at training time. We demonstrate its efficacy using the ScanNet and S3DIS datasets, outperforming prior state of the art on weakly-supervised segmentation by more than 6% mIoU. In addition, we set up the first benchmark for weakly-supervised 3D object detection on both datasets, where WyPR outperforms standard approaches and establishes strong baselines for future work.