# 3D Guided Weakly Supervised Semantic Segmentation

arXiv: https://arxiv.org/abs/2012.00242 · HTML: https://mlsanity.com/paper/2012.00242/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [Segment Anything Model (SAM) Enhanced Pseudo Labels for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.05803/) (2023)
- [An Alternative to WSSS? An Empirical Study of the Segment Anything Model (SAM) on Weakly-Supervised Semantic Segmentation Problems](https://mlsanity.com/paper/2305.01586/) (2023)
- [2D-3D Interlaced Transformer for Point Cloud Segmentation with Scene-Level Supervision](https://mlsanity.com/paper/2310.12817/) (2023)
- [Inferring the Class Conditional Response Map for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2110.14309/) (2021)
- [GETAM: Gradient-weighted Element-wise Transformer Attention Map for Weakly-supervised Semantic segmentation](https://mlsanity.com/paper/2112.02841/) (2021)
- [Self-Improving Semantic Perception for Indoor Localisation](https://mlsanity.com/paper/2105.01595/) (2021)
- [All-pairs Consistency Learning for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2308.04321/) (2023)
- [ToNNO: Tomographic Reconstruction of a Neural Network's Output for Weakly Supervised Segmentation of 3D Medical Images](https://mlsanity.com/paper/2404.13103/) (2024)
- [A Data-efficient Framework for Robotics Large-scale LiDAR Scene Parsing](https://mlsanity.com/paper/2312.02208/) (2023)
- [Rewis3d: Reconstruction Improves Weakly-Supervised Semantic Segmentation](https://mlsanity.com/paper/2603.06374/) (2026)

## Abstract (arXiv)

Pixel-wise clean annotation is necessary for fully-supervised semantic segmentation, which is laborious and expensive to obtain. In this paper, we propose a weakly supervised 2D semantic segmentation model by incorporating sparse bounding box labels with available 3D information, which is much easier to obtain with advanced sensors. We manually labeled a subset of the 2D-3D Semantics(2D-3D-S) dataset with bounding boxes, and introduce our 2D-3D inference module to generate accurate pixel-wise segment proposal masks. Guided by 3D information, we first generate a point cloud of objects and calculate objectness probability score for each point. Then we project the point cloud with objectness probabilities back to 2D images followed by a refinement step to obtain segment proposals, which are treated as pseudo labels to train a semantic segmentation network. Our method works in a recursive manner to gradually refine the above-mentioned segment proposals. Extensive experimental results on the 2D-3D-S dataset show that the proposed method can generate accurate segment proposals when bounding box labels are available on only a small subset of training images. Performance comparison with recent state-of-the-art methods further illustrates the effectiveness of our method.