# Points2Polygons: Context-Based Segmentation from Weak Labels Using Adversarial Networks

arXiv: https://arxiv.org/abs/2106.02804 · HTML: https://mlsanity.com/paper/2106.02804/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 5 Jun 2021.

## Builds on (most cited first, 17 of 17)

- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1503.01640/) (2015)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [Unsupervised Object Segmentation by Redrawing](https://mlsanity.com/paper/1905.13539/) (2019)
- [Semantic Segmentation using Adversarial Networks](https://mlsanity.com/paper/1611.08408/) (2016)
- [Domain Adaptive Semantic Segmentation Using Weak Labels](https://mlsanity.com/paper/2007.15176/) (2020)
- [MAP-Net: Multi Attending Path Neural Network for Building Footprint Extraction from Remote Sensed Imagery](https://mlsanity.com/paper/1910.12060/) (2019)
- [Generalized Contrastive Optimization of Siamese Networks for Place Recognition](https://mlsanity.com/paper/2103.06638/) (2021)
- [Pick-and-Learn: Automatic Quality Evaluation for Noisy-Labeled Image Segmentation](https://mlsanity.com/paper/1907.11835/) (2019)
- [Instance Segmentation with Point Supervision](https://mlsanity.com/paper/1906.06392/) (2019)
- [Self-Supervised Learning for Segmentation](https://mlsanity.com/paper/2101.05456/) (2021)
- [gSLICr: SLIC superpixels at over 250Hz](https://mlsanity.com/paper/1509.04232/) (2015)
- [Gradient Weighted Superpixels for Interpretability in CNNs](https://mlsanity.com/paper/1908.08997/) (2019)

## Abstract (arXiv)

In applied image segmentation tasks, the ability to provide numerous and precise labels for training is paramount to the accuracy of the model at inference time. However, this overhead is often neglected, and recently proposed segmentation architectures rely heavily on the availability and fidelity of ground truth labels to achieve state-of-the-art accuracies. Failure to acknowledge the difficulty in creating adequate ground truths can lead to an over-reliance on pre-trained models or a lack of adoption in real-world applications. We introduce Points2Polygons (P2P), a model which makes use of contextual metric learning techniques that directly addresses this problem. Points2Polygons performs well against existing fully-supervised segmentation baselines with limited training data, despite using lightweight segmentation models (U-Net with a ResNet18 backbone) and having access to only weak labels in the form of object centroids and no pre-training. We demonstrate this on several different small but non-trivial datasets. We show that metric learning using contextual data provides key insights for self-supervised tasks in general, and allow segmentation models to easily generalize across traditionally label-intensive domains in computer vision.