# Full or Weak annotations? An adaptive strategy for budget-constrained annotation campaigns

arXiv: https://arxiv.org/abs/2303.11678 · HTML: https://mlsanity.com/paper/2303.11678/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 61 references that are in the map.
- First posted on arXiv 21 Mar 2023.

## Builds on (most cited first, 30 of 35)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Revisiting Unreasonable Effectiveness of Data in Deep Learning Era](https://mlsanity.com/paper/1707.02968/) (2017)
- [Big Transfer (BiT): General Visual Representation Learning](https://mlsanity.com/paper/1912.11370/) (2019)
- [Taskonomy: Disentangling Task Transfer Learning](https://mlsanity.com/paper/1804.08328/) (2018)
- [ScribbleSup: Scribble-Supervised Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1604.05144/) (2016)
- [Learning Pixel-level Semantic Affinity with Image-level Supervision for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1803.10464/) (2018)
- [What's the Point: Semantic Segmentation with Point Supervision](https://mlsanity.com/paper/1506.02106/) (2015)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?](https://mlsanity.com/paper/1706.00712/) (2017)
- [Weakly- and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation](https://mlsanity.com/paper/1502.02734/) (2015)
- [FickleNet: Weakly and Semi-supervised Semantic Image Segmentation using Stochastic Inference](https://mlsanity.com/paper/1902.10421/) (2019)
- [Deep learning for cardiac image segmentation: A review](https://mlsanity.com/paper/1911.03723/) (2019)
- [Normalized Cut Loss for Weakly-supervised CNN Segmentation](https://mlsanity.com/paper/1804.01346/) (2018)
- [Large-scale interactive object segmentation with human annotators](https://mlsanity.com/paper/1903.10830/) (2019)
- [Box-driven Class-wise Region Masking and Filling Rate Guided Loss for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/1904.11693/) (2019)
- [ViewAL: Active Learning with Viewpoint Entropy for Semantic Segmentation](https://mlsanity.com/paper/1911.11789/) (2019)
- [Semantic Segmentation of Underwater Imagery: Dataset and Benchmark](https://mlsanity.com/paper/2004.01241/) (2020)
- [Extreme clicking for efficient object annotation](https://mlsanity.com/paper/1708.02750/) (2017)
- [Reinforced active learning for image segmentation](https://mlsanity.com/paper/2002.06583/) (2020)
- [Factors of Influence for Transfer Learning across Diverse Appearance Domains and Task Types](https://mlsanity.com/paper/2103.13318/) (2021)
- [Constrained Efficient Global Optimization of Expensive Black-box Functions](https://mlsanity.com/paper/2211.00162/) (2022)
- [Affinity Attention Graph Neural Network for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2106.04054/) (2021)
- [Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery](https://mlsanity.com/paper/1805.02475/) (2018)
- [Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challenges](https://mlsanity.com/paper/1707.02432/) (2017)
- [How Much More Data Do I Need? Estimating Requirements for Downstream Tasks](https://mlsanity.com/paper/2207.01725/) (2022)
- [Knowledge Transfer for Melanoma Screening with Deep Learning](https://mlsanity.com/paper/1703.07479/) (2017)
- [Fluid Annotation: A Human-Machine Collaboration Interface for Full Image Annotation](https://mlsanity.com/paper/1806.07527/) (2018)

## Built on by (most cited first, 7 of 7)

- [Instance-wise Supervision-level Optimization in Active Learning](https://mlsanity.com/paper/2503.06517/) (2025)
- [Leveraging Vision-Language Models as Weak Annotators in Active Learning](https://mlsanity.com/paper/2605.00480/) (2026)
- [How Much Data are Enough? Investigating Dataset Requirements for Patch-Based Brain MRI Segmentation Tasks](https://mlsanity.com/paper/2404.03451/) (2024)
- [Automatic Intermodal Loading Unit Identification using Computer Vision: A Scoping Review](https://mlsanity.com/paper/2509.17707/) (2025)
- [GALA: A GlobAl-LocAl Approach for Multi-Source Active Domain Adaptation](https://mlsanity.com/paper/2510.22214/) (2025)
- [From Ambiguity to Action: A POMDP Perspective on Partial Multi-Label Ambiguity and Its Horizon-One Resolution](https://mlsanity.com/paper/2602.04255/) (2026)
- [When Is Coarse Supervision Worth It? Cost-Aware Learning under Unknown Aggregation](https://mlsanity.com/paper/2609.36704/) (2026)

## Abstract (arXiv)

Annotating new datasets for machine learning tasks is tedious, time-consuming, and costly. For segmentation applications, the burden is particularly high as manual delineations of relevant image content are often extremely expensive or can only be done by experts with domain-specific knowledge. Thanks to developments in transfer learning and training with weak supervision, segmentation models can now also greatly benefit from annotations of different kinds. However, for any new domain application looking to use weak supervision, the dataset builder still needs to define a strategy to distribute full segmentation and other weak annotations. Doing so is challenging, however, as it is a priori unknown how to distribute an annotation budget for a given new dataset. To this end, we propose a novel approach to determine annotation strategies for segmentation datasets, whereby estimating what proportion of segmentation and classification annotations should be collected given a fixed budget. To do so, our method sequentially determines proportions of segmentation and classification annotations to collect for budget-fractions by modeling the expected improvement of the final segmentation model. We show in our experiments that our approach yields annotations that perform very close to the optimal for a number of different annotation budgets and datasets.