# Active WeaSuL: Improving Weak Supervision with Active Learning

arXiv: https://arxiv.org/abs/2104.14847 · HTML: https://mlsanity.com/paper/2104.14847/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Visual Relationship Detection with Language Priors](https://mlsanity.com/paper/1608.00187/) (2016)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [Learning from Rules Generalizing Labeled Exemplars](https://mlsanity.com/paper/2004.06025/) (2020)
- [Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale](https://mlsanity.com/paper/1812.00417/) (2018)
- [Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling](https://mlsanity.com/paper/2012.06046/) (2020)
- [Learning Dependency Structures for Weak Supervision Models](https://mlsanity.com/paper/1903.05844/) (2019)
- [Cross-Modal Data Programming Enables Rapid Medical Machine Learning](https://mlsanity.com/paper/1903.11101/) (2019)
- [Scene Graph Prediction with Limited Labels](https://mlsanity.com/paper/1904.11622/) (2019)
- [Active and Incremental Learning with Weak Supervision](https://mlsanity.com/paper/2001.07100/) (2020)

## Built on by (most cited first, 12 of 12)

- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Language Models in the Loop: Incorporating Prompting into Weak Supervision](https://mlsanity.com/paper/2205.02318/) (2022)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis](https://mlsanity.com/paper/2111.15186/) (2021)
- [Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware](https://mlsanity.com/paper/2402.15943/) (2024)
- [Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents](https://mlsanity.com/paper/2302.09324/) (2023)
- [ActiveDP: Bridging Active Learning and Data Programming](https://mlsanity.com/paper/2402.06056/) (2024)
- [Knowledge Modelling and Active Learning in Manufacturing](https://mlsanity.com/paper/2107.02298/) (2021)
- [Label-Efficient Interactive Time-Series Anomaly Detection](https://mlsanity.com/paper/2212.14621/) (2022)
- [Weakly Supervised Label Learning Flows](https://mlsanity.com/paper/2302.09649/) (2023)
- [Local Boosting for Weakly-Supervised Learning](https://mlsanity.com/paper/2306.02859/) (2023)

## Abstract (arXiv)

The availability of labelled data is one of the main limitations in machine learning. We can alleviate this using weak supervision: a framework that uses expert-defined rules boldsymbolλ to estimate probabilistic labels p(y|boldsymbolλ) for the entire data set. These rules, however, are dependent on what experts know about the problem, and hence may be inaccurate or may fail to capture important parts of the problem-space. To mitigate this, we propose Active WeaSuL: an approach that incorporates active learning into weak supervision. In Active WeaSuL, experts do not only define rules, but they also iteratively provide the true label for a small set of points where the weak supervision model is most likely to be mistaken, which are then used to better estimate the probabilistic labels. In this way, the weak labels provide a warm start, which active learning then improves upon. We make two contributions: 1) a modification of the weak supervision loss function, such that the expert-labelled data inform and improve the combination of weak labels; and 2) the maxKL divergence sampling strategy, which determines for which data points expert labelling is most beneficial. Our experiments show that when the budget for labelling data is limited (e.g. ≤ 60 data points), Active WeaSuL outperforms weak supervision, active learning, and competing strategies, with only a handful of labelled data points. This makes Active WeaSuL ideal for situations where obtaining labelled data is difficult.