# Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming

arXiv: https://arxiv.org/abs/2203.01382 · HTML: https://mlsanity.com/paper/2203.01382/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 46 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 28 of 28)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods](https://mlsanity.com/paper/2002.11955/) (2020)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Learning from Rules Generalizing Labeled Exemplars](https://mlsanity.com/paper/2004.06025/) (2020)
- [Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale](https://mlsanity.com/paper/1812.00417/) (2018)
- [Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling](https://mlsanity.com/paper/2012.06046/) (2020)
- [Denoising Multi-Source Weak Supervision for Neural Text Classification](https://mlsanity.com/paper/2010.04582/) (2020)
- [End-to-End Weak Supervision](https://mlsanity.com/paper/2107.02233/) (2021)
- [Universalizing Weak Supervision](https://mlsanity.com/paper/2112.03865/) (2021)
- [Overton: A Data System for Monitoring and Improving Machine-Learned Products](https://mlsanity.com/paper/1909.05372/) (2019)
- [Weakly Supervised Named Entity Tagging with Learnable Logical Rules](https://mlsanity.com/paper/2107.02282/) (2021)
- [GOGGLES: Automatic Image Labeling with Affinity Coding](https://mlsanity.com/paper/1903.04552/) (2019)
- [Inferring Generative Model Structure with Static Analysis](https://mlsanity.com/paper/1709.02477/) (2017)
- [Multi-Resolution Weak Supervision for Sequential Data](https://mlsanity.com/paper/1910.09505/) (2019)
- [Semi-Supervised Data Programming with Subset Selection](https://mlsanity.com/paper/2008.09887/) (2020)
- [Adaptive Rule Discovery for Labeling Text Data](https://mlsanity.com/paper/2005.06133/) (2020)
- [GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition](https://mlsanity.com/paper/2104.06230/) (2021)
- [Active WeaSuL: Improving Weak Supervision with Active Learning](https://mlsanity.com/paper/2104.14847/) (2021)
- [Automatic Synthesis of Diverse Weak Supervision Sources for Behavior Analysis](https://mlsanity.com/paper/2111.15186/) (2021)
- [Iterative Data Programming for Expanding Text Classification Corpora](https://mlsanity.com/paper/2002.01412/) (2020)

## Built on by (most cited first, 10 of 10)

- [PRBoost: Prompt-Based Rule Discovery and Boosting for Interactive Weakly-Supervised Learning](https://mlsanity.com/paper/2203.09735/) (2022)
- [Understanding Programmatic Weak Supervision via Source-aware Influence Function](https://mlsanity.com/paper/2205.12879/) (2022)
- [AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels](https://mlsanity.com/paper/2208.14362/) (2022)
- [Adaptive Multi-view Rule Discovery for Weakly-Supervised Compatible Products Prediction](https://mlsanity.com/paper/2206.13749/) (2022)
- [LLM-assisted Labeling Function Generation for Semantic Type Detection](https://mlsanity.com/paper/2408.16173/) (2024)
- [Interactive Machine Teaching by Labeling Rules and Instances](https://mlsanity.com/paper/2409.05199/) (2024)
- [XPASC: Measuring Generalization in Weak Supervision by Explainability and Association](https://mlsanity.com/paper/2206.01444/) (2022)
- [ActiveDP: Bridging Active Learning and Data Programming](https://mlsanity.com/paper/2402.06056/) (2024)
- [Refining Labeling Functions with Limited Labeled Data](https://mlsanity.com/paper/2505.23470/) (2025)
- [Regularized Data Programming with Automated Bayesian Prior Selection](https://mlsanity.com/paper/2210.08677/) (2022)

## Abstract (arXiv)

Weak Supervision (WS) techniques allow users to efficiently create large training datasets by programmatically labeling data with heuristic sources of supervision. While the success of WS relies heavily on the provided labeling heuristics, the process of how these heuristics are created in practice has remained under-explored. In this work, we formalize the development process of labeling heuristics as an interactive procedure, built around the existing workflow where users draw ideas from a selected set of development data for designing the heuristic sources. With the formalism, we study two core problems of how to strategically select the development data to guide users in efficiently creating informative heuristics, and how to exploit the information within the development process to contextualize and better learn from the resultant heuristics. Building upon two novel methodologies that effectively tackle the respective problems considered, we present Nemo, an end-to-end interactive system that improves the overall productivity of WS learning pipeline by an average 20% (and up to 47% in one task) compared to the prevailing WS approach.