# Snorkel: Rapid Training Data Creation with Weak Supervision

arXiv: https://arxiv.org/abs/1711.10160 · HTML: https://mlsanity.com/paper/1711.10160/

## Key facts

- Cited by 372 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2017.

## Built on by (most cited first, 30 of 372)

- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision](https://mlsanity.com/paper/2312.09390/) (2023)
- [Can Large Language Models Transform Computational Social Science?](https://mlsanity.com/paper/2305.03514/) (2023)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Data-centric Artificial Intelligence: A Survey](https://mlsanity.com/paper/2303.10158/) (2023)
- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [On the Tool Manipulation Capability of Open-source Large Language Models](https://mlsanity.com/paper/2305.16504/) (2023)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes](https://mlsanity.com/paper/2304.09433/) (2023)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation](https://mlsanity.com/paper/2310.15638/) (2023)
- [Harnessing Multiple Large Language Models: A Survey on LLM Ensemble](https://mlsanity.com/paper/2502.18036/) (2025)
- [Learning to Compose Soft Prompts for Compositional Zero-Shot Learning](https://mlsanity.com/paper/2204.03574/) (2022)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Leveraging Explanations in Interactive Machine Learning: An Overview](https://mlsanity.com/paper/2207.14526/) (2022)
- [Self-Training with Weak Supervision](https://mlsanity.com/paper/2104.05514/) (2021)
- [Theoretical Analysis of Weak-to-Strong Generalization](https://mlsanity.com/paper/2405.16043/) (2024)
- [Improved Regularization and Robustness for Fine-tuning in Neural Networks](https://mlsanity.com/paper/2111.04578/) (2021)
- [TwiBot-22: Towards Graph-Based Twitter Bot Detection](https://mlsanity.com/paper/2206.04564/) (2022)
- [Contrastive language and vision learning of general fashion concepts](https://mlsanity.com/paper/2204.03972/) (2022)
- [End-to-End Weak Supervision](https://mlsanity.com/paper/2107.02233/) (2021)
- [A Data Quality-Driven View of MLOps](https://mlsanity.com/paper/2102.07750/) (2021)
- [Universalizing Weak Supervision](https://mlsanity.com/paper/2112.03865/) (2021)
- [VolcanoML: Speeding up End-to-End AutoML via Scalable Search Space Decomposition](https://mlsanity.com/paper/2107.08861/) (2021)
- [Operationalizing Machine Learning: An Interview Study](https://mlsanity.com/paper/2209.09125/) (2022)
- [Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks](https://mlsanity.com/paper/2106.06235/) (2021)
- [Co-training Improves Prompt-based Learning for Large Language Models](https://mlsanity.com/paper/2202.00828/) (2022)
- [Shrinking the Generation-Verification Gap with Weak Verifiers](https://mlsanity.com/paper/2506.18203/) (2025)

## Abstract (arXiv)

Labeling training data is increasingly the largest bottleneck in deploying machine learning systems. We present Snorkel, a first-of-its-kind system that enables users to train state-of-the-art models without hand labeling any training data. Instead, users write labeling functions that express arbitrary heuristics, which can have unknown accuracies and correlations. Snorkel denoises their outputs without access to ground truth by incorporating the first end-to-end implementation of our recently proposed machine learning paradigm, data programming. We present a flexible interface layer for writing labeling functions based on our experience over the past year collaborating with companies, agencies, and research labs. In a user study, subject matter experts build models 2.8x faster and increase predictive performance an average 45.5% versus seven hours of hand labeling. We study the modeling tradeoffs in this new setting and propose an optimizer for automating tradeoff decisions that gives up to 1.8x speedup per pipeline execution. In two collaborations, with the U.S. Department of Veterans Affairs and the U.S. Food and Drug Administration, and on four open-source text and image data sets representative of other deployments, Snorkel provides 132% average improvements to predictive performance over prior heuristic approaches and comes within an average 3.60% of the predictive performance of large hand-curated training sets.