# Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale

arXiv: https://arxiv.org/abs/1812.00417 · HTML: https://mlsanity.com/paper/1812.00417/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2018.

## Built on by (most cited first, 30 of 52)

- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotation](https://mlsanity.com/paper/2310.15638/) (2023)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Language Models in the Loop: Incorporating Prompting into Weak Supervision](https://mlsanity.com/paper/2205.02318/) (2022)
- [Self-Training with Weak Supervision](https://mlsanity.com/paper/2104.05514/) (2021)
- [End-to-End Weak Supervision](https://mlsanity.com/paper/2107.02233/) (2021)
- [Universalizing Weak Supervision](https://mlsanity.com/paper/2112.03865/) (2021)
- [Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision](https://mlsanity.com/paper/2203.13270/) (2022)
- [Learning Hyper Label Model for Programmatic Weak Supervision](https://mlsanity.com/paper/2207.13545/) (2022)
- [Learning from Multiple Noisy Partial Labelers](https://mlsanity.com/paper/2106.04530/) (2021)
- [Creating Training Sets via Weak Indirect Supervision](https://mlsanity.com/paper/2110.03484/) (2021)
- [CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation](https://mlsanity.com/paper/2603.00039/) (2026)
- [Active WeaSuL: Improving Weak Supervision with Active Learning](https://mlsanity.com/paper/2104.14847/) (2021)
- [Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming](https://mlsanity.com/paper/2109.11410/) (2021)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [Toward a `Standard Model' of Machine Learning](https://mlsanity.com/paper/2108.07783/) (2021)
- [Understanding Programmatic Weak Supervision via Source-aware Influence Function](https://mlsanity.com/paper/2205.12879/) (2022)
- [Multimodal Data Curation via Object Detection and Filter Ensembles](https://mlsanity.com/paper/2401.12225/) (2024)
- [AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels](https://mlsanity.com/paper/2208.14362/) (2022)
- [Dependency Structure Misspecification in Multi-Source Weak Supervision Models](https://mlsanity.com/paper/2106.10302/) (2021)
- [SPEAR : Semi-supervised Data Programming in Python](https://mlsanity.com/paper/2108.00373/) (2021)
- [Learning to Detect Noisy Labels Using Model-Based Features](https://mlsanity.com/paper/2212.13767/) (2022)
- [Can Large Language Models Design Accurate Label Functions?](https://mlsanity.com/paper/2311.00739/) (2023)
- [Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMware](https://mlsanity.com/paper/2402.15943/) (2024)
- [Quality over Quantity: Boosting Data Efficiency Through Ensembled Multimodal Data Curation](https://mlsanity.com/paper/2502.08211/) (2025)
- [Mitigating Source Bias for Fairer Weak Supervision](https://mlsanity.com/paper/2303.17713/) (2023)
- [MLOps Spanning Whole Machine Learning Life Cycle: A Survey](https://mlsanity.com/paper/2304.07296/) (2023)
- [ScriptoriumWS: A Code Generation Assistant for Weak Supervision](https://mlsanity.com/paper/2502.12366/) (2025)

## Abstract (arXiv)

Labeling training data is one of the most costly bottlenecks in developing machine learning-based applications. We present a first-of-its-kind study showing how existing knowledge resources from across an organization can be used as weak supervision in order to bring development time and cost down by an order of magnitude, and introduce Snorkel DryBell, a new weak supervision management system for this setting. Snorkel DryBell builds on the Snorkel framework, extending it in three critical aspects: flexible, template-based ingestion of diverse organizational knowledge, cross-feature production serving, and scalable, sampling-free execution. On three classification tasks at Google, we find that Snorkel DryBell creates classifiers of comparable quality to ones trained with tens of thousands of hand-labeled examples, converts non-servable organizational resources to servable models for an average 52% performance improvement, and executes over millions of data points in tens of minutes.