mlsanityData to 14 Oct 2026

Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale

arXiv 1812.00417 ↗First posted 2 Dec 2018cs.LG stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Labeling training data is one of the most costly bottlenecks in developing machine learning-based applications. We present a first-of-its-kind study showing how existing knowledge resources from across an organization can be used as weak supervision in order to bring development time and cost down by an order of magnitude, and introduce Snorkel DryBell, a new weak supervision management system for this setting. Snorkel DryBell builds on the Snorkel framework, extending it in three critical aspects: flexible, template-based ingestion of diverse organizational knowledge, cross-feature production serving, and scalable, sampling-free execution. On three classification tasks at Google, we find that Snorkel DryBell creates classifiers of comparable quality to ones trained with tens of thousands of hand-labeled examples, converts non-servable organizational resources to servable models for an average 52% performance improvement, and executes over millions of data points in tens of minutes.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). First posted on arXiv 2 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 14; 2022: 17; 2023: 9; 2024: 7; 2025: 3; 2026: 2.

52 papers.

Sort:

By area:

  1. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  2. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  3. 2023CoAnnotating: Uncertainty-Guided Work Allocation between Human and Large Language Models for Data Annotationcited by 94
  4. 2022A Survey on Programmatic Weak Supervisioncited by 66
  5. 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 52
  6. 2021Self-Training with Weak Supervisioncited by 49
  7. 2021End-to-End Weak Supervisioncited by 33
  8. 2021Universalizing Weak Supervisioncited by 27
  9. 2022Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervisioncited by 18
  10. 2022Learning Hyper Label Model for Programmatic Weak Supervisioncited by 16
  11. 2021Learning from Multiple Noisy Partial Labelerscited by 14
  12. 2021Creating Training Sets via Weak Indirect Supervisioncited by 14
38 more
  1. 2026CARE: Confounder-Aware Aggregation for Reliable LLM Evaluationcited by 13
  2. 2021Active WeaSuL: Improving Weak Supervision with Active Learningcited by 12
  3. 2021Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programmingcited by 11
  4. 2022Losses over Labels: Weakly Supervised Learning via Direct Loss Constructioncited by 11
  5. 2022Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programmingcited by 10
  6. 2021Toward a `Standard Model' of Machine Learningcited by 8
  7. 2022Understanding Programmatic Weak Supervision via Source-aware Influence Functioncited by 8
  8. 2024Multimodal Data Curation via Object Detection and Filter Ensemblescited by 8
  9. 2022AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labelscited by 7
  10. 2021Dependency Structure Misspecification in Multi-Source Weak Supervision Modelscited by 6
  11. 2021SPEAR : Semi-supervised Data Programming in Pythoncited by 6
  12. 2022Learning to Detect Noisy Labels Using Model-Based Featurescited by 6
  13. 2023Can Large Language Models Design Accurate Label Functions?cited by 6
  14. 2024Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMwarecited by 6
  15. 2025Quality over Quantity: Boosting Data Efficiency Through Ensembled Multimodal Data Curationcited by 5
  16. 2023Mitigating Source Bias for Fairer Weak Supervisioncited by 3
  17. 2023MLOps Spanning Whole Machine Learning Life Cycle: A Surveycited by 3
  18. 2025ScriptoriumWS: A Code Generation Assistant for Weak Supervisioncited by 3
  19. 2021CHEF: A Cheap and Fast Pipeline for Iteratively Cleaning Label Uncertainties (Technical Report)cited by 2
  20. 2022Data Consistency for Weakly Supervised Learningcited by 2
  21. 2022Binary Classification with Positive Labeling Sourcescited by 2
  22. 2023Going beyond research datasets: Novel intent discovery in the industry settingcited by 2
  23. 2024Interactive Machine Teaching by Labeling Rules and Instancescited by 2
  24. 2022Label Augmentation with Reinforced Labeling for Weak Supervisioncited by 1
  25. 2023An Adaptive Method for Weak Supervision with Drifting Datacited by 1
  26. 2023Fusing Conditional Submodular GAN and Programmatic Weak Supervisioncited by 1
  27. 2024Leveraging Large Language Models for Structure Learning in Prompted Weak Supervisioncited by 1
  28. 2024FAIR: Filtering of Automatically Induced Rulescited by 1
  29. 2024WeShap: Weak Supervision Source Evaluation with Shapley Valuescited by 1
  30. 2025How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomycited by 1
  31. 2021Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)cited by 0
  32. 2021Categorizing Items with Short and Noisy Descriptions using Ensembled Transferred Embeddingscited by 0
  33. 2022Generative Modeling Helps Weak Supervision (and Vice Versa)cited by 0
  34. 2022Integrated Weak Learningcited by 0
  35. 2022Weak Supervision in Analysis of News: Application to Economic Policy Uncertaintycited by 0
  36. 2022Regularized Data Programming with Automated Bayesian Prior Selectioncited by 0
  37. 2022Label-Efficient Interactive Time-Series Anomaly Detectioncited by 0
  38. 2023Weakly Supervised Label Learning Flowscited by 0

50 most cited of 52 All 52 (JSON)