# Universalizing Weak Supervision

arXiv: https://arxiv.org/abs/2112.03865 · HTML: https://mlsanity.com/paper/2112.03865/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 53 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Builds on (most cited first, 19 of 19)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection](https://mlsanity.com/paper/2004.10643/) (2020)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Representation Tradeoffs for Hyperbolic Embeddings](https://mlsanity.com/paper/1804.03329/) (2018)
- [Training Complex Models with Multi-Task Weak Supervision](https://mlsanity.com/paper/1810.02840/) (2018)
- [Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods](https://mlsanity.com/paper/2002.11955/) (2020)
- [Neural Ranking Models with Weak Supervision](https://mlsanity.com/paper/1704.08803/) (2017)
- [Snorkel DryBell: A Case Study in Deploying Weak Supervision at Industrial Scale](https://mlsanity.com/paper/1812.00417/) (2018)
- [Learning Dependency Structures for Weak Supervision Models](https://mlsanity.com/paper/1903.05844/) (2019)
- [Overton: A Data System for Monitoring and Improving Machine-Learned Products](https://mlsanity.com/paper/1909.05372/) (2019)
- [Multi-Resolution Weak Supervision for Sequential Data](https://mlsanity.com/paper/1910.09505/) (2019)
- [Optimal Learning of Mallows Block Model](https://mlsanity.com/paper/1906.01009/) (2019)
- [Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimation](https://mlsanity.com/paper/2103.02761/) (2021)
- [Estimation from Indirect Supervision with Linear Moments](https://mlsanity.com/paper/1608.03100/) (2016)
- [Comprehensive and Reliable Crowd Assessment Algorithms](https://mlsanity.com/paper/1411.3377/) (2014)
- [Evaluating the Crowd with Confidence](https://mlsanity.com/paper/1411.6562/) (2014)

## Built on by (most cited first, 27 of 27)

- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Language Models in the Loop: Incorporating Prompting into Weak Supervision](https://mlsanity.com/paper/2205.02318/) (2022)
- [Smoothie: Label Free Language Model Routing](https://mlsanity.com/paper/2412.04692/) (2024)
- [CARE: Confounder-Aware Aggregation for Reliable LLM Evaluation](https://mlsanity.com/paper/2603.00039/) (2026)
- [Losses over Labels: Weakly Supervised Learning via Direct Loss Construction](https://mlsanity.com/paper/2212.06921/) (2022)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [Lifting Weak Supervision To Structured Prediction](https://mlsanity.com/paper/2211.13375/) (2022)
- [Understanding Programmatic Weak Supervision via Source-aware Influence Function](https://mlsanity.com/paper/2205.12879/) (2022)
- [Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweets](https://mlsanity.com/paper/2309.06503/) (2023)
- [Multimodal Data Curation via Object Detection and Filter Ensembles](https://mlsanity.com/paper/2401.12225/) (2024)
- [AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels](https://mlsanity.com/paper/2208.14362/) (2022)
- [Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision](https://mlsanity.com/paper/2210.02724/) (2022)
- [Promises and Pitfalls of Threshold-based Auto-labeling](https://mlsanity.com/paper/2211.12620/) (2022)
- [Geometry-Aware Adaptation for Pretrained Models](https://mlsanity.com/paper/2307.12226/) (2023)
- [Mitigating Source Bias for Fairer Weak Supervision](https://mlsanity.com/paper/2303.17713/) (2023)
- [Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification](https://mlsanity.com/paper/2307.11031/) (2023)
- [Pearls from Pebbles: Improved Confidence Functions for Auto-labeling](https://mlsanity.com/paper/2404.16188/) (2024)
- [Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights](https://mlsanity.com/paper/2501.06708/) (2025)
- [An Adaptive Method for Weak Supervision with Drifting Data](https://mlsanity.com/paper/2306.01658/) (2023)
- [Leveraging Large Language Models for Structure Learning in Prompted Weak Supervision](https://mlsanity.com/paper/2402.01867/) (2024)
- [Reliable Programmatic Weak Supervision with Confidence Intervals for Label Probabilities](https://mlsanity.com/paper/2508.03896/) (2025)
- [Learning Stable Predictors from Weak Supervision under Distribution Shift](https://mlsanity.com/paper/2604.05002/) (2026)
- [Generative Modeling Helps Weak Supervision (and Vice Versa)](https://mlsanity.com/paper/2203.12023/) (2022)
- [Weakly Supervised Label Learning Flows](https://mlsanity.com/paper/2302.09649/) (2023)
- [Local Boosting for Weakly-Supervised Learning](https://mlsanity.com/paper/2306.02859/) (2023)
- [Codifying the Judge: Scalable Evaluation via Program Distillation](https://mlsanity.com/paper/2607.22561/) (2026)

## Abstract (arXiv)

Weak supervision (WS) frameworks are a popular way to bypass hand-labeling large datasets for training data-hungry models. These approaches synthesize multiple noisy but cheaply-acquired estimates of labels into a set of high-quality pseudolabels for downstream training. However, the synthesis technique is specific to a particular kind of label, such as binary labels or sequences, and each new label type requires manually designing a new synthesis algorithm. Instead, we propose a universal technique that enables weak supervision over any label type while still offering desirable properties, including practical flexibility, computational efficiency, and theoretical guarantees. We apply this technique to important problems previously not tackled by WS frameworks including learning to rank, regression, and learning in hyperbolic space. Theoretically, our synthesis approach produces a consistent estimators for learning some challenging but important generalizations of the exponential family model. Experimentally, we validate our framework and show improvement over baselines in diverse settings including real-world learning-to-rank and regression problems along with learning on hyperbolic manifolds.