# Learning Dependency Structures for Weak Supervision Models

arXiv: https://arxiv.org/abs/1903.05844 · HTML: https://mlsanity.com/paper/1903.05844/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2019.

## Built on by (most cited first, 30 of 32)

- [Ask Me Anything: A simple strategy for prompting language models](https://mlsanity.com/paper/2210.02441/) (2022)
- [Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes](https://mlsanity.com/paper/2304.09433/) (2023)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [Language Models in the Loop: Incorporating Prompting into Weak Supervision](https://mlsanity.com/paper/2205.02318/) (2022)
- [Theoretical Analysis of Weak-to-Strong Generalization](https://mlsanity.com/paper/2405.16043/) (2024)
- [End-to-End Weak Supervision](https://mlsanity.com/paper/2107.02233/) (2021)
- [Smoothie: Label Free Language Model Routing](https://mlsanity.com/paper/2412.04692/) (2024)
- [Universalizing Weak Supervision](https://mlsanity.com/paper/2112.03865/) (2021)
- [DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples](https://mlsanity.com/paper/2110.13740/) (2021)
- [Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision](https://mlsanity.com/paper/2203.13270/) (2022)
- [Learning from Multiple Noisy Partial Labelers](https://mlsanity.com/paper/2106.04530/) (2021)
- [Active WeaSuL: Improving Weak Supervision with Active Learning](https://mlsanity.com/paper/2104.14847/) (2021)
- [Trustable Co-label Learning from Multiple Noisy Annotators](https://mlsanity.com/paper/2203.04199/) (2022)
- [Comparing the Value of Labeled and Unlabeled Data in Method-of-Moments Latent Variable Estimation](https://mlsanity.com/paper/2103.02761/) (2021)
- [Knodle: Modular Weakly Supervised Learning with PyTorch](https://mlsanity.com/paper/2104.11557/) (2021)
- [Dependency Structure Misspecification in Multi-Source Weak Supervision Models](https://mlsanity.com/paper/2106.10302/) (2021)
- [Resonant Anomaly Detection with Multiple Reference Datasets](https://mlsanity.com/paper/2212.10579/) (2022)
- [Making Binary Classification from Multiple Unlabeled Datasets Almost Free of Supervision](https://mlsanity.com/paper/2306.07036/) (2023)
- [Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classification](https://mlsanity.com/paper/2307.11031/) (2023)
- [A data-centric weak supervised learning for highway traffic incident detection](https://mlsanity.com/paper/2112.09792/) (2021)
- [Label Augmentation with Reinforced Labeling for Weak Supervision](https://mlsanity.com/paper/2204.06436/) (2022)
- [An Adaptive Method for Weak Supervision with Drifting Data](https://mlsanity.com/paper/2306.01658/) (2023)
- [Leveraging Large Language Models for Structure Learning in Prompted Weak Supervision](https://mlsanity.com/paper/2402.01867/) (2024)
- [Automating Weak Label Generation for Data Programming with Clinicians in the Loop](https://mlsanity.com/paper/2407.07982/) (2024)
- [Reliable Programmatic Weak Supervision with Confidence Intervals for Label Probabilities](https://mlsanity.com/paper/2508.03896/) (2025)
- [Proceedings of the First Workshop on Weakly Supervised Learning (WeaSuL)](https://mlsanity.com/paper/2107.03690/) (2021)
- [WeaNF: Weak Supervision with Normalizing Flows](https://mlsanity.com/paper/2204.13409/) (2022)
- [Weak Supervision with Incremental Source Accuracy Estimation](https://mlsanity.com/paper/2205.05302/) (2022)
- [A Benchmark Generative Probabilistic Model for Weak Supervised Learning](https://mlsanity.com/paper/2303.17841/) (2023)

## Abstract (arXiv)

Labeling training data is a key bottleneck in the modern machine learning pipeline. Recent weak supervision approaches combine labels from multiple noisy sources by estimating their accuracies without access to ground truth labels; however, estimating the dependencies among these sources is a critical challenge. We focus on a robust PCA-based algorithm for learning these dependency structures, establish improved theoretical recovery rates, and outperform existing methods on various real-world tasks. Under certain conditions, we show that the amount of unlabeled data needed can scale sublinearly or even logarithmically with the number of sources m, improving over previous efforts that ignore the sparsity pattern in the dependency structure and scale linearly in m. We provide an information-theoretic lower bound on the minimum sample complexity of the weak supervision setting. Our method outperforms weak supervision approaches that assume conditionally-independent sources by up to 4.64 F1 points and previous structure learning approaches by up to 4.41 F1 points on real-world relation extraction and image classification tasks.