mlsanityData to 14 Oct 2026

Training Subset Selection for Weak Supervision

arXiv 2206.02914 ↗First posted 6 Jun 2022stat.ML cs.AI cs.LGcited by 19

Not on the map. Why?

Abstract (arXiv)

Existing weak supervision approaches use all the data covered by weak signals to train a classifier. We show both theoretically and empirically that this is not always optimal. Intuitively, there is a tradeoff between the amount of weakly-labeled data and the precision of the weak labels. We explore this tradeoff by combining pretrained data representations with the cut statistic (Muhlenbach et al., 2004) to select (hopefully) high-quality subsets of the weakly-labeled training data. Subset selection applies to any label model and classifier and is very simple to plug in to existing weak supervision pipelines, requiring just a few lines of code. We show our subset selection method improves the performance of weak supervision for a wide range of label models, classifiers, and datasets. Using less weakly-labeled data improves the accuracy of weak supervision pipelines by up to 19% (absolute) on benchmark tasks.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 26 of its 53 references that are in the map. First posted on arXiv 6 Jun 2022.

Builds on

26 of 53 references.

References · 26

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  5. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,947
  6. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  7. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  8. 2020Domain-Specific Language Model Pretraining for Biomedical Natural Language Processingcited by 1,009
  9. 2020DivideMix: Learning with Noisy Labels as Semi-supervised Learningcited by 692
  10. 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 665
  11. 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 413
  12. 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 372
14 more
  1. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  2. 2020Learning with Instance-Dependent Label Noise: A Sample Sieve Approachcited by 158
  3. 2020CheXclusion: Fairness gaps in deep chest X-ray classifierscited by 153
  4. 2018Training Complex Models with Multi-Task Weak Supervisioncited by 106
  5. 2020Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methodscited by 98
  6. 2021WRENCH: A Comprehensive Benchmark for Weak Supervisioncited by 96
  7. 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
  8. 2013Classification with Asymmetric Label Noise: Consistency and Maximal Denoisingcited by 68
  9. 2021Self-Training with Weak Supervisioncited by 49
  10. 2022Co-training Improves Prompt-based Learning for Large Language Modelscited by 24
  11. 2022Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervisioncited by 18
  12. 2020Semi-Supervised Data Programming with Subset Selectioncited by 16
  13. 2022LOPS: Learning Order Inspired Pseudo-Label Selection for Weakly Supervised Text Classificationcited by 11
  14. 2018Adversarial Label Learningcited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 5; 2025: 2; 2026: 4.

19 papers.

Sort:

By area:

  1. 2024Theoretical Analysis of Weak-to-Strong Generalizationcited by 48
  2. 2023ReGen: Zero-Shot Text Classification via Training Data Generation with Progressive Dense Retrievalcited by 26
  3. 2023Neighborhood-Regularized Self-Training for Learning with Few Labelscited by 14
  4. 2023SLaM: Student-Label Mixing for Distillation with Unlabeled Examplescited by 10
  5. 2024Learning with Less: Knowledge Distillation from Large Language Models via Unlabeled Datacited by 8
  6. 2025Merge to Mix: Mixing Datasets via Model Mergingcited by 7
  7. 2025Generalization Bounds and Stopping Rules for Learning with Self-Selected Datacited by 5
  8. 2023Mitigating Source Bias for Fairer Weak Supervisioncited by 3
  9. 2023Embroid: Unsupervised Prediction Smoothing Can Improve Few-Shot Classificationcited by 3
  10. 2024WISER: Weak supervISion and supErvised Representation learning to improve drug response prediction in cancercited by 3
  11. 2023PV2TEA: Patching Visual Modality to Textual-Established Information Extractioncited by 2
  12. 2023Fusing Conditional Submodular GAN and Programmatic Weak Supervisioncited by 1
7 more
  1. 2026DC-W2S: Dual-Consensus Weak-to-Strong Training for Reliable Process Reward Modeling in Biological Reasoningcited by 1
  2. 2022Regularized Data Programming with Automated Bayesian Prior Selectioncited by 0
  3. 2024Reciprocal Learningcited by 0
  4. 2024SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolationcited by 0
  5. 2026From Skeletons to Pixels: Few-Shot Precise Event Spotting via Representation and Prediction Distillationcited by 0
  6. 2026Leveraging Data Symmetries to Select an Optimal Subset of Training Data under Label Noisecited by 0
  7. 2026Trust Functions: Near-Lossless Weak-to-Strong Generalization by Learning When to Trust the Weak Teachercited by 0