ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision
Not on the map. Why?
Abstract (arXiv)
A cost-effective alternative to manual data labeling is weak supervision (WS), where data samples are automatically annotated using a predefined set of labeling functions (LFs), rule-based mechanisms that generate artificial labels for the associated classes. In this work, we investigate noise reduction techniques for WS based on the principle of k-fold cross-validation. We introduce a new algorithm ULF for Unsupervised Labeling Function correction, which denoises WS data by leveraging models trained on all but some LFs to identify and correct biases specific to the held-out LFs. Specifically, ULF refines the allocation of LFs to classes by re-estimating this assignment on highly reliable cross-validated samples. Evaluation on multiple datasets confirms ULF's effectiveness in enhancing WS learning without the need for manual labeling.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 71 references that are in the map. First posted on arXiv 14 Apr 2022.
Builds on
29 of 71 references.
References · 29
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2016Making Deep Neural Networks Robust to Label Noise: a Loss Correction Approachcited by 663
- 2019Symmetric Cross Entropy for Robust Learning with Noisy Labelscited by 499
- 2019Confident Learning: Estimating Uncertainty in Dataset Labelscited by 411
- 2017Snorkel: Rapid Training Data Creation with Weak Supervisioncited by 370
- 2018Training Complex Models with Multi-Task Weak Supervisioncited by 106
- 2020Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methodscited by 98
- 2021WRENCH: A Comprehensive Benchmark for Weak Supervisioncited by 95
- 2020Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approachcited by 83
- 2019Fine-Grained Spoiler Detection from Large-Scale Review Corporacited by 76
- 2019CrossWeigh: Training Named Entity Tagger from Imperfect Annotationscited by 60
17 more
- 2020Learning from Rules Generalizing Labeled Exemplarscited by 59
- 2020Transfer Learning and Distant Supervision for Multilingual Transformer Models: A Study on African Languagescited by 52
- 2020Interactive Weak Supervision: Learning Useful Heuristics for Data Labelingcited by 52
- 2021Self-Training with Weak Supervisioncited by 49
- 2020Denoising Multi-Source Weak Supervision for Neural Text Classificationcited by 42
- 2021End-to-End Weak Supervisioncited by 33
- 2020Leveraging Multi-Source Weak Social Supervision for Early Detection of Fake Newscited by 24
- 2019Learning to Contextually Aggregate Multi-Source Supervision for Sequence Labelingcited by 22
- 2020Ontology-driven weak supervision for clinical entity classification in electronic health recordscited by 18
- 2020Semi-Supervised Data Programming with Subset Selectioncited by 16
- 2018Training a Neural Network in a Low-Resource Setting on Automatically Annotated Noisy Datacited by 8
- 2020Weak Supervision and Referring Attention for Temporal-Textual Association Learningcited by 8
- 2021ANEA: Distant Supervision for Low-Resource Named Entity Recognitioncited by 8
- 2021Knodle: Modular Weakly Supervised Learning with PyTorchcited by 7
- 2018Distant Supervision for Relation Extraction with Linear Attenuation Simulation and Non-IID Relevance Embeddingcited by 5
- 2019Feature-Dependent Confusion Matrices for Low-Resource NER Labeling with Noisy Labelscited by 5
- 2021Hybrid Supervision Learning for Pathology Whole Slide Image Classificationcited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.
- 2025Refining Labeling Functions with Limited Labeled Datacited by 1
- 2025Refining Labeling Functions with Limited Labeled Data29 May 2025