# Refining Labeling Functions with Limited Labeled Data

arXiv: https://arxiv.org/abs/2505.23470 · HTML: https://mlsanity.com/paper/2505.23470/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 29 May 2025.

## Builds on (most cited first, 22 of 22)

- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Confident Learning: Estimating Uncertainty in Dataset Labels](https://mlsanity.com/paper/1911.00068/) (2019)
- [Snorkel: Rapid Training Data Creation with Weak Supervision](https://mlsanity.com/paper/1711.10160/) (2017)
- [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019)
- [Data Programming: Creating Large Training Sets, Quickly](https://mlsanity.com/paper/1605.07723/) (2016)
- [Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes](https://mlsanity.com/paper/2304.09433/) (2023)
- [Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods](https://mlsanity.com/paper/2002.11955/) (2020)
- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach](https://mlsanity.com/paper/2010.07835/) (2020)
- [HoloClean: Holistic Data Repairs with Probabilistic Inference](https://mlsanity.com/paper/1702.00820/) (2017)
- [A Survey on Programmatic Weak Supervision](https://mlsanity.com/paper/2202.05433/) (2022)
- [CrossWeigh: Training Named Entity Tagger from Imperfect Annotations](https://mlsanity.com/paper/1909.01441/) (2019)
- [Training Classifiers with Natural Language Explanations](https://mlsanity.com/paper/1805.03818/) (2018)
- [Interactive Weak Supervision: Learning Useful Heuristics for Data Labeling](https://mlsanity.com/paper/2012.06046/) (2020)
- [Adaptive Rule Discovery for Labeling Text Data](https://mlsanity.com/paper/2005.06133/) (2020)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [Understanding Programmatic Weak Supervision via Source-aware Influence Function](https://mlsanity.com/paper/2205.12879/) (2022)
- [Alfred: A System for Prompted Weak Supervision](https://mlsanity.com/paper/2305.18623/) (2023)
- [LLM-assisted Labeling Function Generation for Semantic Type Detection](https://mlsanity.com/paper/2408.16173/) (2024)
- [ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision](https://mlsanity.com/paper/2204.06863/) (2022)
- [Optimising Human-Machine Collaboration for Efficient High-Precision Information Extraction from Text Documents](https://mlsanity.com/paper/2302.09324/) (2023)

## Built on by (most cited first, 1 of 1)

- [Structured Exploration and Exploitation of Label Functions for Automated Data Annotation](https://mlsanity.com/paper/2604.08578/) (2026)

## Abstract (arXiv)

Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints. However, the quality of the generated labels depends directly on the accuracy of the LFs. In this work, we study the problem of fixing LFs based on a small set of labeled examples. Towards this goal, we develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples such that the fixed LFs ensure that (i) there is sufficient evidence for the correct label of each labeled datapoint and (ii) the accuracy of each repaired LF is sufficiently high. We model LFs as conditional rules which enables us to refine them, i.e., to selectively change their output for some inputs. We demonstrate experimentally that our system improves the quality of LFs based on surprisingly small sets of labeled datapoints.