# Pairwise Feedback for Data Programming

arXiv: https://arxiv.org/abs/1912.07685 · HTML: https://mlsanity.com/paper/1912.07685/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Dec 2019.

## Built on by (most cited first, 3 of 3)

- [Shoring Up the Foundations: Fusing Model Embeddings and Weak Supervision](https://mlsanity.com/paper/2203.13270/) (2022)
- [Classifying Unstructured Clinical Notes via Automatic Weak Supervision](https://mlsanity.com/paper/2206.12088/) (2022)
- [The Word is Mightier than the Label: Learning without Pointillistic Labels using Data Programming](https://mlsanity.com/paper/2108.10921/) (2021)

## Abstract (arXiv)

The scalability of the labeling process and the attainable quality of labels have become limiting factors for many applications of machine learning. The programmatic creation of labeled datasets via the synthesis of noisy heuristics provides a promising avenue to address this problem. We propose to improve modeling of latent class variables in the programmatic creation of labeled datasets by incorporating pairwise feedback into the process. We discuss the ease with which such pairwise feedback can be obtained or generated in many application domains. Our experiments show that even a small number of sources of pairwise feedback can substantially improve the quality of the posterior estimate of the latent class variable.