# GOGGLES: Automatic Image Labeling with Affinity Coding

arXiv: https://arxiv.org/abs/1903.04552 · HTML: https://mlsanity.com/paper/1903.04552/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2019.

## Built on by (most cited first, 19 of 19)

- [WRENCH: A Comprehensive Benchmark for Weak Supervision](https://mlsanity.com/paper/2109.11377/) (2021)
- [OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning](https://mlsanity.com/paper/2103.09055/) (2021)
- [DP-SSL: Towards Robust Semi-supervised Learning with A Few Labeled Samples](https://mlsanity.com/paper/2110.13740/) (2021)
- [Learning Hyper Label Model for Programmatic Weak Supervision](https://mlsanity.com/paper/2207.13545/) (2022)
- [Nemo: Guiding and Contextualizing Weak Supervision for Interactive Data Programming](https://mlsanity.com/paper/2203.01382/) (2022)
- [The ALCHEmist: Automated Labeling 500x CHEaper Than LLM Data Annotators](https://mlsanity.com/paper/2407.11004/) (2024)
- [AutoWS-Bench-101: Benchmarking Automated Weak Supervision with 100 Labels](https://mlsanity.com/paper/2208.14362/) (2022)
- [Ground Truth Inference for Weakly Supervised Entity Matching](https://mlsanity.com/paper/2211.06975/) (2022)
- [Rapid Image Labeling via Neuro-Symbolic Learning](https://mlsanity.com/paper/2306.10490/) (2023)
- [Zeus: Efficiently Localizing Actions in Videos using Reinforcement Learning](https://mlsanity.com/paper/2104.06142/) (2021)
- [ScriptoriumWS: A Code Generation Assistant for Weak Supervision](https://mlsanity.com/paper/2502.12366/) (2025)
- [A Cost-Effective LLM-based Approach to Identify Wildlife Trafficking in Online Marketplaces](https://mlsanity.com/paper/2504.21211/) (2025)
- [CHEF: A Cheap and Fast Pipeline for Iteratively Cleaning Label Uncertainties (Technical Report)](https://mlsanity.com/paper/2107.08588/) (2021)
- [Binary Classification with Positive Labeling Sources](https://mlsanity.com/paper/2208.01704/) (2022)
- [RBT4DNN: Requirements-based Testing of Neural Networks](https://mlsanity.com/paper/2504.02737/) (2025)
- [Label Augmentation with Reinforced Labeling for Weak Supervision](https://mlsanity.com/paper/2204.06436/) (2022)
- [Automating Weak Label Generation for Data Programming with Clinicians in the Loop](https://mlsanity.com/paper/2407.07982/) (2024)
- [Reliable Programmatic Weak Supervision with Confidence Intervals for Label Probabilities](https://mlsanity.com/paper/2508.03896/) (2025)
- [Structured Exploration and Exploitation of Label Functions for Automated Data Annotation](https://mlsanity.com/paper/2604.08578/) (2026)

## Abstract (arXiv)

Generating large labeled training data is becoming the biggest bottleneck in building and deploying supervised machine learning models. Recently, the data programming paradigm has been proposed to reduce the human cost in labeling training data. However, data programming relies on designing labeling functions which still requires significant domain expertise. Also, it is prohibitively difficult to write labeling functions for image datasets as it is hard to express domain knowledge using raw features for images (pixels). We propose affinity coding, a new domain-agnostic paradigm for automated training data labeling. The core premise of affinity coding is that the affinity scores of instance pairs belonging to the same class on average should be higher than those of pairs belonging to different classes, according to some affinity functions. We build the GOGGLES system that implements affinity coding for labeling image datasets by designing a novel set of reusable affinity functions for images, and propose a novel hierarchical generative model for class inference using a small development set. We compare GOGGLES with existing data programming systems on 5 image labeling tasks from diverse domains. GOGGLES achieves labeling accuracies ranging from a minimum of 71% to a maximum of 98% without requiring any extensive human annotation. In terms of end-to-end performance, GOGGLES outperforms the state-of-the-art data programming system Snuba by 21% and a state-of-the-art few-shot learning technique by 5%, and is only 7% away from the fully supervised upper bound.