# Generating the Ground Truth: Synthetic Data for Soft Label and Label Noise Research

arXiv: https://arxiv.org/abs/2309.04318 · HTML: https://mlsanity.com/paper/2309.04318/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 8 Sep 2023.

## Builds on (most cited first, 11 of 11)

- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Learning from Noisy Labels with Deep Neural Networks: A Survey](https://mlsanity.com/paper/2007.08199/) (2020)
- [WebVision Database: Visual Learning and Understanding from Web Data](https://mlsanity.com/paper/1708.02862/) (2017)
- [CleanNet: Transfer Learning for Scalable Image Classifier Training with Label Noise](https://mlsanity.com/paper/1711.07131/) (2017)
- [Training Deep Networks for Facial Expression Recognition with Crowd-Sourced Label Distribution](https://mlsanity.com/paper/1608.01041/) (2016)
- [Label Distribution Learning](https://mlsanity.com/paper/1408.6027/) (2014)
- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Beyond Class-Conditional Assumption: A Primary Attempt to Combat Instance-Dependent Label Noise](https://mlsanity.com/paper/2012.05458/) (2020)
- [Confidence Scores Make Instance-dependent Label-noise Learning Possible](https://mlsanity.com/paper/2001.03772/) (2020)
- [Learning with Bounded Instance- and Label-dependent Label Noise](https://mlsanity.com/paper/1709.03768/) (2017)
- [An Instance-Dependent Simulation Framework for Learning with Label Noise](https://mlsanity.com/paper/2107.11413/) (2021)

## Built on by (most cited first, 2 of 2)

- [Learning with Confidence: Training Better Classifiers from Soft Labels](https://mlsanity.com/paper/2409.16071/) (2024)
- [Position: Every Ground Truth is a Human Construction, not an Objective Truth](https://mlsanity.com/paper/2607.09668/) (2026)

## Abstract (arXiv)

In many real-world classification tasks, label noise is an unavoidable issue that adversely affects the generalization error of machine learning models. Additionally, evaluating how methods handle such noise is complicated, as the effect label noise has on their performance cannot be accurately quantified without clean labels. Existing research on label noise typically relies on either noisy or oversimplified simulated data as a baseline, into which additional noise with known properties is injected. In this paper, we introduce SYNLABEL, a framework designed to address these limitations by creating noiseless datasets informed by real-world data. SYNLABEL supports defining a pre-specified or learned function as the ground truth function, which can then be used for generating new clean labels. Furthermore, by repeatedly resampling values for selected features within the domain of the function, evaluating the function and aggregating the resulting labels, each data point can be assigned a soft label or label distribution. These distributions capture the inherent uncertainty present in many real-world datasets and enable the direct injection and quantification of label noise. The generated datasets serve as a clean baseline of adjustable complexity, into which various types of noise can be introduced. Additionally, they facilitate research into soft label learning and related applications. We demonstrate the application of SYNLABEL, showcasing its ability to precisely quantify label noise and its improvement over existing methodologies.