# Position: Every Ground Truth is a Human Construction, not an Objective Truth

arXiv: https://arxiv.org/abs/2607.09668 · HTML: https://mlsanity.com/paper/2607.09668/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 61 references that are in the map.
- First posted on arXiv 28 May 2026.

## Builds on (most cited first, 9 of 9)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation](https://mlsanity.com/paper/2211.02570/) (2022)
- [Toward a Perspectivist Turn in Ground Truthing for Predictive Computing](https://mlsanity.com/paper/2109.04270/) (2021)
- [Procedural Modeling and Physically Based Rendering for Synthetic Data Generation in Automotive Applications](https://mlsanity.com/paper/1710.06270/) (2017)
- [Ground Truth Or Dare: Factors Affecting The Creation Of Medical Datasets For Training AI](https://mlsanity.com/paper/2309.12327/) (2023)
- [Generating the Ground Truth: Synthetic Data for Soft Label and Label Noise Research](https://mlsanity.com/paper/2309.04318/) (2023)

## Abstract (arXiv)

Ground truth datasets play a fundamental role as reference values in the training and evaluation of machine learning models. This position paper argues that ground truths are not neutral objective measurements that are naturally given, but instead that they are constructed by arrangements of humans and technologies. We argue that the ML community will benefit from articulating and discussing these often invisible or unreported choices and acknowledging that reference data sets are contingent, not universal. Focusing on the situated and context-dependent nature of ground truths can improve reliability by enabling a better informed perspective on where, when, and how the datasets, and the models they have shaped, can best be used. We argue for increasing `situated reliability' which includes articulating the limits and strengths of models and their truth claims. Finally, paying more attention to the construction of ground truths can support transparency, accountability, and interdisciplinary work.