# Data Representativity for Machine Learning and AI Systems

arXiv: https://arxiv.org/abs/2203.04706 · HTML: https://mlsanity.com/paper/2203.04706/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 117 references that are in the map.
- First posted on arXiv 9 Mar 2022.

## Builds on (most cited first, 30 of 36)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [Retiring Adult: New Datasets for Fair Machine Learning](https://mlsanity.com/paper/2108.04884/) (2021)
- [An Underwater Image Enhancement Benchmark Dataset and Beyond](https://mlsanity.com/paper/1901.05495/) (2019)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [A Dataset for Answering Time-Sensitive Questions](https://mlsanity.com/paper/2108.06314/) (2021)
- [No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World](https://mlsanity.com/paper/1711.08536/) (2017)
- [Why Is My Classifier Discriminatory?](https://mlsanity.com/paper/1805.12002/) (2018)
- [AP-10K: A Benchmark for Animal Pose Estimation in the Wild](https://mlsanity.com/paper/2108.12617/) (2021)
- [Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks](https://mlsanity.com/paper/2107.07455/) (2021)
- [Chaos as an interpretable benchmark for forecasting and data-driven modelling](https://mlsanity.com/paper/2110.05266/) (2021)
- [PRM-RL: Long-range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-based Planning](https://mlsanity.com/paper/1710.03937/) (2017)
- [Synthetic Benchmarks for Scientific Research in Explainable Machine Learning](https://mlsanity.com/paper/2106.12543/) (2021)
- [PASS: An ImageNet replacement for self-supervised pretraining without humans](https://mlsanity.com/paper/2109.13228/) (2021)
- [Achieving Equalized Odds by Resampling Sensitive Attributes](https://mlsanity.com/paper/2006.04292/) (2020)
- [Generating Datasets of 3D Garments with Sewing Patterns](https://mlsanity.com/paper/2109.05633/) (2021)
- [Value-laden Disciplinary Shifts in Machine Learning](https://mlsanity.com/paper/1912.01172/) (2019)
- [An Extensible Benchmark Suite for Learning to Simulate Physical Systems](https://mlsanity.com/paper/2108.07799/) (2021)
- [Assessing and Remedying Coverage for a Given Dataset](https://mlsanity.com/paper/1810.06742/) (2018)
- [A Deep Reinforcement Learning Approach to Marginalized Importance Sampling with the Successor Representation](https://mlsanity.com/paper/2106.06854/) (2021)
- [How to be Fair and Diverse?](https://mlsanity.com/paper/1610.07183/) (2016)
- [SynthBio: A Case Study in Human-AI Collaborative Curation of Text Datasets](https://mlsanity.com/paper/2111.06467/) (2021)
- [Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management](https://mlsanity.com/paper/2108.01764/) (2021)
- [Discrimination through optimization: How Facebook's ad delivery can lead to skewed outcomes](https://mlsanity.com/paper/1904.02095/) (2019)
- [HumBugDB: A Large-scale Acoustic Mosquito Dataset](https://mlsanity.com/paper/2110.07607/) (2021)
- [Sampling Algorithms, from Survey Sampling to Monte Carlo Methods: Tutorial and Literature Review](https://mlsanity.com/paper/2011.00901/) (2020)

## Built on by (most cited first, 15 of 15)

- [Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences](https://mlsanity.com/paper/2407.09499/) (2024)
- [Vox Populi, Vox AI? Using Language Models to Estimate German Public Opinion](https://mlsanity.com/paper/2407.08563/) (2024)
- [United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections](https://mlsanity.com/paper/2409.09045/) (2024)
- [Coresets from Trajectories: Selecting Data via Correlation of Loss Differences](https://mlsanity.com/paper/2508.20230/) (2025)
- [A Self-Organizing Clustering System for Unsupervised Distribution Shift Detection](https://mlsanity.com/paper/2404.16656/) (2024)
- [Dataset Diversity Metrics and Impact on Classification Models](https://mlsanity.com/paper/2603.15276/) (2026)
- [Connecting Algorithmic Fairness to Quality Dimensions in Machine Learning in Official Statistics and Survey Production](https://mlsanity.com/paper/2402.09328/) (2024)
- [Unsupervised Assessment of Landscape Shifts Based on Persistent Entropy and Topological Preservation](https://mlsanity.com/paper/2410.04183/) (2024)
- [Predictive Representativity: Uncovering Racial Bias in AI-based Skin Cancer Detection](https://mlsanity.com/paper/2507.14176/) (2025)
- [Intuitions of Machine Learning Researchers about Transfer Learning for Medical Image Classification](https://mlsanity.com/paper/2510.00902/) (2025)
- [Predict Training Data Quality via Its Geometry in Metric Space](https://mlsanity.com/paper/2510.15970/) (2025)
- [Unravelling the (In)compatibility of Statistical-Parity and Equalized-Odds](https://mlsanity.com/paper/2601.19035/) (2026)
- [Measuring Dataset Diversity from a Geometric Perspective](https://mlsanity.com/paper/2602.09340/) (2026)
- [Position: Stop Preaching and Start Practising Data Frugality for Responsible Development of AI](https://mlsanity.com/paper/2602.19789/) (2026)
- [Robustness of transferability estimation metrics for medical imaging](https://mlsanity.com/paper/2608.09999/) (2026)

## Abstract (arXiv)

Data representativity is crucial when drawing inference from data through machine learning models. Scholars have increased focus on unraveling the bias and fairness in models, also in relation to inherent biases in the input data. However, limited work exists on the representativity of samples (datasets) for appropriate inference in AI systems. This paper reviews definitions and notions of a representative sample and surveys their use in scientific AI literature. We introduce three measurable concepts to help focus the notions and evaluate different data samples. Furthermore, we demonstrate that the contrast between a representative sample in the sense of coverage of the input space, versus a representative sample mimicking the distribution of the target population is of particular relevance when building AI systems. Through empirical demonstrations on US Census data, we evaluate the opposing inherent qualities of these concepts. Finally, we propose a framework of questions for creating and documenting data with data representativity in mind, as an addition to existing dataset documentation templates.