# What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clustering

arXiv: https://arxiv.org/abs/2604.13610 · HTML: https://mlsanity.com/paper/2604.13610/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 44 references that are in the map.
- First posted on arXiv 15 Apr 2026.

## Builds on (most cited first, 28 of 28)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts](https://mlsanity.com/paper/2102.08981/) (2021)
- [DataComp: In search of the next generation of multimodal datasets](https://mlsanity.com/paper/2304.14108/) (2023)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition](https://mlsanity.com/paper/1310.1531/) (2013)
- [WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning](https://mlsanity.com/paper/2103.01913/) (2021)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Adversarial Filters of Dataset Biases](https://mlsanity.com/paper/2002.04108/) (2020)
- [REPAIR: Removing Representation Bias by Dataset Resampling](https://mlsanity.com/paper/1904.07911/) (2019)
- [Learning from Failure: Training Debiased Classifier from Biased Classifier](https://mlsanity.com/paper/2007.02561/) (2020)
- [xGen-MM (BLIP-3): A Family of Open Large Multimodal Models](https://mlsanity.com/paper/2408.08872/) (2024)
- [A Deeper Look at Dataset Bias](https://mlsanity.com/paper/1505.01257/) (2015)
- [A Decade's Battle on Dataset Bias: Are We There Yet?](https://mlsanity.com/paper/2403.08632/) (2024)
- [Detect and Correct Bias in Multi-Site Neuroimaging Datasets](https://mlsanity.com/paper/2002.05049/) (2020)
- [Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias](https://mlsanity.com/paper/1808.02212/) (2018)
- [Deflating Dataset Bias Using Synthetic Data Augmentation](https://mlsanity.com/paper/2004.13866/) (2020)
- [Mitigating Dataset Bias by Using Per-sample Gradient](https://mlsanity.com/paper/2205.15704/) (2022)
- [A First Look at Dataset Bias in License Plate Recognition](https://mlsanity.com/paper/2208.10657/) (2022)
- [Cross-Dataset Adaptation for Visual Question Answering](https://mlsanity.com/paper/1806.03726/) (2018)
- [The Impact of Scanner Domain Shift on Deep Learning Performance in Medical Imaging: an Experimental Study](https://mlsanity.com/paper/2409.04368/) (2024)
- [Understanding Bias in Large-Scale Visual Datasets](https://mlsanity.com/paper/2412.01876/) (2024)
- [Quantifying Confounding Bias in Neuroimaging Datasets with Causal Inference](https://mlsanity.com/paper/1907.04102/) (2019)
- [Processing and acquisition traces in visual encoders: What does CLIP know about your camera?](https://mlsanity.com/paper/2508.10637/) (2025)

## Abstract (arXiv)

In computer vision, a prevailing method for quantifying dataset bias is to train a model to distinguish between datasets. High classification accuracy is then interpreted as evidence of meaningful semantic differences. This approach assumes that standard image augmentations successfully suppress low-level, non-semantic cues, and that any remaining performance must therefore reflect true semantic divergence. We demonstrate that this fundamental assumption is flawed within the domain of large-scale natural image collections. High classification accuracy is often driven by resolution-based artifacts, which are structural fingerprints arising from native image resolution distributions and interpolation effects during resizing. These artifacts form robust, dataset-specific signatures that persist despite conventional image corruptions. Through controlled experiments, we show that models achieve strong dataset classification even on non-semantic, procedurally generated images, proving their reliance on superficial cues. To address this issue, we revisit this decades-old idea of dataset separability, but not with supervised classification. Instead, we introduce an unsupervised approach that measures true semantic separability. Our framework directly assesses semantic similarity by clustering semantically-rich features from foundational vision models, deliberately bypassing supervised classification on dataset labels. When applied to major web-scale datasets, the primary focus of this work, the high separability reported by supervised methods largely vanishes, with clustering accuracy dropping to near-chance levels. This reveals that conventional classification-based evaluation systematically overstates semantic bias by an overwhelming margin.