mlsanityData to 14 Oct 2026

A Decade's Battle on Dataset Bias: Are We There Yet?

arXiv 2403.08632 ↗First posted 13 Mar 2024cs.CV cs.LGcited by 47

Not on the map. Why?

Abstract (arXiv)

We revisit the "dataset classification" experiment suggested by Torralba & Efros (2011) a decade ago, in the new era with large-scale, diverse, and hopefully less biased datasets as well as more capable neural network architectures. Surprisingly, we observe that modern neural networks can achieve excellent accuracy in classifying which dataset an image is from: e.g., we report 84.7% accuracy on held-out validation data for the three-way classification problem consisting of the YFCC, CC, and DataComp datasets. Our further experiments show that such a dataset classifier could learn semantic features that are generalizable and transferable, which cannot be explained by memorization. We hope our discovery will inspire the community to rethink issues involving dataset bias.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). Builds on 49 of its 74 references that are in the map. First posted on arXiv 13 Mar 2024.

Builds on

49 of 74 references.

References · 49

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  9. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  10. 2016Layer Normalizationcited by 5,669
  11. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
37 more
  1. 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,444
  2. 2014Going Deeper with Convolutionscited by 4,241
  3. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  4. 2022A ConvNet for the 2020scited by 3,579
  5. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  6. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  7. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  8. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  9. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,162
  10. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  12. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  13. 2019Natural Adversarial Examplescited by 1,408
  14. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  15. 2014How transferable are features in deep neural networks?cited by 1,225
  16. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,216
  17. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 919
  18. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  19. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  20. 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 600
  21. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 424
  22. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  23. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  24. 2021WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learningcited by 287
  25. 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
  26. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  27. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
  28. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  29. 2017No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing Worldcited by 153
  30. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  31. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  32. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  33. 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 102
  34. 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
  35. 2015A Deeper Look at Dataset Biascited by 87
  36. 2016Stereotyping and Bias in the Flickr30K Datasetcited by 25
  37. 2020Identifying Statistical Bias in Dataset Replicationcited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 18; 2026: 13.

47 papers.

Sort:

By area:

  1. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  2. 2024Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directionscited by 77
  3. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  4. 2025Scaling Language-Free Visual Representation Learningcited by 66
  5. 2024LLM as Dataset Analyst: Subpopulation Structure Discovery with Large Language Modelcited by 44
  6. 2025Idiosyncrasies in Large Language Modelscited by 33
  7. 2024A Practitioner's Guide to Continual Multimodal Pretrainingcited by 23
  8. 2025Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentationcited by 23
  9. 2025Drag-and-Drop LLMs: Zero-Shot Prompt-to-Weightscited by 20
  10. 2025Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Modelscited by 16
  11. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 15
  12. 2024Understanding Bias in Large-Scale Visual Datasetscited by 15
35 more
  1. 2024ImageNot: A contrast with ImageNet preserves model rankingscited by 14
  2. 2025Always Skip Attentioncited by 13
  3. 2025In Pursuit of Pixel Supervision for Visual Pre-trainingcited by 13
  4. 2024Accessing Vision Foundation Models via ImageNet-1Kcited by 10
  5. 2024BA-LoRA: Bias-Alleviating Low-Rank Adaptation to Mitigate Catastrophic Inheritance in Large Language Modelscited by 7
  6. 2025Natural Fingerprints of Large Language Modelscited by 6
  7. 2026Shortcut Learning in Binary Classifier Black Boxes: Applications to Voice Anti-Spoofing and Biometricscited by 6
  8. 2024Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers?cited by 5
  9. 2024Measuring Fingerprints of Web-filtered Text Datasets and Fingerprint Propagation Through Trainingcited by 5
  10. 2025DATA: Multi-Disentanglement based Contrastive Learning for Open-World Semi-Supervised Deepfake Attributioncited by 5
  11. 2025ConceptScope: Characterizing Dataset Bias via Disentangled Visual Conceptscited by 5
  12. 2025Modeling Saliency Dataset Biascited by 4
  13. 2025Processing and acquisition traces in visual encoders: What does CLIP know about your camera?cited by 3
  14. 2024NaVIP: An Image-Centric Indoor Navigation Solution for Visually Impaired Peoplecited by 2
  15. 2025BiMa: Towards Biases Mitigation for Text-Video Retrieval via Scene Element Guidancecited by 2
  16. 2025Limitations of Public Chest Radiography Datasets for Artificial Intelligence: Label Quality, Domain Shift, Bias and Evaluation Challengescited by 2
  17. 2025Representation-Level Counterfactual Calibration for Debiased Zero-Shot Recognitioncited by 2
  18. 2024Real-World Benchmarks Make Membership Inference Attacks Fail on Diffusion Modelscited by 1
  19. 2024Low-Biased General Annotated Dataset Generationcited by 1
  20. 2025A Unified and Scalable Membership Inference Method for Visual Self-supervised Encoder via Part-aware Capabilitycited by 1
  21. 2025Understanding Dataset Bias in Medical Imaging: A Case Study on Chest X-rayscited by 1
  22. 2026Self-Aware Object Detection via Degradation Manifoldscited by 1
  23. 2026Asymmetric Idiosyncrasies in Multimodal Modelscited by 1
  24. 2026Training-free Task Classification for Multi-Task Model Mergingcited by 1
  25. 2026Dataset-Origin Signatures and Shortcut Learning in Screening Mammography AI: A Cross-Dataset Case Studycited by 1
  26. 2026Visual General Intelligence: A White Papercited by 1
  27. 2024Renormalized Connection for Scale-preferred Object Detection in Satellite Imagerycited by 0
  28. 2025SNAP: A Benchmark for Testing the Effects of Capture Conditions on Fundamental Vision Taskscited by 0
  29. 2026Coverage-aware Semantic Representation Learning for Underrepresented Visual Conceptscited by 0
  30. 2026Generalizable Knowledge Distillation from Vision Foundation Models for Semantic Segmentationcited by 0
  31. 2026What Are We Really Measuring? Rethinking Dataset Bias in Web-Scale Natural Image Collections via Unsupervised Semantic Clusteringcited by 0
  32. 2026DeconDTN-Toolkit: A Library for Evaluation and Enhancement of Robustness to Provenance Shiftcited by 0
  33. 2026Invisible Shortcuts: Why Vision Encoders Know Your Cameracited by 0
  34. 2026A New Transformer-Based Approach for Audio-Based Kinship Verification and a New Uncontrolled Mandarin Kinship Speech Datasetcited by 0
  35. 2026Dataset Signatures in Human-LLM Interactions and User Modelingcited by 0