mlsanityData to 7 Oct 2026

A Taxonomy of Challenges to Curating Fair Datasets

arXiv 2406.06407 ↗First posted 10 Jun 2024cs.LG cs.CYcited by 2

Not on the map. Why?

Abstract (arXiv)

Despite extensive efforts to create fairer machine learning (ML) datasets, there remains a limited understanding of the practical aspects of dataset curation. Drawing from interviews with 30 ML dataset curators, we present a comprehensive taxonomy of the challenges and trade-offs encountered throughout the dataset curation lifecycle. Our findings underscore overarching issues within the broader fairness landscape that impact data curation. We conclude with recommendations aimed at fostering systemic changes to better facilitate fair dataset curation practices.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 92 of its 168 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

92 of 168 references.

References · 92

  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  3. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  5. 2018Datasheets for Datasetscited by 1,371
  6. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  7. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  8. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  9. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  10. 2021Retiring Adult: New Datasets for Fair Machine Learningcited by 443
  11. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  12. 2017Algorithmic decision making and the cost of fairnesscited by 402
38 more
  1. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
  2. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  3. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  4. 2018Improving fairness in machine learning systems: What do industry practitioners need?cited by 262
  5. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  6. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  7. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  8. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  9. 2023Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Modelscited by 204
  10. 2021The Values Encoded in Machine Learning Researchcited by 172
  11. 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 169
  12. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  13. 2020Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructurecited by 138
  14. 2020Fair Attribute Classification through Latent Space De-biasingcited by 125
  15. 2019Towards a Critical Race Methodology in Algorithmic Fairnesscited by 124
  16. 2021On Releasing Annotator-Level Labels and Information in Datasetscited by 123
  17. 2023Evaluating and Mitigating Discrimination in Language Model Decisionscited by 117
  18. 2019Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learningcited by 116
  19. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 115
  20. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  21. 2021Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Researchcited by 100
  22. 2022DigiFace-1M: 1 Million Digital Face Images for Face Recognitioncited by 92
  23. 2023NLPositionality: Characterizing Design Biases of Datasets and Modelscited by 91
  24. 2021Do Datasets Have Politics? Disciplinary Values in Computer Vision Dataset Developmentcited by 85
  25. 2021Towards Measuring Fairness in AI: the Casual Conversations Datasetcited by 80
  26. 2021What's in the Box? A Preliminary Analysis of Undesirable Content in the Common Crawl Corpuscited by 79
  27. 2019Predictive Inequity in Object Detectioncited by 77
  28. 2021Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreementcited by 71
  29. 2019Garbage In, Garbage Out? Do Machine Learning Application Papers in Social Computing Report Where Human-Labeled Training Data Comes From?cited by 64
  30. 2021Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Supportcited by 61
  31. 2022CrowdWorkSheets: Accounting for Individual and Collective Identities Underlying Crowdsourced Dataset Annotationcited by 59
  32. 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
  33. 2020"What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairnesscited by 56
  34. 2023Into the LAIONs Den: Investigating Hate in Multimodal Datasetscited by 54
  35. 2023Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Informationcited by 52
  36. 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
  37. 2022Algorithmic Fairness Datasets: the Story so Farcited by 51
  38. 2023Uncurated Image-Text Datasets: Shedding Light on Demographic Biascited by 51

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Building Machine Learning Challenges for Anomaly Detection in Sciencecited by 5
  2. 2025Bias Begins with Data: The FairGround Corpus for Robust and Reproducible Research on Algorithmic Fairnesscited by 2