mlsanityData to 7 Oct 2026

Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data

arXiv 2103.03399 ↗First posted 5 Mar 2021cs.LG stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

Collecting more diverse and representative training data is often touted as a remedy for the disparate performance of machine learning predictors across subpopulations. However, a precise framework for understanding how dataset properties like diversity affect learning outcomes is largely lacking. By casting data collection as part of the learning process, we demonstrate that diverse representation in training data is key not only to increasing subgroup performances, but also to achieving population level objectives. Our analysis and experiments describe how dataset compositions influence performance and provide constructive results for using trends in existing data, alongside domain knowledge, to help guide intentional, objective-aware dataset design.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 20 of its 53 references that are in the map. First posted on arXiv 5 Mar 2021.

Builds on

20 of 53 references.

References · 20

  1. 2018Datasheets for Datasetscited by 1,371
  2. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  3. 2019Data Shapley: Equitable Valuation of Data for Machine Learningcited by 617
  4. 2017Skin Lesion Analysis Toward Melanoma Detection: A Challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), Hosted by the International Skin Imaging Collaboration (ISIC)cited by 549
  5. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 510
  6. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 363
  7. 2019Measurement and Fairnesscited by 299
  8. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  9. 2018What is the Effect of Importance Weighting in Deep Learning?cited by 250
  10. 2018Multiaccuracy: Black-Box Post-Processing for Fairness in Classificationcited by 221
  11. 2020No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problemscited by 208
  12. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
8 more
  1. 2017No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing Worldcited by 152
  2. 2018Why Is My Classifier Discriminatory?cited by 143
  3. 2019Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learningcited by 116
  4. 2020Bringing the People Back In: Contesting Benchmark Machine Learning Datasetscited by 43
  5. 2019Value-laden Disciplinary Shifts in Machine Learningcited by 25
  6. 2021Representativeness in Statistics, Politics, and Machine Learningcited by 17
  7. 2018Are All Training Examples Created Equal? An Empirical Studycited by 16
  8. 2021Narratives and Counternarratives on Data Sharing in Africacited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 4; 2024: 9; 2025: 8; 2026: 4.

31 papers.

Sort:

By area:

  1. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Imagescited by 41
  2. 2021Algorithm Fairness in AI for Medicine and Healthcarecited by 21
  3. 2024Who's in and who's out? A case study of multimodal CLIP-filtering in DataCompcited by 15
  4. 2024The Data Addition Dilemmacited by 15
  5. 2022Conditional Generation of Medical Time Series for Extrapolation to Underrepresented Populationscited by 14
  6. 2024Scaling Laws for the Value of Individual Data Points in Machine Learningcited by 10
  7. 2023RoleCraft-GLM: Advancing Personalized Role-Playing in Large Language Modelscited by 9
  8. 2024Mind the Graph When Balancing Data for Fairness or Robustnesscited by 9
  9. 2024No Free Lunch: Retrieval-Augmented Generation Undermines Fairness in LLMs, Even for Vigilant Userscited by 9
  10. 2021Models of fairness in federated learningcited by 8
  11. 2023Delegating Data Collection in Decentralized Machine Learningcited by 8
  12. 2023Accuracy on the Curve: On the Nonlinear Correlation of ML Performance Between Data Subpopulationscited by 7
19 more
  1. 2024On Logical Extrapolation for Mazes with Recurrent and Implicit Networkscited by 7
  2. 2025How Does the Spatial Distribution of Pre-training Data Affect Geospatial Foundation Models?cited by 5
  3. 2025Relative Scaling Laws for LLMscited by 5
  4. 2022Fairness in Forecasting of Observations of Linear Dynamical Systemscited by 4
  5. 2022Striving for data-model efficiency: Identifying data externalities on group performancecited by 4
  6. 2024Safety vs. Performance: How Multi-Objective Learning Reduces Barriers to Market Entrycited by 4
  7. 2022Achieving Representative Data via Convex Hull Feasibility Sampling Algorithmscited by 2
  8. 2023Pushing the Accuracy-Group Robustness Frontier with Introspective Self-playcited by 2
  9. 2024Operationalizing the Blueprint for an AI Bill of Rights: Recommendations for Practitioners, Researchers, and Policy Makerscited by 2
  10. 2025Fairness of Deep Ensembles: On the interplay between per-group task difficulty and under-representationcited by 2
  11. 2025Mapping on a Budget: Optimizing Spatial Data Collection for MLcited by 2
  12. 2025Aligning NLP Models with Target Population Perspectives using PAIR: Population-Aligned Instance Replicationcited by 1
  13. 2025GRID: Scaling Task-Agnostic Inference in Continual Prompt Tuningcited by 1
  14. 2026Pretrain Where? Investigating How Pretraining Data Diversity Impacts Geospatial Foundation Model Performancecited by 1
  15. 2025Predict Training Data Quality via Its Geometry in Metric Spacecited by 0
  16. 2025Representation Invariance and Allocation: When Subgroup Balance Matterscited by 0
  17. 2026Measuring Dataset Diversity from a Geometric Perspectivecited by 0
  18. 2026Problems with Chinchilla Approach 2: Systematic Biases in IsoFLOP Parabola Fitscited by 0
  19. 2026Investigating Data Interventions for Subgroup Fairness: An ICU Case Studycited by 0