mlsanityData to 7 Oct 2026

Position: Cracking the Code of Cascading Disparity Towards Marginalized Communities

arXiv 2406.01757 ↗First posted 3 Jun 2024cs.LG cs.AI cs.CYcited by 4

Not on the map. Why?

Abstract (arXiv)

The rise of foundation models holds immense promise for advancing AI, but this progress may amplify existing risks and inequalities, leaving marginalized communities behind. In this position paper, we discuss that disparities towards marginalized communities - performance, representation, privacy, robustness, interpretability and safety - are not isolated concerns but rather interconnected elements of a cascading disparity phenomenon. We contrast foundation models with traditional models and highlight the potential for exacerbated disparity against marginalized communities. Moreover, we emphasize the unique threat of cascading impacts in foundation models, where interconnected disparities can trigger long-lasting negative consequences, specifically to the people on the margin. We define marginalized communities within the machine learning context and explore the multifaceted nature of disparities. We analyze the sources of these disparities, tracing them from data creation, training and deployment procedures to highlight the complex technical and socio-technical landscape. To mitigate the pressing crisis, we conclude with a set of calls to action to mitigate disparity at its source.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 68 of its 102 references that are in the map. First posted on arXiv 3 Jun 2024.

Builds on

68 of 102 references.

References · 68

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  6. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  7. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  8. 2016Deep Learning with Differential Privacycited by 2,682
  9. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  10. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  11. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  12. 2019Invariant Risk Minimizationcited by 1,832
38 more
  1. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  3. 2019A Comprehensive Survey on Transfer Learningcited by 1,154
  4. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  5. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  6. 2020What Makes for Good Views for Contrastive Learning?cited by 843
  7. 2022Quantifying Memorization Across Neural Language Modelscited by 755
  8. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  9. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  10. 2023RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedbackcited by 615
  11. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  12. 2019Defending Against Neural Fake Newscited by 553
  13. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  14. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  15. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 377
  16. 2023Toxicity in ChatGPT: Analyzing Persona-assigned Language Modelscited by 356
  17. 2023Explainability for Large Language Models: A Surveycited by 331
  18. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  19. 2021Towards Understanding and Mitigating Social Biases in Language Modelscited by 294
  20. 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
  21. 2018Audio Adversarial Examples: Targeted Attacks on Speech-to-Textcited by 268
  22. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  23. 2023Evaluating the Moral Beliefs Encoded in LLMscited by 211
  24. 2017Data Decisions and Theoretical Implications when Adversarially Learning Fair Representationscited by 165
  25. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  26. 2020Characterising Bias in Compressed Modelscited by 162
  27. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  28. 2022The Privacy Onion Effect: Memorization is Relativecited by 128
  29. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  30. 2019Fairness in Deep Learning: A Computational Perspectivecited by 123
  31. 2023Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarizationcited by 122
  32. 2018Assessing Gender Bias in Machine Translation -- A Case Study with Google Translatecited by 121
  33. 2018Distribution Matching Losses Can Hallucinate Features in Medical Image Translationcited by 99
  34. 2020Characterizing Structural Regularities of Labeled Data in Overparameterized Modelscited by 93
  35. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 88
  36. 2021Do Transformer Modifications Transfer Across Implementations and Applications?cited by 87
  37. 2019Problem Formulation and Fairnesscited by 85
  38. 2023Ethical Reasoning over Moral Alignment: A Case and Framework for In-Context Ethical Policies in LLMscited by 64

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025HESEIA: A community-based dataset for evaluating social biases in large language models, co-designed in real school settings in Latin Americacited by 6
  2. 2025Position: Neglecting the Sustainability of AI is Fuelling a Global AI Arms Racecited by 2
  3. 2026Position: the Stochastic Parrot in the Coal Mine. Model Collapse is a Threat to Low-Resource Communitiescited by 1
  4. 2026Narrowing the Horizon: Quantifying Topic Saliency Shifts in Generative Monoculturecited by 0