mlsanityData to 7 Oct 2026

Directional Bias Amplification

arXiv 2102.12594 ↗First posted 24 Feb 2021cs.LG cs.AIcited by 54

Not on the map. Why?

Abstract (arXiv)

Mitigating bias in machine learning systems requires refining our understanding of bias propagation pathways: from societal structures to large-scale data to trained models to impact on society. In this work, we focus on one aspect of the problem, namely bias amplification: the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. (2017); however, as we demonstrate, this metric suffers from a number of shortcomings including conflating different types of bias amplification and failing to account for varying base rates of protected attributes. We introduce and analyze a new, decoupled metric for measuring bias amplification, BiasAmprightarrow (Directional Bias Amplification). We thoroughly analyze and discuss both the technical assumptions and normative implications of this metric. We provide suggestions about its measurement by cautioning against predicting sensitive attributes, encouraging the use of confidence intervals due to fluctuations in the fairness of models across runs, and discussing the limitations of what this metric captures. Throughout this paper, we work to provide an interrogative look at the technical measurement of bias amplification, guided by our normative ideas of what we want it to encompass. Code is located at https://github.com/princetonvisualai/directional-bias-amp

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 37 of its 67 references that are in the map. First posted on arXiv 24 Feb 2021.

Builds on

37 of 67 references.

References · 37

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  8. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  9. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  10. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 951
  11. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  12. 2017Counterfactual Fairnesscited by 802
25 more
  1. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  2. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  3. 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 488
  4. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  5. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  6. 2018All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneouslycited by 364
  7. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  8. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  9. 2018Gender Bias in Neural Natural Language Processingcited by 210
  10. 2018A comparative study of fairness-enhancing interventions in machine learningcited by 208
  11. 2020Towards Debiasing Sentence Representationscited by 170
  12. 2019Predictive Multiplicity in Classificationcited by 162
  13. 2018An Intersectional Definition of Fairnesscited by 126
  14. 2019Fair Generative Modeling via Weak Supervisioncited by 114
  15. 2019Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editingcited by 108
  16. 2020Don't Judge an Object by Its Context: Learning to Overcome Contextual Biascited by 91
  17. 2020On Counterfactual Explanations under Predictive Multiplicitycited by 73
  18. 2019Reducing Gender Bias in Word-Level Language Models with a Gender-Equalizing Loss Functioncited by 57
  19. 2020Mitigating Gender Bias in Captioning Systemscited by 50
  20. 2018Feature-Wise Bias Amplificationcited by 31
  21. 2020Imperfect ImaGANation: Implications of GANs Exacerbating Biases on Facial Data Augmentation and Snapchat Selfie Lensescited by 27
  22. 2020Mitigating Gender Bias Amplification in Distribution by Posterior Regularizationcited by 27
  23. 2019Variable Importance Clouds: A Way to Explore Variable Importance for the Set of Good Modelscited by 19
  24. 2017Interventions over Predictions: Reframing the Ethical Debate for Actuarial Risk Assessmentcited by 16
  25. 2020Towards Threshold Invariant Fair Classificationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 7; 2023: 13; 2024: 16; 2025: 11; 2026: 3.

54 papers.

Sort:

By area:

  1. 2021Ethical and social risks of harm from Language Modelscited by 957
  2. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  3. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 69
  4. 2022A Systematic Study of Bias Amplificationcited by 65
  5. 2024On the Societal Impact of Open Foundation Modelscited by 64
  6. 2023FACET: Fairness in Computer Vision Evaluation Benchmarkcited by 58
  7. 2023Uncurated Image-Text Datasets: Shedding Light on Demographic Biascited by 51
  8. 2022Quantifying Societal Bias Amplification in Image Captioningcited by 37
  9. 2023Overwriting Pretrained Bias with Finetuning Datacited by 32
  10. 2022Addressing Fairness Issues in Deep Learning-Based Medical Image Analysis: A Systematic Reviewcited by 27
  11. 2022Men Also Do Laundry: Multi-Attribute Bias Amplificationcited by 21
  12. 2024ChatGPT as Research Scientist: Probing GPT's Capabilities as a Research Librarian, Research Ethicist, Data Generator and Data Predictorcited by 21
38 more
  1. 2025Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMscited by 20
  2. 2021Correcting Exposure Bias for Link Recommendationcited by 17
  3. 2021Modeling Techniques for Machine Learning Fairness: A Surveycited by 17
  4. 2023Model-Agnostic Gender Debiased Image Captioningcited by 17
  5. 2025Systemizing Multiplicity: The Curious Case of Arbitrariness in Machine Learningcited by 17
  6. 2024Harm Amplification in Text-to-Image Modelscited by 14
  7. 2024Images Speak Louder than Words: Understanding and Mitigating Bias in Vision-Language Model from a Causal Mediation Perspectivecited by 13
  8. 2024OxonFair: A Flexible Toolkit for Algorithmic Fairnesscited by 12
  9. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 12
  10. 2022Simplicity Bias Leads to Amplified Performance Disparitiescited by 11
  11. 2022Are Two Heads the Same as One? Identifying Disparate Treatment in Fair Neural Networkscited by 10
  12. 2024Achieving Fairness Through Channel Pruning for Dermatological Disease Diagnosiscited by 9
  13. 2024From Descriptive Richness to Bias: Unveiling the Dark Side of Generative Image Caption Enrichmentcited by 8
  14. 2024Efficient Bias Mitigation Without Privileged Informationcited by 8
  15. 2023Travel Demand Forecasting: A Fair AI Approachcited by 7
  16. 2023Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Toolscited by 7
  17. 2024Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributescited by 7
  18. 2022CAT: Controllable Attribute Translation for Fair Facial Attribute Classificationcited by 6
  19. 2023Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Modelscited by 6
  20. 2023Towards Reliable Assessments of Demographic Disparities in Multi-Label Image Classifierscited by 5
  21. 2023ImageCaptioner2: Image Captioner for Image Captioning Bias Amplification Assessmentcited by 5
  22. 2024The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Modelscited by 5
  23. 2025Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluationcited by 5
  24. 2024Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Wayscited by 4
  25. 2024Addressing Discretization-Induced Bias in Demographic Predictioncited by 4
  26. 2023The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languagescited by 3
  27. 2024Evaluating Model Bias Requires Characterizing its Mistakescited by 3
  28. 2025On the Impact of Performative Risk Minimization for Binary Random Variablescited by 3
  29. 2023Variation of Gender Biases in Visual Recognition Models Before and After Finetuningcited by 2
  30. 2025From Global to Local: Social Bias Transfer in CLIPcited by 2
  31. 2023GELDA: A generative language annotation framework to reveal visual biases in datasetscited by 1
  32. 2024Debias-CLR: A Contrastive Learning Based Debiasing Method for Algorithmic Fairness in Healthcare Applicationscited by 1
  33. 2024DPA: A one-stop metric to measure bias amplification in classification datasetscited by 1
  34. 2025A Woman with a Knife or A Knife with a Woman? Measuring Directional Bias Amplification in Image Captionscited by 1
  35. 2025When majority rules, minority loses: bias amplification of gradient descentcited by 1
  36. 2025A Critical Review of Predominant Bias in Neural Networkscited by 0
  37. 2025A Feature-level Bias Evaluation Framework for Facial Expression Recognition Modelscited by 0
  38. 2025Simulating Biases for Interpretable Fairness in Offline and Online Classifierscited by 0

50 most cited of 54 All 54 (JSON)