mlsanityData to 7 Oct 2026

Perturbation Augmentation for Fairer NLP

arXiv 2205.12586 ↗First posted 25 May 2022cs.CL cs.AIcited by 51

Not on the map. Why?

Abstract (arXiv)

Unwanted and often harmful social biases are becoming ever more salient in NLP research, affecting both models and datasets. In this work, we ask whether training on demographically perturbed data leads to fairer language models. We collect a large dataset of human annotated text perturbations and train a neural perturbation model, which we show outperforms heuristic alternatives. We find that (i) language models (LMs) pre-trained on demographically perturbed corpora are typically more fair, and (ii) LMs finetuned on perturbed GLUE datasets exhibit less demographic bias on downstream tasks, and (iii) fairness improvements do not come at the expense of performance on downstream tasks. Lastly, we discuss outstanding questions about how best to evaluate the (un)fairness of large language models. We hope that this exploration of neural demographic perturbation will help drive more improvement towards fairer NLP.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 89 of its 145 references that are in the map. First posted on arXiv 25 May 2022.

Builds on

89 of 145 references.

References · 89

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  7. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2020Shortcut Learning in Deep Neural Networkscited by 1,928
  10. 2015A large annotated corpus for learning natural language inferencecited by 1,837
  11. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,718
  12. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
38 more
  1. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  2. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  3. 2018Neural Network Acceptability Judgmentscited by 952
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  5. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  6. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  7. 2019Adversarial NLI: A New Benchmark for Natural Language Understandingcited by 798
  8. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  9. 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
  10. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 709
  11. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  12. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  13. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  14. 2018Annotation Artifacts in Natural Language Inference Datacited by 656
  15. 2017Adversarial Examples for Evaluating Reading Comprehension Systemscited by 568
  16. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  17. 2018Gender Bias in Coreference Resolutioncited by 438
  18. 2019Learning the Difference that Makes a Difference with Counterfactually-Augmented Datacited by 428
  19. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  20. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  21. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 346
  22. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 336
  23. 2019Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Themcited by 268
  24. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  25. 2020Measuring and Reducing Gendered Correlations in Pre-trained Modelscited by 222
  26. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  27. 2020HateCheck: Functional Tests for Hate Speech Detection Modelscited by 215
  28. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  29. 2019Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphologycited by 200
  30. 2019Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attackcited by 176
  31. 2020Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgettingcited by 169
  32. 2019Queens are Powerful too: Mitigating Gender Bias in Dialogue Generationcited by 152
  33. 2019Reducing Sentiment Bias in Language Models via Counterfactual Evaluationcited by 148
  34. 2020Intrinsic Bias Metrics Do Not Correlate with Application Biascited by 148
  35. 2017A Convex Framework for Fair Regressioncited by 147
  36. 2019Inherent Tradeoffs in Learning Fair Representationscited by 144
  37. 2017ParlAI: A Dialog Research Software Platformcited by 139
  38. 2020Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructurecited by 138

50 most cited of 89 All 89 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 19; 2024: 17; 2025: 6; 2026: 3.

51 papers.

Sort:

By area:

  1. 2023Bias and Fairness in Large Language Models: A Surveycited by 626
  2. 2023A Survey on Fairness in Large Language Modelscited by 91
  3. 2023Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedbackcited by 86
  4. 2024Safeguarding Large Language Models: A Surveycited by 73
  5. 2024Fairness in Large Language Models: A Taxonomic Surveycited by 57
  6. 2025Embodied AI Agents: Modeling the Worldcited by 57
  7. 2022Multi-VALUE: A Framework for Cross-Dialectal English NLPcited by 51
  8. 2023A Trip Towards Fairness: Bias and De-Biasing in Large Language Modelscited by 32
  9. 2022MABEL: Attenuating Gender Bias using Textual Entailment Datacited by 31
  10. 2025Actions Speak Louder than Words: Agent Decisions Reveal Implicit Biases in Language Modelscited by 30
  11. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  12. 2023Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Setscited by 27
38 more
  1. 2022Toxicity in Multilingual Machine Translation at Scalecited by 26
  2. 2023Stereotypes and Smut: The (Mis)representation of Non-cisgender Identities by Text-to-Image Modelscited by 26
  3. 2023An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Modelscited by 24
  4. 2023Multilingual Holistic Bias: Extending Descriptors and Patterns to Unveil Demographic Biases in Languages at Scalecited by 23
  5. 2024Deconstructing The Ethics of Large Language Models from Long-standing Issues to New-emerging Dilemmas: A Surveycited by 23
  6. 2022Debiasing Methods for Fairer Neural Models in Vision and Language Research: A Surveycited by 22
  7. 2022Evaluate & Evaluation on the Hub: Better Best Practices for Data and Model Measurementscited by 21
  8. 2024LLM-Driven Robots Risk Enacting Discrimination, Violence, and Unlawful Actionscited by 21
  9. 2023What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronounscited by 19
  10. 2026Brittlebench: Quantifying LLM robustness via prompt sensitivitycited by 19
  11. 2023MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logiccited by 18
  12. 2024CEB: Compositional Evaluation Benchmark for Fairness in Large Language Modelscited by 17
  13. 2023TADA: Task-Agnostic Dialect Adapters for Englishcited by 15
  14. 2023Exploiting Biased Models to De-bias Text: A Gender-Fair Rewriting Modelcited by 14
  15. 2023Nichelle and Nancy: The Influence of Demographic Attributes and Tokenization Length on First Name Biasescited by 12
  16. 2025Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metricscited by 12
  17. 2024Your Large Language Model is Secretly a Fairness Proponent and You Should Prompt it Like Onecited by 9
  18. 2023Examining the Causal Effect of First Names on Language Models: The Case of Social Commonsense Reasoningcited by 8
  19. 2023Dual Use Concerns of Generative AI and Large Language Modelscited by 6
  20. 2024The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Pre-trained Language Modelscited by 6
  21. 2022Human-Guided Fair Classification for Natural Language Processingcited by 5
  22. 2023On the Origins of Bias in NLP through the Lens of the Jim Codecited by 5
  23. 2024A survey of recent methods for addressing AI fairness and bias in biomedicinecited by 5
  24. 2025LangFair: A Python Package for Assessing Bias and Fairness in Large Language Model Use Casescited by 5
  25. 2024BTBR: A Bayesian-Theory-Driven Probabilistic-Fuzzy Framework for Implicit Bias Removal in Large Language Modelscited by 4
  26. 2023The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languagescited by 3
  27. 2024BEADs: Bias Evaluation Across Domainscited by 3
  28. 2024The Life Cycle of Large Language Models: A Review of Biases in Educationcited by 3
  29. 2024Mitigating Gender Bias in Code Large Language Models via Model Editingcited by 3
  30. 2025Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methodscited by 3
  31. 2025Investigating Intersectional Bias in Large Language Models using Confidence Disparities in Coreference Resolutioncited by 3
  32. 2023Learning from Red Teaming: Gender Bias Provocation and Mitigation in Large Language Modelscited by 2
  33. 2024FairFlow: An Automated Approach to Model-based Counterfactual Data Augmentation For NLPcited by 2
  34. 2024Expert-Guided Extinction of Toxic Tokens for Debiased Generationcited by 1
  35. 2024Collapsed Language Models Promote Fairnesscited by 1
  36. 2024Implicit Priors Editing in Stable Diffusion via Targeted Token Adjustmentcited by 1
  37. 2024Towards Massive Multilingual Holistic Biascited by 0
  38. 2026Context-Aware Counterfactual Data Augmentation for Gender Bias Mitigation in Language Modelscited by 0

50 most cited of 51 All 51 (JSON)