mlsanityData to 7 Oct 2026

Perturbation Sensitivity Analysis to Detect Unintended Model Biases

arXiv 1910.04210 ↗First posted 9 Oct 2019cs.CLcited by 74

Not on the map. Why?

Abstract (arXiv)

Data-driven statistical Natural Language Processing (NLP) techniques leverage large amounts of language data to build models that can understand language. However, most language data reflect the public discourse at the time the data was produced, and hence NLP models are susceptible to learning incidental associations around named referents at a particular point in time, in addition to general linguistic meaning. An NLP system designed to model notions such as sentiment and toxicity should ideally produce scores that are independent of the identity of such entities mentioned in text and their social associations. For example, in a general purpose sentiment analysis system, a phrase such as I hate Katy Perry should be interpreted as having the same sentiment as I hate Taylor Swift. Based on this idea, we propose a generic evaluation framework, Perturbation Sensitivity Analysis, which detects unintended model biases related to named entities, and requires no new annotations or corpora. We demonstrate the utility of this analysis by employing it on two different NLP models --- a sentiment model and a toxicity model --- applied on online comments in English language from four different genres.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 18; 2023: 19; 2024: 12; 2025: 3; 2026: 3.

74 papers.

Sort:

By area:

  1. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  2. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  3. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  4. 2021Re-imagining Algorithmic Fairness in India and Beyondcited by 102
  5. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  6. 2021Evaluating the Robustness of Neural Language Models to Input Perturbationscited by 87
  7. 2021Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metricscited by 83
  8. 2021On Measures of Biases and Harms in NLPcited by 81
  9. 2022Re-contextualizing Fairness in NLP: The Case of Indiacited by 55
  10. 2022Perturbation Augmentation for Fairer NLPcited by 51
  11. 2023Nationality Bias in Text Generationcited by 50
  12. 2022Evaluation Gaps in Machine Learning Practicecited by 48
38 more
  1. 2022On the Blind Spots of Model-Based Evaluation Metrics for Text Generationcited by 48
  2. 2022Quantifying Social Biases Using Templates is Unreliablecited by 38
  3. 2023Comparing Biases and the Impact of Multilingual Training across Multiple Languagescited by 36
  4. 2021Detecting Cross-Geographic Biases in Toxicity Modeling on Social Mediacited by 31
  5. 2021Evaluating Entity Disambiguation and the Role of Popularity in Retrieval-Based NLPcited by 30
  6. 2023Sociodemographic Bias in Language Models: A Survey and Forward Pathcited by 27
  7. 2021How Robust are Model Rankings: A Leaderboard Customization Approach for Equitable Evaluationcited by 24
  8. 2022Challenges in Applying Explainability Methods to Improve the Fairness of NLP Modelscited by 20
  9. 2022A Human Rights-Based Approach to Responsible AIcited by 20
  10. 2022Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrievalcited by 20
  11. 2023Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Modelscited by 20
  12. 2021Fairness-aware Class Imbalanced Learningcited by 19
  13. 2026Brittlebench: Quantifying LLM robustness via prompt sensitivitycited by 19
  14. 2021Interpreting the Robustness of Neural NLP Models to Textual Perturbationscited by 17
  15. 2023Unmasking Nationality Bias: A Study of Human Perception of Nationalities in AI-Generated Articlescited by 16
  16. 2024The "Colonial Impulse" of Natural Language Processing: An Audit of Bengali Sentiment Analysis Tools and Their Identity-based Biasescited by 16
  17. 2021Quantifying Gender Bias Towards Politicians in Cross-Lingual Language Modelscited by 15
  18. 2021Regression Bugs Are In Your Model! Measuring, Reducing and Analyzing Regressions In NLP Model Updatescited by 13
  19. 2022Flexible text generation for counterfactual fairness probingcited by 13
  20. 2023CALM : A Multi-task Benchmark for Comprehensive Assessment of Language Model Biascited by 13
  21. 2021Hate Speech Classifiers Learn Human-Like Social Stereotypescited by 12
  22. 2023Toward a Theory of Causation for Interpreting Neural Code Modelscited by 12
  23. 2023The Sentiment Problem: A Critical Survey towards Deconstructing Sentiment Analysiscited by 12
  24. 2025Lost in Moderation: How Commercial Content Moderation APIs Over- and Under-Moderate Group-Targeted Hate Speech and Linguistic Variationscited by 12
  25. 2023Surfacing Biases in Large Language Models using Contrastive Input Decodingcited by 11
  26. 2021Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluationcited by 9
  27. 2021From Toxicity in Online Comments to Incivility in American News: Proceed with Cautioncited by 8
  28. 2023Examining the Causal Effect of First Names on Language Models: The Case of Social Commonsense Reasoningcited by 8
  29. 2024Better Explain Transformers by Illuminating Important Informationcited by 8
  30. 2023Removing Non-Stationary Knowledge From Pre-Trained Language Models for Entity-Level Sentiment Classification in Financecited by 7
  31. 2023Evaluating Paraphrastic Robustness in Textual Entailment Modelscited by 7
  32. 2022Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment Informationcited by 6
  33. 2024Unveiling and Manipulating Prompt Influence in Large Language Modelscited by 6
  34. 2021Thinking Beyond Distributions in Testing Machine Learned Modelscited by 5
  35. 2023Counter-GAP: Counterfactual Bias Evaluation through Gendered Ambiguous Pronounscited by 5
  36. 2023On the Origins of Bias in NLP through the Lens of the Jim Codecited by 5
  37. 2022Fairness for Text Classification Tasks with Identity Information Data Augmentation Methodscited by 4
  38. 2022Cultural Re-contextualization of Fairness Research in Language Technologies in Indiacited by 4

50 most cited of 74 All 74 (JSON)