mlsanityData to 7 Oct 2026

An Empirical Study of Metrics to Measure Representational Harms in Pre-Trained Language Models

arXiv 2301.09211 ↗First posted 22 Jan 2023cs.CL cs.AIcited by 24

Not on the map. Why?

Abstract (arXiv)

Large-scale Pre-Trained Language Models (PTLMs) capture knowledge from massive human-written data which contains latent societal biases and toxic contents. In this paper, we leverage the primary task of PTLMs, i.e., language modeling, and propose a new metric to quantify manifested implicit representational harms in PTLMs towards 13 marginalized demographics. Using this metric, we conducted an empirical analysis of 24 widely used PTLMs. Our analysis provides insights into the correlation between the proposed metric in this work and other related metrics for representational harm. We observe that our metric correlates with most of the gender-specific metrics in the literature. Through extensive experiments, we explore the connections between PTLMs architectures and representational harms across two dimensions: depth and width of the networks. We found that prioritizing depth over width, mitigates representational harms in some PTLMs. Our code and data can be found at https://github.com/microsoft/SafeNLP.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 22 Jan 2023.

Builds on

28 of 48 references.

References · 28

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  9. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 911
  10. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  11. 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 703
  12. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
16 more
  1. 2019The Woman Worked as a Babysitter: On Biases in Language Generationcited by 540
  2. 2018Gender Bias in Coreference Resolutioncited by 438
  3. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  4. 2021BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generationcited by 410
  5. 2019On Measuring Social Biases in Sentence Encoderscited by 397
  6. 2019Measuring Bias in Contextualized Word Representationscited by 275
  7. 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
  8. 2021Latent Hatred: A Benchmark for Understanding Implicit Hate Speechcited by 216
  9. 2019Gender Bias in Contextualized Word Embeddingscited by 203
  10. 2021Societal Biases in Language Generation: Progress and Challengescited by 201
  11. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  12. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  13. 2022On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representationscited by 75
  14. 2020Investigating African-American Vernacular English in Transformer-Based Text Generationcited by 52
  15. 2022Perturbation Augmentation for Fairer NLPcited by 51
  16. 2020Controlling Style in Generated Dialoguecited by 38

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 6; 2024: 12; 2025: 4; 2026: 2.

24 papers.

Sort:

By area:

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  3. 2023Large Language Model Alignment: A Surveycited by 241
  4. 2023Evaluating Large Language Models: A Comprehensive Surveycited by 200
  5. 2024ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teamingcited by 96
  6. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  7. 2024LlavaGuard: An Open VLM-based Framework for Safeguarding Vision Datasets and Modelscited by 49
  8. 2024BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Modelscited by 44
  9. 2024Airavata: Introducing Hindi Instruction-tuned LLMcited by 32
  10. 2023Trojan Activation Attack: Red-Teaming Large Language Models using Activation Steering for Safety-Alignmentcited by 27
  11. 2024The Better Angels of Machine Personality: How Personality Relates to LLM Safetycited by 21
  12. 2024A Causal Explainable Guardrails for Large Language Modelscited by 18
12 more
  1. 2024Beyond Behaviorist Representational Harms: A Plan for Measurement and Mitigationcited by 17
  2. 2025K2-Think: A Parameter-Efficient Reasoning Systemcited by 11
  3. 2024Developing Safe and Responsible Large Language Model : Can We Balance Bias Reduction and Language Understanding in Large Language Models?cited by 8
  4. 2024LLMs Lost in Translation: M-ALERT uncovers Cross-Linguistic Safety Inconsistenciescited by 6
  5. 2024Harm Mitigation in Recommender Systems under User Preference Dynamicscited by 5
  6. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 4
  7. 2024BeanCounter: A low-toxicity, large-scale, and open dataset of business-oriented textcited by 3
  8. 2023MetaAID 2.5: A Secure Framework for Developing Metaverse Applications via Large Language Modelscited by 2
  9. 2025Don't Change My View: Ideological Bias Auditing in Large Language Modelscited by 1
  10. 2025A Multifaceted Analysis of Social Biases in Large Language Modelscited by 0
  11. 2026Compounding Disadvantage: Auditing Intersectional Bias in LLM-Generated Explanations Across Indian and American STEM Educationcited by 0
  12. 2026Navigating the Sea of LLM Evaluation: Investigating Bias in Toxicity Benchmarkscited by 0