mlsanityData to 7 Oct 2026

Causal Mediation Analysis for Interpreting Neural NLP: The Case of Gender Bias

arXiv 2004.12265 ↗First posted 26 Apr 2020cs.CLcited by 183

Not on the map. Why?

Abstract (arXiv)

Common methods for interpreting neural models in natural language processing typically examine either their structure or their behavior, but not both. We propose a methodology grounded in the theory of causal mediation analysis for interpreting which parts of a model are causally implicated in its behavior. It enables us to analyze the mechanisms by which information flows from input to output through various model components, known as mediators. We apply this methodology to analyze gender bias in pre-trained Transformer language models. We study the role of individual neurons and attention heads in mediating gender bias across three datasets designed to gauge a model's sensitivity to gender bias. Our mediation analysis reveals that gender bias effects are (i) sparse, concentrated in a small part of the network; (ii) synergistic, amplified or repressed by different components; and (iii) decomposable into effects flowing directly from the input and indirectly through the mediators.

Key facts. Cited by 183 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 21; 2022: 18; 2023: 20; 2024: 44; 2025: 34; 2026: 46.

183 papers.

Sort:

By area:

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  3. 2021Editing Factual Knowledge in Language Modelscited by 548
  4. 2021Causal Abstractions of Neural Networkscited by 401
  5. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  6. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  7. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  8. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  9. 2024How to use and interpret activation patchingcited by 191
  10. 2021Towards Continual Knowledge Learning of Language Modelscited by 172
  11. 2023The Hydra Effect: Emergent Self-repair in Language Model Computationscited by 145
  12. 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 127
38 more
  1. 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 121
  2. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  3. 2021An Interpretability Illusion for BERTcited by 94
  4. 2021A Survey on Gender Bias in Natural Language Processingcited by 89
  5. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 83
  6. 2021Contrastive Explanations for Model Interpretabilitycited by 75
  7. 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 75
  8. 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Modelscited by 68
  9. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 64
  10. 2022Theories of "Gender" in NLP Bias Researchcited by 62
  11. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 60
  12. 2021Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Predictioncited by 58
  13. 2022Probing for the Usage of Grammatical Numbercited by 57
  14. 2022CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behaviorcited by 54
  15. 2021Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Modelscited by 52
  16. 2024Understanding Information Storage and Transfer in Multi-modal Large Language Modelscited by 48
  17. 2024From Understanding to Utilization: A Survey on Explainability for Large Language Modelscited by 47
  18. 2024Interpreting Attention Layer Outputs with Sparse Autoencoderscited by 43
  19. 2021Human Trajectory Prediction via Counterfactual Analysiscited by 42
  20. 2021What if This Modified That? Syntactic Interventions via Counterfactual Embeddingscited by 39
  21. 2023COCO-Counterfactuals: Automatically Constructed Counterfactual Examples for Image-Text Pairscited by 39
  22. 2024Explorations of Self-Repair in Language Modelscited by 35
  23. 2024Activation Scaling for Steering and Interpreting Language Modelscited by 33
  24. 2022FaiRR: Faithful and Robust Deductive Reasoning over Natural Languagecited by 29
  25. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 29
  26. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modelingcited by 27
  27. 2024Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?cited by 26
  28. 2023Debiasing Algorithm through Model Adaptationcited by 24
  29. 2023Interpretability Illusions in the Generalization of Simplified Modelscited by 24
  30. 2022NeuroCounterfactuals: Beyond Minimal-Edit Counterfactuals for Richer Data Augmentationcited by 23
  31. 2024Efficient Automated Circuit Discovery in Transformers using Contextual Decompositioncited by 23
  32. 2023Grokking Group Multiplication with Cosetscited by 22
  33. 2023A Causal View of Entity Bias in (Large) Language Modelscited by 21
  34. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  35. 2024Beneath the Surface of Consistency: Exploring Cross-lingual Knowledge Representation Sharing in LLMscited by 20
  36. 2025Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysiscited by 20
  37. 2021Causal Distillation for Language Modelscited by 19
  38. 2022Using Natural Sentences for Understanding Biases in Language Modelscited by 19

50 most cited of 183 All 183 (JSON)