mlsanityData to 7 Oct 2026

Causal Proxy Models for Concept-Based Model Explanations

arXiv 2209.14279 ↗First posted 28 Sep 2022cs.CLcited by 29

Not on the map. Why?

Abstract (arXiv)

Explainability methods for NLP systems encounter a version of the fundamental problem of causal inference: for a given ground-truth input text, we never truly observe the counterfactual texts necessary for isolating the causal effects of model representations on outputs. In response, many explainability methods make no use of counterfactual texts, assuming they will be unavailable. In this paper, we show that robust causal explainability methods can be created using approximate counterfactuals, which can be written by humans to approximate a specific counterfactual or simply sampled using metadata-guided heuristics. The core of our proposal is the Causal Proxy Model (CPM). A CPM explains a black-box model N because it is trained to have the same actual input/output behavior as N while creating neural representations that can be intervened upon to simulate the counterfactual input/output behavior of N. Furthermore, we show that the best CPM for N performs comparably to N in making factual predictions, which means that the CPM can simply replace N, leading to more explainable deployed models. Our code is available at https://github.com/frankaging/Causal-Proxy-Model.

Key facts. Cited by 29 papers in the map (data through 7 Oct 2026). Builds on 41 of its 63 references that are in the map. First posted on arXiv 28 Sep 2022.

Builds on

41 of 63 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  8. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  10. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  11. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  12. 2016Concrete Problems in AI Safetycited by 1,582
29 more
  1. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  2. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  3. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  4. 2020Concept Bottleneck Modelscited by 1,022
  5. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 994
  6. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  7. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  8. 2016The Mythos of Model Interpretabilitycited by 716
  9. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 590
  10. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  11. 2017Meta-learners for Estimating Heterogeneous Treatment Effects using Machine Learningcited by 475
  12. 2018What you can cram into a single vector: Probing sentence embeddings for linguistic propertiescited by 452
  13. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 442
  14. 2021Causal Abstractions of Neural Networkscited by 401
  15. 2016European Union regulations on algorithmic decision-making and a "right to explanation"cited by 343
  16. 2019Counterfactual Visual Explanationscited by 312
  17. 2019On Completeness-aware Concept-Based Explanations in Deep Neural Networkscited by 239
  18. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 219
  19. 2016Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layerscited by 197
  20. 2019Explaining Classifiers with Causal Concept Effect (CaCE)cited by 135
  21. 2021Expanding Explainability: Towards Social Transparency in AI systemscited by 124
  22. 2020CausaLM: Causal Model Explanation Through Counterfactual Language Modelscited by 117
  23. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  24. 2022CEBaB: Estimating the Causal Effects of Real-World Concepts on NLP Model Behaviorcited by 54
  25. 2020Causal Effects of Linguistic Propertiescited by 39
  26. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  27. 2019Discovering the Compositional Structure of Vector Representations with Role Learning Networkscited by 33
  28. 2022Unit Testing for Concepts in Neural Networkscited by 23
  29. 2022Testing Pre-trained Language Models' Understanding of Distributivity via Causal Mediation Analysiscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 14; 2024: 11; 2025: 2; 2026: 1.

29 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  3. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  4. 2023Concept-based Explainable Artificial Intelligence: A Surveycited by 97
  5. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  6. 2023Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactualscited by 44
  7. 2023Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZerocited by 41
  8. 2023Interpreting Pretrained Language Models via Concept Bottleneckscited by 33
  9. 2024Mission: Impossible Language Modelscited by 31
  10. 2023A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepediacited by 24
  11. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  12. 2023Interpretable-by-Design Text Understanding with Iteratively Generated Concept Bottleneckcited by 21
17 more
  1. 2023ScoNe: Benchmarking Negation Reasoning in Language Models With Fine-Tuning and In-Context Learningcited by 17
  2. 2022Inducing Character-level Structure in Subword-based Language Models with Type-level Interchange Intervention Trainingcited by 16
  3. 2023Data Augmentations for Improved (Large) Language Model Generalizationcited by 14
  4. 2024Evaluating Readability and Faithfulness of Concept-based Explanationscited by 10
  5. 2023Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Modelscited by 7
  6. 2024Validating Mechanistic Interpretations: An Axiomatic Approachcited by 4
  7. 2023Estimating the Causal Effects of Natural Logic Features in Neural NLI Modelscited by 3
  8. 2024Updating CLIP to Prefer Descriptions Over Captionscited by 3
  9. 2024Automated Molecular Concept Generation and Labeling with Large Language Modelscited by 2
  10. 2023Estimation of Concept Explanations Should be Uncertainty Awarecited by 1
  11. 2024Decompose the model: Mechanistic interpretability in image models with Generalized Integrated Gradients (GIG)cited by 1
  12. 2024Beyond Attribution: Unified Concept-Level Explanationscited by 1
  13. 2025Enhancing the Comprehensibility of Text Explanations via Unsupervised Concept Discoverycited by 1
  14. 2026LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactualscited by 1
  15. 2023Measuring and Improving Attentiveness to Partial Inputs with Counterfactualscited by 0
  16. 2024Estimating the Causal Effects of Natural Logic Features in Transformer-Based NLI Modelscited by 0
  17. 2025Learning Counterfactual Outcomes Under Rank Preservationcited by 0