mlsanityData to 14 Oct 2026

Counterfactual Explanations for Models of Code

arXiv 2111.05711 ↗First posted 10 Nov 2021cs.SE cs.AI cs.LGcited by 25

Not on the map. Why?

Abstract (arXiv)

Machine learning (ML) models play an increasingly prevalent role in many software engineering tasks. However, because most models are now powered by opaque deep neural networks, it can be difficult for developers to understand why the model came to a certain conclusion and how to act upon the model's prediction. Motivated by this problem, this paper explores counterfactual explanations for models of source code. Such counterfactual explanations constitute minimal changes to the source code under which the model "changes its mind". We integrate counterfactual explanation generation to models of source code in a real-world setting. We describe considerations that impact both the ability to find realistic and plausible counterfactual explanations, as well as the usefulness of such explanation to the user of the model. In a series of experiments we investigate the efficacy of our approach on three different models, each based on a BERT-like architecture operating over source code.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 35 of its 49 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

35 of 49 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,070
  4. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  5. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  6. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  7. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,206
  8. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
  9. 2019Attention is not Explanationcited by 929
  10. 2018code2vec: Learning Distributed Representations of Codecited by 250
  11. 2020Algorithmic Recourse: from Counterfactual Explanations to Interventionscited by 250
  12. 2019FACE: Feasible and Actionable Counterfactual Explanationscited by 233
23 more
  1. 2016Understanding Neural Networks through Representation Erasurecited by 224
  2. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 222
  3. 2015Visualizing and Understanding Neural Models in NLPcited by 200
  4. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  5. 2018Pathologies of Neural Models Make Interpretations Difficultcited by 140
  6. 2020A Transformer-based Approach for Source Code Summarizationcited by 133
  7. 2018Automated Vulnerability Detection in Source Code Using Deep Representation Learningcited by 114
  8. 2019Learning to Deceive with Attention-Based Explanationscited by 111
  9. 2020Explaining NLP Models via Minimal Contrastive Editing (MiCE)cited by 106
  10. 2019Attention in Natural Language Processingcited by 101
  11. 2020General Pitfalls of Model-Agnostic Interpretation Methods for Machine Learning Modelscited by 75
  12. 2020Generate Your Counterfactuals: Towards Controlled Counterfactual Generation for Textcited by 75
  13. 2019Attention Interpretability Across NLP Taskscited by 71
  14. 2018DeepBugs: A Learning Approach to Name-based Bug Detectioncited by 65
  15. 2019Adversarial Examples for Models of Codecited by 63
  16. 2017Interpretable & Explorable Approximations of Black Box Modelscited by 54
  17. 2020Semantic Robustness of Models of Source Codecited by 45
  18. 2019TypeWriter: Neural Type Prediction with Search-based Validationcited by 32
  19. 2021Understanding Neural Code Intelligence Through Program Simplificationcited by 22
  20. 2020Learning Autocompletion from Real-World Datasetscited by 20
  21. 2020Probing Model Signal-Awareness via Prediction-Preserving Input Minimizationcited by 14
  22. 2021Generative Counterfactuals for Neural Networks via Attribute-Informed Perturbationcited by 11
  23. 2017Navigate, Understand, Communicate: How Developers Locate Performance Bugscited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 10; 2024: 7; 2025: 6.

25 papers.

Sort:

By area:

  1. 2024NExT: Teaching Large Language Models to Reason about Code Executioncited by 63
  2. 2023Stealthy Backdoor Attack for Code Modelscited by 49
  3. 2024Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicatescited by 14
  4. 2023Toward a Theory of Causation for Interpreting Neural Code Modelscited by 12
  5. 2024Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanationcited by 12
  6. 2024Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanationscited by 11
  7. 2023Towards Causal Deep Learning for Vulnerability Detectioncited by 9
  8. 2024PromptExp: Multi-granularity Prompt Explanation of Large Language Modelscited by 6
  9. 2025Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?cited by 5
  10. 2023ExplainableFold: Understanding AlphaFold Prediction with Explainable AIcited by 4
  11. 2024A Critical Study of What Code-LLMs (Do Not) Learncited by 4
  12. 2023Study of Distractors in Neural Models of Codecited by 2
13 more
  1. 2023xASTNN: Improved Code Representations for Industrial Practicecited by 2
  2. 2023On a Foundation Model for Operating Systemscited by 2
  3. 2024A Systematic Literature Review on Explainability for Machine/Deep Learning-based Software Engineering Researchcited by 2
  4. 2025CodeSCM: Causal Analysis for Multi-Modal Code Generationcited by 2
  5. 2025Analyzing Latent Concepts in Code Language Modelscited by 2
  6. 2022Learning to Learn to Predict Performance Regressions in Production at Metacited by 1
  7. 2023COMET: Neural Cost Model Explanation Frameworkcited by 1
  8. 2023Redundancy and Concept Analysis for Code-trained Language Modelscited by 1
  9. 2023T-COL: Generating Counterfactual Explanations for General User Preferences on Variable Machine Learning Systemscited by 1
  10. 2025Enabling Global, Human-Centered Explanations for LLMs:From Tokens to Interpretable Code and Test Generationcited by 1
  11. 2025Success is in the Details: Evaluate and Enhance Details Sensitivity of Code LLMs through Counterfactualscited by 1
  12. 2022A Retrospective on ICSE 2022cited by 0
  13. 2025Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectivescited by 0