mlsanityData to 14 Oct 2026

Toward a Theory of Causation for Interpreting Neural Code Models

arXiv 2302.03788 ↗First posted 7 Feb 2023cs.SE cs.AI cs.LGcited by 12

Not on the map. Why?

Abstract (arXiv)

Neural Language Models of Code, or Neural Code Models (NCMs), are rapidly progressing from research prototypes to commercial developer tools. As such, understanding the capabilities and limitations of such models is becoming critical. However, the abilities of these models are typically measured using automated metrics that often only reveal a portion of their real-world performance. While, in general, the performance of NCMs appears promising, currently much is unknown about how such models arrive at decisions. To this end, this paper introduces docode, a post hoc interpretability method specific to NCMs that is capable of explaining model predictions. docode is based upon causal inference to enable programming language-oriented explanations. While the theoretical underpinnings of docode are extensible to exploring different model properties, we provide a concrete instantiation that aims to mitigate the impact of spurious correlations by grounding explanations of model behavior in properties of programming languages. To demonstrate the practical benefit of docode, we illustrate the insights that our framework can provide by performing a case study on two popular deep learning architectures and ten NCMs. The results of this case study illustrate that our studied NCMs are sensitive to changes in code syntax. All our NCMs, except for the BERT-like model, statistically learn to predict tokens related to blocks of code (eg brackets, parenthesis, semicolon) with less confounding bias as compared to other programming language constructs. These insights demonstrate the potential of docode as a useful method to detect and facilitate the elimination of confounding bias in NCMs.

Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). Builds on 53 of its 106 references that are in the map. First posted on arXiv 7 Feb 2023.

Builds on

53 of 106 references.

References · 53

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
  6. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  8. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  9. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  10. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,319
  11. 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
  12. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,304
38 more
  1. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
  2. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  3. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
  4. 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 744
  5. 2016The Mythos of Model Interpretabilitycited by 721
  6. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  7. 2019CodeSearchNet Challenge: Evaluating the State of Semantic Code Searchcited by 654
  8. 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 609
  9. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  10. 2020Unsupervised Translation of Programming Languagescited by 263
  11. 2021Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Modelscited by 222
  12. 2017Learning to Represent Programs with Graphscited by 220
  13. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 163
  14. 2015Visualizing and Understanding Recurrent Networkscited by 160
  15. 2018SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repaircited by 145
  16. 2021The Benchmark Lotterycited by 136
  17. 2018The Adverse Effects of Code Duplication in Machine Learning Models of Codecited by 126
  18. 2020Interpretable Machine Learning -- A Brief History, State-of-the-Art and Challengescited by 120
  19. 2018Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Contextcited by 113
  20. 2018An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translationcited by 108
  21. 2022Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanationscited by 77
  22. 2019Perturbation Sensitivity Analysis to Detect Unintended Model Biasescited by 74
  23. 2021Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Taskscited by 70
  24. 2020Big Code != Big Vocabulary: Open-Vocabulary Models for Source Codecited by 54
  25. 2021Towards Automating Code Review Activitiescited by 54
  26. 2019On Learning Meaningful Code Changes via Neural Machine Translationcited by 50
  27. 2020On the Generalizability of Neural Program Models with respect to Semantic-Preserving Program Transformationscited by 44
  28. 2020On Learning Meaningful Assert Statements for Unit Test Casescited by 42
  29. 2022From Statistical to Causal Learningcited by 40
  30. 2015On the "Naturalness" of Buggy Codecited by 35
  31. 2020A Systematic Literature Review on the Use of Deep Learning in Software Engineering Researchcited by 35
  32. 2017Sorting and Transforming Program Repair Ingredients via Deep Learning Code Similaritiescited by 26
  33. 2021Counterfactual Explanations for Models of Codecited by 25
  34. 2021An Empirical Study on the Usage of BERT Models for Code Completioncited by 22
  35. 2021An Empirical Study on the Usage of Transformer Models for Code Completioncited by 22
  36. 2021DoWhy: Addressing Challenges in Expressing and Validating Causal Assumptionscited by 20
  37. 2022TSSB-3M: Mining single statement bugs at massive scalecited by 14
  38. 2022On the Relationship Between Explanation and Prediction: A Causal Viewcited by 14

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 4; 2025: 6.

12 papers.

  1. 2024HumanEvalComm: Benchmarking the Communication Competence of Code Generation for LLMs and LLM Agentcited by 24
  2. 2023Benchmarking Causal Study to Interpret Large Language Models for Source Codecited by 14
  3. 2024Do Large Code Models Understand Programming Concepts? Counterfactual Analysis for Code Predicatescited by 14
  4. 2024Towards More Trustworthy and Interpretable LLMs for Code through Syntax-Grounded Explanationscited by 11
  5. 2023Evaluating and Explaining Large Language Models for Code Using Syntactic Structurescited by 9
  6. 2024How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Studycited by 6
  7. 2025Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?cited by 5
  8. 2025Enabling Global, Human-Centered Explanations for LLMs:From Tokens to Interpretable Code and Test Generationcited by 1
  9. 2025Understanding Privacy Risks in Code Models Through Training Dynamics: A Causal Approachcited by 1
  10. 2025Toward Neurosymbolic Program Comprehensioncited by 0
  11. 2025Mapping the Trust Terrain: LLMs in Software Engineering -- Insights and Perspectivescited by 0
  12. 2025Toward Explaining Large Language Models in Software Engineering Taskscited by 0