CodeSCM: Causal Analysis for Multi-Modal Code Generation
Not on the map. Why?
Abstract (arXiv)
In this paper, we propose CodeSCM, a Structural Causal Model (SCM) for analyzing multi-modal code generation using large language models (LLMs). By applying interventions to CodeSCM, we measure the causal effects of different prompt modalities, such as natural language, code, and input-output examples, on the model. CodeSCM introduces latent mediator variables to separate the code and natural language semantics of a multi-modal code generation prompt. Using the principles of Causal Mediation Analysis on these mediators we quantify direct effects representing the model's spurious leanings. We find that, in addition to natural language instructions, input-output examples significantly influence code generation.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 51 references that are in the map. First posted on arXiv 7 Feb 2025.
Builds on
33 of 51 references.
References · 33
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2021Program Synthesis with Large Language Modelscited by 3,637
- 2023Qwen Technical Reportcited by 3,344
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
- 2023Code Llama: Open Foundation Models for Codecited by 2,139
- 2022Competition-Level Code Generation with AlphaCodecited by 1,462
- 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,420
- 2023Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generationcited by 1,391
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,310
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
21 more
- 2021Measuring Coding Challenge Competence With APPScited by 902
- 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 817
- 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 733
- 2023A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPTcited by 619
- 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
- 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
- 2024DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligencecited by 331
- 2013Direct and Indirect Effectscited by 280
- 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 178
- 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
- 2022Conversing with Copilot: Exploring Prompt Engineering for Solving CS1 Problems Using Natural Languagecited by 82
- 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Modelscited by 68
- 2019PC-Fairness: A Unified Framework for Measuring Causality-based Fairnesscited by 67
- 2015Program Synthesis using Natural Languagecited by 41
- 2023Self-Prompting Large Vision Models for Few-Shot Medical Image Segmentationcited by 35
- 2023Outline, Then Details: Syntactically Guided Coarse-To-Fine Code Generationcited by 27
- 2021An Isotropy Analysis in the Multilingual BERT Embedding Spacecited by 25
- 2021Counterfactual Explanations for Models of Codecited by 25
- 2023Towards Causal Deep Learning for Vulnerability Detectioncited by 9
- 2024Intent Detection and Entity Extraction from BioMedical Literaturecited by 6
Built on by
Citing papers per year: 2025: 2.
2 papers.