# Causal Interpretation of Self-Attention in Pre-Trained Transformers

arXiv: https://arxiv.org/abs/2310.20307 · HTML: https://mlsanity.com/paper/2310.20307/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 48 references that are in the map.
- First posted on arXiv 31 Oct 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR](https://mlsanity.com/paper/1711.00399/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism](https://mlsanity.com/paper/1608.05745/) (2016)
- [The Causal-Neural Connection: Expressiveness, Learnability, and Inference](https://mlsanity.com/paper/2107.00793/) (2021)
- [Neural Causal Models for Counterfactual Identification and Estimation](https://mlsanity.com/paper/2210.00035/) (2022)
- [Learning Sparse Causal Models is not NP-hard](https://mlsanity.com/paper/1309.6824/) (2013)
- [Structural Agnostic Modeling: Adversarial Learning of Causal Graphs](https://mlsanity.com/paper/1803.04929/) (2018)
- [Counterfactual Explanations for Neural Recommenders](https://mlsanity.com/paper/2105.05008/) (2021)
- [Iterative Causal Discovery in the Possible Presence of Latent Confounders and Selection Bias](https://mlsanity.com/paper/2111.04095/) (2021)
- [From Temporal to Contemporaneous Iterative Causal Discovery in the Presence of Latent Confounders](https://mlsanity.com/paper/2306.00624/) (2023)
- [Improving Efficiency and Accuracy of Causal Discovery Using a Hierarchical Wrapper](https://mlsanity.com/paper/2107.05001/) (2021)
- [CLEAR: Causal Explanations from Attention in Neural Recommenders](https://mlsanity.com/paper/2210.10621/) (2022)

## Built on by (most cited first, 30 of 30)

- [DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models](https://mlsanity.com/paper/2405.20985/) (2024)
- [Token Activation Map to Visually Explain Multimodal LLMs](https://mlsanity.com/paper/2506.23270/) (2025)
- [Causality for Large Language Models](https://mlsanity.com/paper/2410.15319/) (2024)
- [Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering](https://mlsanity.com/paper/2411.10950/) (2024)
- [Resilient Contrastive Pre-training under Non-Stationary Drift](https://mlsanity.com/paper/2502.07620/) (2025)
- [Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models](https://mlsanity.com/paper/2406.12649/) (2024)
- [A Causal World Model Underlying Next Token Prediction: Exploring GPT in a Controlled Environment](https://mlsanity.com/paper/2412.07446/) (2024)
- [SurgXBench: Explainable Vision-Language Model Benchmark for Surgery](https://mlsanity.com/paper/2505.10764/) (2025)
- [Explain the Black Box for the Sake of Science: the Scientific Method in the Era of Generative Artificial Intelligence](https://mlsanity.com/paper/2406.10557/) (2024)
- [Walking the Tightrope: Disentangling Beneficial and Detrimental Drifts in Non-Stationary Custom-Tuning](https://mlsanity.com/paper/2505.13081/) (2025)
- [Towards Practical Multi-label Causal Discovery in High-Dimensional Event Sequences via One-Shot Graph Aggregation](https://mlsanity.com/paper/2509.19112/) (2025)
- [One-Shot Multi-Label Causal Discovery in High-Dimensional Event Sequences](https://mlsanity.com/paper/2509.23213/) (2025)
- [CausalDetox: Causal Head Selection and Intervention for Language Model Detoxification](https://mlsanity.com/paper/2604.14602/) (2026)
- [Your Autoregressive Model Already Reveals the Causal Graph](https://mlsanity.com/paper/2602.01135/) (2026)
- [Structural Disentanglement of Causal and Correlated Concepts](https://mlsanity.com/paper/2405.16219/) (2024)
- [CHANI: Correlation-based Hawkes Aggregation of Neurons with bio-Inspiration](https://mlsanity.com/paper/2405.18828/) (2024)
- [Inner-Probe: Discovering Copyright-related Data Generation in LLM Architecture](https://mlsanity.com/paper/2410.04454/) (2024)
- [Causal Process Models: Reframing Dynamic Causal Graph Discovery as a Reinforcement Learning Problem](https://mlsanity.com/paper/2507.13920/) (2025)
- [Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems](https://mlsanity.com/paper/2509.15448/) (2025)
- [PropLLM: Propagation-Aware Scene Reconstruction for Network Fault Diagnosis](https://mlsanity.com/paper/2606.00582/) (2026)
- [Transformers with Sparse Attention for Granger Causality](https://mlsanity.com/paper/2411.13264/) (2024)
- [Dual Filter: A Transformer-like Inference Architecture for Hidden Markov Models](https://mlsanity.com/paper/2505.00818/) (2025)
- [Reversal Invariance in Autoregressive Language Models](https://mlsanity.com/paper/2511.00341/) (2025)
- [Memory Retrieval in Transformers: Insights from The Encoding Specificity Principle](https://mlsanity.com/paper/2601.20282/) (2026)
- [Multi-Agent Causal Reasoning System for Error Pattern Rule Automation in Vehicles](https://mlsanity.com/paper/2602.01155/) (2026)
- [ReBeCA: Unveiling Interpretable Behavior Hierarchy behind the Iterative Self-Reflection of Language Models with Causal Analysis](https://mlsanity.com/paper/2602.06373/) (2026)
- [Lifelong Learning in Vision-Language Models: Enhanced EWC with Cross-Modal Knowledge Retention](https://mlsanity.com/paper/2605.12789/) (2026)
- [Transformer-like Inference from Optimal Control](https://mlsanity.com/paper/2605.15608/) (2026)
- [seq2cause: One Autoregressive Backbone, Four Causal Discovery Tasks in Event Sequences](https://mlsanity.com/paper/2609.31801/) (2026)
- [Small transformers track Bayesian evidence for latent common causes via a context-invariant mechanism](https://mlsanity.com/paper/2609.35161/) (2026)

## Abstract (arXiv)

We propose a causal interpretation of self-attention in the Transformer neural network architecture. We interpret self-attention as a mechanism that estimates a structural equation model for a given input sequence of symbols (tokens). The structural equation model can be interpreted, in turn, as a causal structure over the input symbols under the specific context of the input sequence. Importantly, this interpretation remains valid in the presence of latent confounders. Following this interpretation, we estimate conditional independence relations between input symbols by calculating partial correlations between their corresponding representations in the deepest attention layer. This enables learning the causal structure over an input sequence using existing constraint-based algorithms. In this sense, existing pre-trained Transformers can be utilized for zero-shot causal-discovery. We demonstrate this method by providing causal explanations for the outcomes of Transformers in two tasks: sentiment classification (NLP) and recommendation.