mlsanityData to 14 Oct 2026

Causal and Interpretable Structures in LLM Compositional Tasks

arXiv 2609.35970 ↗First posted 28 Sep 2026cs.CL cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models are able to solve tasks whose answers depend on not only individual input tokens, but also on relations among them. How is such relational information represented and processed across transformer layers? We study activations from ensembles of prompts that require inferring relationships between three tokens corresponding to a cyclic concept (months, hours, weekdays, and musical notes) to correctly predict the next token. Across model families (Llama, Qwen, Gemma, and Mistral) and cyclic concepts, we find a consistent layerwise progression in how the joint dependence among the tokens is geometrically organized and causally used: intermediate layers use a joint representation based on the inferred relationship between two tokens, while later layers use a joint representation associated with all three tokens to correctly complete the task. We also find other relationships between tokens that are geometrically structured but remain causally inert in the next-token prediction. Crucially, when taken together, these geometric and causal investigations reveal the representation-level mechanism that progressively organizes and composes the relational information to form the answer. More surprisingly, restricting the models to such causally relevant joint representations improves next-token prediction accuracy.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 30 of its 37 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

30 of 37 references.

References · 30

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  3. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
  4. 2021Causal Abstractions of Neural Networkscited by 422
  5. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 359
  6. 2023Function Vectors in Large Language Modelscited by 308
  7. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 225
  8. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  9. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  10. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  11. 2024The Geometry of Categorical and Hierarchical Concepts in Large Language Modelscited by 159
  12. 2023How do Language Models Bind Entities in Context?cited by 108
18 more
  1. 2023Language Models Implement Simple Word2Vec-style Vector Arithmeticcited by 98
  2. 2025Language Models Use Trigonometry to Do Additioncited by 70
  3. 2025The Origins of Representation Manifolds in Large Language Modelscited by 51
  4. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  5. 2026When Models Manipulate Manifolds: The Geometry of a Counting Taskcited by 46
  6. 2024Pre-trained Large Language Models Use Fourier Features to Compute Additioncited by 39
  7. 2023Discovering Variable Binding Circuitry with Desideratacited by 34
  8. 2024Language Models Encode Numbers Using Digit Representations in Base 10cited by 32
  9. 2026Manifold Steering Reveals the Shared Geometry of Neural Network Representation and Behaviorcited by 26
  10. 2026Symmetry in language statistics shapes the geometry of model representationscited by 24
  11. 2026The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriorscited by 15
  12. 2024Towards Empirical Interpretation of Internal Circuits and Properties in Grokked Transformers on Modular Polynomialscited by 14
  13. 2026From Data Statistics to Feature Geometry: How Correlations Shape Superpositioncited by 10
  14. 2026Arithmetic in the Wild: Llama uses Base-10 Addition to Reason About Cyclic Conceptscited by 10
  15. 2025In-Context Algebracited by 5
  16. 2026Cell-Based Representation of Relational Binding in Language Modelscited by 5
  17. 2024Locating and Extracting Relational Concepts in Large Language Modelscited by 3
  18. 2026Language Models Represent and Transform Concepts with Shared Geometrycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Unveiling the Reasoning Process of Large Language Models
  2. 2025Bridging the Dimensional Chasm: Uncover Layer-wise Dimensional Reduction in Transformers through Token Correlation
  3. 2024Interpreting token compositionality in LLMs: A robustness analysis
  4. 2025How Language Directions Align with Token Geometry in Multilingual LLMs
  5. 2024Exploring Concept Depth: How Large Language Models Acquire Knowledge and Concept at Different Layers?
  6. 2025Can an LLM Induce a Graph? Investigating Memory Drift and Context Length
  7. 2024Does Representation Matter? Exploring Intermediate Layers in Large Language Models
  8. 2026StructLens: A Structural Lens for Language Models via Maximum Spanning Trees