Test, then Route: How Language Models Execute In-Context Conditional Rules Across Models and Languages
Not on the map. Why?
Abstract (arXiv)
When a language model follows an in-context conditional rule such as "if P(x) then A else B," does it assemble a runtime circuit with one module that tests the predicate and another that routes the answer? We probe this with activation patching under a four-donor design whose two swapped-rule donors make the condition and the answer word disagree, so each layer reveals which of the two it carries. Across three open models from two families and six languages sharing one fixed item bank, a mid-stack residual band carries the predicate's truth value: patching it reroutes the answer with predicate-outcome flip near 1.0 and mapping flip near 0.0, meeting a strict pre-specified isolation criterion in 17 of 18 cells, and the same localization holds across five predicate families. The router shows the opposite profile. A learned subspace flips A and B near-perfectly within the trained pair yet transfers to a new pair at approximately 0 in every model, while in Gemma-3-4B (the only model probed cross-lingually) it transfers at approximately 0.98 to the same pair in other languages. Under every probe we ran, the router direction is token-bound and non-transferable (largely answer-readout in Gemma, pair-specific in Qwen) rather than an abstract routing module. Test is modular; under these probes, route is not.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 4 Aug 2026.
Builds on
29 of 43 references.
References · 29
- 2022Locating and Editing Factual Associations in GPTcited by 2,609
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
- 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
- 2023In-Context Learning Creates Task Vectorscited by 302
- 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
- 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
- 2024How to use and interpret activation patchingcited by 191
- 2024Programming Refusal with Conditional Activation Steeringcited by 191
17 more
- 2023Localizing Model Behavior with Path Patchingcited by 182
- 2024How do Large Language Models Handle Multilingualism?cited by 148
- 2023How do Language Models Bind Entities in Context?cited by 102
- 2023Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patchingcited by 75
- 2024Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformerscited by 42
- 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 34
- 2023Discovering Variable Binding Circuitry with Desideratacited by 32
- 2024Reasoning Circuits in Language Models: A Mechanistic Interpretation of Syllogistic Inferencecited by 28
- 2024On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Taskcited by 24
- 2025Mixing Mechanisms: How Language Models Retrieve Bound Entities In-Contextcited by 24
- 2025How Do Transformers Learn Variable Binding in Symbolic Programs?cited by 19
- 2024A Implies B: Circuit Analysis in LLMs for Propositional Logical Reasoningcited by 15
- 2025The Transfer Neurons Hypothesis: An Underlying Mechanism for Language Latent Space Transitions in Multilingual LLMscited by 8
- 2025LLMs Process Lists With General Filter Headscited by 7
- 2026Causality neq Invariance: Function and Concept Vectors in LLMscited by 7
- 2026Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lenscited by 5
- 2025Function Induction and Task Generalization: An Interpretability Study with Off-by-One Additioncited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.