mlsanityData to 14 Oct 2026

Task Schema and Binding: A Double Dissociation Study of In-Context Learning

arXiv 2512.17325 ↗First posted 19 Dec 2025cs.LG cs.CLcited by 3

Not on the map. Why?

Abstract (arXiv)

We provide causal mechanistic validation that in-context learning (ICL) decomposes into two separable mechanisms: Task Schema (abstract task type recognition) and Binding (specific input-output associations). Through activation patching experiments across 9 models from 7 Transformer families plus Mamba (370M-13B parameters), we establish three key findings: 1. Double dissociation: Task Schema transfers at 100% via late MLP patching; Binding transfers at 62% via residual stream patching -- proving separable mechanisms 2. Prior-Schema trade-off: Schema reliance inversely correlates with prior knowledge (Spearman rho = -0.596, p < 0.001, N=28 task-model pairs) 3. Architecture generality: The mechanism operates across all tested architectures including the non-Transformer Mamba These findings offer a mechanistic account of the ICL puzzle that contrasts with prior views treating ICL as a monolithic mechanism (whether retrieval-based, gradient descent-like, or purely Bayesian). By establishing that Schema and Binding are neurally dissociable -- not merely behavioral modes -- we provide causal evidence for dual-process theories of ICL. Models rely on Task Schema when prior knowledge is absent, but prior knowledge interferes through attentional mis-routing (72.7% recency bias) rather than direct output competition (0%). This explains why arbitrary mappings succeed (zero prior leads to full Schema reliance) while factual overrides fail -- and reveals that the true bottleneck is attentional, not output-level. Practical implications: Understanding these dual mechanisms enables more efficient prompt engineering -- reliable schema transfer reduces required demonstrations for novel tasks, while prior-aware design can mitigate the 38% binding failure rate in high-prior scenarios, improving ICL system reliability in production deployments.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 14 of its 17 references that are in the map. First posted on arXiv 19 Dec 2025.

Builds on

14 of 17 references.

References · 14

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  5. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  6. 2021An Explanation of In-context Learning as Implicit Bayesian Inferencecited by 938
  7. 2022Transformers learn in-context by gradient descentcited by 732
  8. 2023Larger language models do in-context learning differentlycited by 394
  9. 2022Data Distributional Properties Drive Emergent In-Context Learning in Transformerscited by 309
  10. 2023Function Vectors in Large Language Modelscited by 308
  11. 2023Transformers as Algorithms: Generalization and Stability in In-context Learningcited by 222
  12. 2023What In-Context Learning "Learns" In-Context: Disentangling Task Recognition and Task Learningcited by 152
2 more
  1. 2024Is Mamba Capable of In-Context Learning?cited by 45
  2. 2023Understanding In-Context Learning from Repetitionscited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Belief or Circuitry? Causal Evidence for In-Context Graph Learningcited by 0
  2. 2026Context Is King: How In-Context Specification Shapes the Geometry of Conceptscited by 0
  3. 2026From Copying to Composition: How Language Models Develop In-Context Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks
  2. 2025Schema for In-Context Learning
  3. 2025Probing In-Context Learning: Impact of Task Complexity and Model Architecture on Generalization and Efficiency
  4. 2023Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability
  5. 2024Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
  6. 2025Understanding In-Context Learning Beyond Transformers: An Investigation of State Space and Hybrid Architectures
  7. 2026Pattern Selectivity is Not Task-Causal Structure: A Cross-Architecture Mechanistic Study of Composed-Task Circuits in 1B-Class Language Models
  8. 2026Single-Position Intervention Fails: Distributed Output Templates Drive In-Context Learning