mlsanityData to 14 Oct 2026

Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures

arXiv 2607.27617 ↗First posted 30 Jul 2026cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

Identical language-model answers can arise from hidden states that support different future computations, so current-answer probes do not establish a reusable internal interface. We introduce forked futures: future operations are sampled only after a prefix state has formed, and states are compared through the response distributions induced by those operations. This yields an empirical causal quotient over hidden states without requiring researcher-specified latent labels. Shared, Local, Mixture, and Distributed interfaces then compete under prequential causal description length subject to future-signature fidelity and matched capacity constraints. In the two detailed model evaluations, Shared has the lowest held-out description length, with gains of 0.216 nats on Qwen2.5-1.5B and 0.294 nats on Llama-3-8B, while maintaining tightly clustered mean future-signature distortion; a five-backbone sweep preserves the positive direction of Sharedness Gain. The figure-aligned transplantation analysis gives Shared the strongest joint target-correctness, locality, copy-preservation, and composite profile, and API-aligned paths mediate 0.749 of the target effect versus 0.150 for matched null paths. In the blind four-class model-organism test, 14/16 architectures are recovered, with one observed non-Shared to Shared error among 12 non-Shared organisms. These results support an economical reusable causal interface within the tested operation banks, while keeping the claim explicitly conditional on the candidate architectures, interventions, and held-out futures.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 30 Jul 2026.

Builds on

26 of 37 references.

References · 26

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,375
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023Steering Language Models With Activation Engineeringcited by 922
  6. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  7. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  8. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
  9. 2024Scaling and evaluating sparse autoencoderscited by 553
  10. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
  11. 2021Causal Abstractions of Neural Networkscited by 422
  12. 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 324
14 more
  1. 2023In-Context Learning Creates Task Vectorscited by 312
  2. 2024Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Modelscited by 235
  3. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  4. 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
  5. 2023Localizing Model Behavior with Path Patchingcited by 195
  6. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 191
  7. 2017The Consciousness Priorcited by 107
  8. 2018Abstracting Causal Modelscited by 107
  9. 2025Model Organisms for Emergent Misalignmentcited by 91
  10. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
  11. 2026Verbalizable Representations Form a Global Workspace in Language Modelscited by 75
  12. 2018The Description Length of Deep Learning Modelscited by 72
  13. 2022Causal Abstraction with Soft Interventionscited by 23
  14. 2026The Model Organism Lottery: Model Organism Interpretability Strongly Depends on Training Methodologycited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.