mlsanityData to 7 Oct 2026

When Does Correction Become Repair? Mechanistic Auditing of Internal Interventions in Tool-Using LLMs

arXiv 2609.36138 ↗First posted 28 Sep 2026cs.CL cs.SEcited by 0

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Before invoking external tools, an agentic LLM must select among a K-way action space: executing a call, seeking clarification, answering directly, or declining. While internal activation steering can alter these pre-execution decisions, conventional aggregate metrics obscure where altered states land and what collateral damage they inflict. We present SAKIKO, an auditing framework that formalizes representation repair via directional error discovery, router-conditioned intervention, destination-resolved verification, and prospectively frozen statistical licensing. Across seven LLMs on When2Call and MetaTool, channel-keyed interventions induce direction-specific net gains in five models; across three sealed evaluations, none of 59 budget-matched random directions matches calibrated target gain. Crucially, destination auditing shows that behavioral movement does not equal repair: an intervention achieving +55 net gain corrupts over half of the baseline-correct decisions it touches, and promising point estimates on Qwen3-4B and Gemma-2-9B are formally declined due to finite-sample uncertainty. SAKIKO establishes the necessity of outcome-resolved adjudication before claiming internal repair. Code: https://github.com/ruizheliUOA/mechanistic-tool-use-llm.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 28 of its 30 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

28 of 30 references.

Closest ancestors · 16

Same direction, newest first

  1. 2026Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling1 Sep
  2. 2026To Call or Not to Call: Diagnosing Intrinsic Over-Calling Bias in LLM Agents16 May
  3. 2026Tool Calling is Linearly Readable and Steerable in Language Models8 May
  4. 2026To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling1 May
  5. 2026ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agents4 Feb
  6. 2026ToolPRMBench: Evaluating and Advancing Process Reward Models for Tool-using Agents18 Jan
  7. 2026Lost in Execution: On the Multilingual Robustness of Tool Calling in Large Language Models8 Jan
  8. 2025ToolRM: Outcome Reward Models for Tool-Calling Large Language Models15 Sep
  9. 2025Position: Agent Should Invoke External Tools ONLY When Epistemically Necessary1 Jun
  10. 2025When2Call: When (not) to Call Tools26 Apr
  11. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  12. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
4 more
  1. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  2. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  3. 2023Tool Learning with Foundation Models17 Apr
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 12

  1. 2022Emergent Abilities of Large Language Modelscited by 2,483
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  4. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  5. 2025Structured Uncertainty guided Clarification for LLM Agentscited by 34
  6. 2026Internal Representations as Indicators of Hallucinations in Agent Tool Selectioncited by 18
  7. 2026Spurious Rewards Paradox: Mechanistically Understanding How RLVR Activates Memorization Shortcuts in LLMscited by 13
  8. 2025Attributing Response to Context: A Jensen-Shannon Divergence Driven Mechanistic Study of Context Attribution in Retrieval-Augmented Generationcited by 12
  9. 2026From Text to Voice: A Reproducible and Verifiable Framework for Evaluating Tool Calling LLM Agentscited by 4
  10. 2026Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learningcited by 4
  11. 2026Beyond the Black Box: Interpretability of Agentic AI Tool Usecited by 3
  12. 2026When Behavioral Safety Evaluation Fails: A Representation-Level Perspectivecited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Auditing Action Settlement in LLM Agent Environments: Order, Progress, and Replay
  2. 2026Credit Without Ground Truth: Auditing Step-Level Credit Assignment in LLM Agents Against Executed Replay
  3. 2026Feedback That Backfires: Why Small Language Model Agents Repeat the Call They Just Watched Fail
  4. 2026Designing Reliable LLM-as-a-Judge Measurement Systems for Multi-Turn Business Agents
  5. 2026Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation
  6. 2026Phantom Gains: Auditing Self-Improvement Against a Measured Null
  7. 2026From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents
  8. 2026Loud Failures, Quiet Failures: Fault Detection and Recovery in Tool-Using Language Model Agents