# Internal Representations as Indicators of Hallucinations in Agent Tool Selection

arXiv: https://arxiv.org/abs/2601.05214 · HTML: https://mlsanity.com/paper/2601.05214/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use.
- Builds on 15 of its 32 references that are in the map.
- First posted on arXiv 8 Jan 2026.

## Placed in

- LLM agents and tool use / [Medical and scientific domain agents](https://mlsanity.com/agents/medical-and-scientific-domain-agents/)

## Builds on (most cited first, 15 of 15)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases](https://mlsanity.com/paper/2306.05301/) (2023)
- [An Overview of Catastrophic AI Risks](https://mlsanity.com/paper/2306.12001/) (2023)
- [Alignment of Language Agents](https://mlsanity.com/paper/2103.14659/) (2021)
- [Enhancing Uncertainty-Based Hallucination Detection with Stronger Focus](https://mlsanity.com/paper/2311.13230/) (2023)
- [Lemur: Harmonizing Natural Language and Code for Language Agents](https://mlsanity.com/paper/2310.06830/) (2023)
- [A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagation](https://mlsanity.com/paper/2406.06950/) (2024)

## Built on by (most cited first, 19 of 19)

- [Tool Calling is Linearly Readable and Steerable in Language Models](https://mlsanity.com/paper/2605.07990/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [Your Agentic LLMs Secretly Encode Indirect Prompt-Injection Exposure in Hidden States](https://mlsanity.com/paper/2608.02657/) (2026)
- [Exploring Agentic Tool-Calling Decisions via Uncertainty-Aligned Reinforcement Learning](https://mlsanity.com/paper/2606.06976/) (2026)
- [Beyond the Black Box: Interpretability of Agentic AI Tool Use](https://mlsanity.com/paper/2605.06890/) (2026)
- [Functional Entropy: Predicting Functional Correctness in LLM-Generated Code with Uncertainty Quantification](https://mlsanity.com/paper/2605.28500/) (2026)
- [A Few Neurons Reveal When LLMs Misuse Tools: Sparse Detection and Selective Steering for Reliable Tool Use](https://mlsanity.com/paper/2608.00218/) (2026)
- [Spectral Guardrails for Agents in the Wild: Detecting Tool Use Hallucinations via Attention Topology](https://mlsanity.com/paper/2602.08082/) (2026)
- [Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critique](https://mlsanity.com/paper/2608.10430/) (2026)
- [Tool-Call Dependency Structure is Linearly Decodable in LLM Agent Residual Streams](https://mlsanity.com/paper/2605.25310/) (2026)
- [How to Speculate about Uncertainty in Agentic Coding? A Draft-Model Gate Method](https://mlsanity.com/paper/2609.05274/) (2026)
- [Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls](https://mlsanity.com/paper/2608.03071/) (2026)
- [Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards](https://mlsanity.com/paper/2608.07531/) (2026)
- [Invocation-Level Reliability of Tool-Using Agents](https://mlsanity.com/paper/2608.26189/) (2026)
- [The Calls are Coming from Inside the Model: Investigating Probe-based Detection of Tool-Calling Errors in LLMs](https://mlsanity.com/paper/2608.27750/) (2026)
- [Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study](https://mlsanity.com/paper/2609.07395/) (2026)
- [Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations](https://mlsanity.com/paper/2609.09448/) (2026)
- [When Does Correction Become Repair? Mechanistic Auditing of Internal Interventions in Tool-Using LLMs](https://mlsanity.com/paper/2609.36138/) (2026)
- [Structured but Silent: Probing Capability Requirements in LLM Hidden States](https://mlsanity.com/paper/2610.08018/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have shown remarkable capabilities in tool calling and tool usage, but suffer from hallucinations where they choose incorrect tools, provide malformed parameters and exhibit 'tool bypass' behavior by performing simulations and generating outputs instead of invoking specialized tools or external systems. This undermines the reliability of LLM based agents in production systems as it leads to inconsistent results, and bypasses security and audit controls. Such hallucinations in agent tool selection require early detection and error handling. Unlike existing hallucination detection methods that require multiple forward passes or external validation, we present a computationally efficient framework that detects tool-calling hallucinations in real-time by leveraging LLMs' internal representations during the same forward pass used for generation. We evaluate this approach on reasoning tasks across multiple domains, demonstrating strong detection performance (up to 86.4% accuracy) while maintaining real-time inference capabilities with minimal computational overhead, particularly excelling at detecting parameter-level hallucinations and inappropriate tool selections, critical for reliable agent deployment.