Spectral Guardrails for Agents in the Wild: Detecting Tool Use Hallucinations via Attention Topology
On the map: LLM safety and security → Machine unlearning
Abstract (arXiv)
Deploying autonomous agents in the wild requires reliable safeguards against tool use failures. We propose a training free guardrail based on spectral analysis of attention topology that complements supervised approaches. On Llama 3.1 8B, our method achieves 97.7% recall with multi-feature detection and 86.1% recall with 81.0% precision for balanced deployment, without requiring any labeled training data. Most remarkably, we discover that single layer spectral features act as near-perfect hallucination detectors: Llama L26 Smoothness achieves 98.2% recall (213/217 hallucinations caught) with a single threshold, and Mistral L3 Entropy achieves 94.7% recall. This suggests hallucination is not merely a wrong token but a thermodynamic state change: the model's attention becomes noise when it errs. Through controlled cross-model evaluation on matched domains (N=1000, T=0.3, same General domain, hallucination rates 20--22%), we reveal the ``Loud Liar'' phenomenon: Llama 3.1 8B's failures are spectrally catastrophic and dramatically easier to detect, while Mistral 7B achieves the best discrimination (AUC 0.900). These findings establish spectral analysis as a principled, efficient framework for agent safety.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Machine unlearning, one of 12 directions in LLM safety and security. Builds on 15 of its 27 references that are in the map. First posted on arXiv 8 Feb 2026.
Builds on
15 of 27 references.
References · 15
- 2023Let's Verify Step by Stepcited by 3,598
- 2023Mistral 7Bcited by 3,065
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023The Internal State of an LLM Knows When It's Lyingcited by 644
- 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
- 2020Hutch++: Optimal Stochastic Trace Estimationcited by 62
- 2025Hallucination Detection in LLMs Using Spectral Features of Attention Mapscited by 42
- 2024A Probabilistic Framework for LLM Hallucination Detection via Belief Tree Propagationcited by 21
- 2026Internal Representations as Indicators of Hallucinations in Agent Tool Selectioncited by 18
- 2025Towards Mechanistic Interpretability of Graph Transformers via Attention Graphscited by 12
Built on by
Citing papers per year: 2026: 2.
2 papers.