Beyond Final Answers: Auditing Trajectory-Level Hallucinations in Multi-Agent Industrial Workflows
On the map: LLM agents and tool use → Medical and scientific domain agents
Abstract (arXiv)
Large Language Models (LLMs) are increasingly deployed as autonomous agents that reason, use tools, and act over multiple steps. Yet most hallucination benchmarks still evaluate only the final output, missing failures that originate in intermediate Thought-Action-Observation steps. We present Trajel, a dataset and evaluation framework for auditing trajectory-level hallucinations in multi-agent industrial workflows. Trajel introduces a five-type hallucination taxonomy (factual, referential, logical, procedural, and scope-based) over expert-annotated agent traces from AssetOpsBench. We benchmark supervised detection models at the subtask, trajectory, and long-context levels. Our results show that the most common failure modes are missed by existing benchmarks, that nearly half of hallucinated trajectories involve multiple types at once, and that automated detectors with high binary accuracy still misclassify the subtlest types. Trajectory-aware detection significantly outperforms standard post-hoc verification, making taxonomy-grounded evaluation necessary for safer agentic deployment.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 14 of its 23 references that are in the map. First posted on arXiv 22 May 2026.
Builds on
14 of 23 references.
Closest ancestors · 2
Same direction, newest first
- 2025MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them28 Jul
- 2025Using multi-agent architecture to mitigate the risk of LLM hallucinations2 Jul
Other references · 12
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2025Why Do Multi-Agent LLM Systems Fail?cited by 604
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 542
- 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
- 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Usecited by 51
- 2024ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Modelscited by 25
- 2025MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLMcited by 17
- 2025AssetOpsBench: Benchmarking AI Agents for Task Automation in Industrial Asset Operations and Maintenancecited by 15
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents
- 2026Internal Representations as Indicators of Hallucinations in Agent Tool Selection
- 2026LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agents
- 2026Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations
- 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory
- 2025HEAL: An Empirical Study on Hallucinations in Embodied Agents Driven by Large Language Models
- 2025Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models
- 2026HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents