mlsanityData to 7 Oct 2026

HalluClear: Diagnosing, Evaluating and Mitigating Hallucinations in GUI Agents

arXiv 2604.17284 ↗First posted 19 Apr 2026cs.AIcited by 3

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

While progress in GUI agents has been largely driven by industrial-scale training, ungrounded hallucinations often trigger cascading failures in real-world deployments.Unlike general VLM domains, the GUI agent field lacks a hallucination-focused suite for fine-grained diagnosis, reliable evaluation, and targeted mitigation.To bridge this gap, we introduce HalluClear, a comprehensive suite for hallucination mitigation in GUI agents as a complement to computation-intensive scaling. HalluClear comprises: (1) a GUI-specific hallucination taxonomy derived from empirical failure analysis; (2) a calibrated three-stage evaluation workflow which enhances VLM-as-a-judge reliability via expert-annotated benchmarking and ensemble credibility estimation; and (3) a mitigation scheme based on closed-loop structured reasoning, enabling lightweight continual post-training with cold-start initialization for both generalist and GUI-specialist agents. Experiments across representative agents and public benchmarks demonstrate that post-training on only 9K samples within our suite can significantly reduce hallucinations, thereby improving grounding and action fidelity, offering a compute-efficient pathway to robust GUI automation.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 58 of its 67 references that are in the map. First posted on arXiv 19 Apr 2026.

Builds on

58 of 67 references.

References · 58

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  6. 2025Qwen2.5-VL Technical Reportcited by 5,418
  7. 2024GPT-4o System Cardcited by 4,467
  8. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  9. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  11. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  12. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
38 more
  1. 2025Qwen3-VL Technical Reportcited by 2,534
  2. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  3. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  4. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  6. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  7. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  8. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  9. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  10. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 584
  11. 2024MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?cited by 573
  12. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 572
  13. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  14. 2024Measuring Multimodal Mathematical Reasoning with MATH-Vision Datasetcited by 516
  15. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 501
  16. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 380
  17. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agentscited by 373
  18. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agentscited by 370
  19. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  20. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Modelscited by 329
  21. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 284
  22. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  23. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  24. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Usecited by 246
  25. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agentscited by 240
  26. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  27. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  28. 2025Why Language Models Hallucinatecited by 205
  29. 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 190
  30. 2024On the Effects of Data Scale on UI Control Agentscited by 187
  31. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 183
  32. 2025UI-R1: Enhancing Efficient Action Prediction of GUI Agents by Reinforcement Learningcited by 166
  33. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
  34. 2025OpenCUA: Open Foundations for Computer-Use Agentscited by 151
  35. 2023The Troubling Emergence of Hallucination in Large Language Models -- An Extensive Definition, Quantification, and Prescriptive Remediationscited by 130
  36. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 123
  37. 2024GUICourse: From General Vision Language Models to Versatile GUI Agentscited by 112
  38. 2025GTA1: GUI Test-time Scaling Agentcited by 102

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026DiagEval: Trajectory-Conditioned Diagnosis for Reliable Software Evaluation with GUI Agentscited by 2
  2. 2026A History-Aware Visually Grounded Critic for Computer Use Agentscited by 1
  3. 2026The GUI Is Not the State: Diagnosing State Aliasing in GUI World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026AgentHallu: Benchmarking Automated Hallucination Attribution of LLM-based Agents
  2. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directions
  3. 2026PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinations
  4. 2025MIRAGE-Bench: LLM Agent is Hallucinating and Where to Find Them
  5. 2025InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration
  6. 2025Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systems
  7. 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory
  8. 2025HEAL: An Empirical Study on Hallucinations in Embodied Agents Driven by Large Language Models