mlsanityData to 7 Oct 2026

Small Agent Can Also Rock! Empowering Small Language Models as Hallucination Detector

arXiv 2406.11277 ↗First posted 17 Jun 2024cs.CLcited by 20

On the map: LLM agents and tool use → Medical and scientific domain agents

Abstract (arXiv)

Hallucination detection is a challenging task for large language models (LLMs), and existing studies heavily rely on powerful closed-source LLMs such as GPT-4. In this paper, we propose an autonomous LLM-based agent framework, called HaluAgent, which enables relatively smaller LLMs (e.g. Baichuan2-Chat 7B) to actively select suitable tools for detecting multiple hallucination types such as text, code, and mathematical expression. In HaluAgent, we integrate the LLM, multi-functional toolbox, and design a fine-grained three-stage detection framework along with memory mechanism. To facilitate the effectiveness of HaluAgent, we leverage existing Chinese and English datasets to synthesize detection trajectories for fine-tuning, which endows HaluAgent with the capability for bilingual hallucination detection. Extensive experiments demonstrate that only using 2K samples for tuning LLMs, HaluAgent can perform hallucination detection on various types of tasks and datasets, achieving performance comparable to or even higher than GPT-4 without tool enhancements on both in-domain and out-of-domain datasets. We release our dataset and code at https://github.com/RUCAIBox/HaluAgent.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Medical and scientific domain agents, one of 16 directions in LLM agents and tool use. Builds on 25 of its 35 references that are in the map. First posted on arXiv 17 Jun 2024.

Builds on

25 of 35 references.

References · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  5. 2023Qwen Technical Reportcited by 3,273
  6. 2023A Survey of Large Language Modelscited by 2,537
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  9. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  10. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 793
  11. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 787
  12. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 751
13 more
  1. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  2. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
  3. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  4. 2023AgentTuning: Enabling Generalized Agent Abilities for LLMscited by 319
  5. 2023FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarioscited by 247
  6. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 155
  7. 2024Long-form factuality in large language modelscited by 146
  8. 2024KG-Agent: An Efficient Autonomous Agent Framework for Complex Reasoning over Knowledge Graphcited by 90
  9. 2021A Survey on Complex Knowledge Base Question Answering: Methods, Challenges and Solutionscited by 77
  10. 2020Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problemscited by 60
  11. 2016Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answeringcited by 35
  12. 2023Chain of Natural Language Inference for Reducing Large Language Model Ungrounded Hallucinationscited by 34
  13. 2024Hallucination Detection and Hallucination Mitigation: An Investigationcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 12; 2026: 5.

20 papers.

Sort:

By area:

  1. 2025LLM-based Agents Suffer from Hallucinations: A Survey of Taxonomy, Methods, and Directionscited by 49
  2. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  3. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  4. 2024FIRE: Fact-checking with Iterative Retrieval and Verificationcited by 39
  5. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  6. 2025A Unified Agentic Framework for Evaluating Conditional Image Generationcited by 15
  7. 2025ComfyUI-Copilot: An Intelligent Assistant for Automated Workflow Developmentcited by 11
  8. 2025Don't Adapt Small Language Models for Tools; Adapt Tool Schemas to the Modelscited by 6
  9. 2025D2HScore: Reasoning-Aware Hallucination Detection via Semantic Breadth and Depth Analysis in LLMscited by 5
  10. 2026Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectorycited by 5
  11. 2024SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detectioncited by 4
  12. 2025Consistency Is the Key: Detecting Hallucinations in LLM Generated Text By Checking Inconsistencies About Key Factscited by 4
8 more
  1. 2026When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agentscited by 3
  2. 2026Interpreting and Controlling LLM Reasoning through Integrated Policy Gradientcited by 3
  3. 2025Does It Run and Is That Enough? Revisiting Text-to-Chart Generation with a Multi-Agent Approachcited by 1
  4. 2025JointCQ: Improving Factual Hallucination Detection with Joint Claim and Query Generationcited by 0
  5. 2025Can a Small Model Learn to Look Before It Leaps? Dynamic Learning and Proactive Correction for Hallucination Detectioncited by 0
  6. 2025TrueBrief: Faithful Summarization through Small Language Modelscited by 0
  7. 2026Beyond Caption-Based Queries for Video Moment Retrievalcited by 0
  8. 2026LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucination
  2. 2025MMD-Flagger: Leveraging Maximum Mean Discrepancy to Detect Hallucinations
  3. 2024Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework
  4. 2025(Im)possibility of Automated Hallucination Detection in Large Language Models
  5. 2025Towards Mitigation of Hallucination for LLM-empowered Agents: Progressive Generalization Bound Exploration and Watchdog Monitor
  6. 2026BEACON: Behavioral Entropy Aggregation for Cross-Model Hallucination Detection in Large Language Models
  7. 2026Zero-source LLM Hallucination Detection with Human-like Criteria Probing
  8. 2025C-FAITH: A Chinese Fine-Grained Benchmark for Automated Hallucination Evaluation