mlsanityData to 7 Oct 2026

Bi'an: A Bilingual Benchmark and Model for Hallucination Detection in Retrieval-Augmented Generation

arXiv 2502.19209 ↗First posted 26 Feb 2025cs.CLcited by 1

On the map: Retrieval-augmented generation → Hallucination detection in RAG

Abstract (arXiv)

Retrieval-Augmented Generation (RAG) effectively reduces hallucinations in Large Language Models (LLMs) but can still produce inconsistent or unsupported content. Although LLM-as-a-Judge is widely used for RAG hallucination detection due to its implementation simplicity, it faces two main challenges: the absence of comprehensive evaluation benchmarks and the lack of domain-optimized judge models. To bridge these gaps, we introduce Bi'an, a novel framework featuring a bilingual benchmark dataset and lightweight judge models. The dataset supports rigorous evaluation across multiple RAG scenarios, while the judge models are fine-tuned from compact open-source LLMs. Extensive experimental evaluations on Bi'anBench show our 14B model outperforms baseline models with over five times larger parameter scales and rivals state-of-the-art closed-source LLMs. We will release our data and models soon at https://github.com/OpenSPG/KAG.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 24 of its 39 references that are in the map. First posted on arXiv 26 Feb 2025.

Builds on

24 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Lynx: An Open Source Hallucination Evaluation Model11 Jul
  2. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models31 Dec

Other references · 22

  1. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,442
  5. 2021Improving language models by retrieving from trillions of tokenscited by 1,278
  6. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,271
  7. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  8. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  9. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 349
  10. 2023Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflictscited by 333
  11. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 261
  12. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
10 more
  1. 2022ASQA: Factoid Questions Meet Long-Form Answerscited by 231
  2. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  3. 2022On the Origin of Hallucinations in Conversational Models: Is it the Datasets or the Models?cited by 137
  4. 2024TableBench: A Comprehensive and Complex Benchmark for Table Question Answeringcited by 127
  5. 2023WebCPM: Interactive Web Search for Chinese Long-form Question Answeringcited by 93
  6. 2022Evaluating the Factual Consistency of Large Language Models Through News Summarizationcited by 90
  7. 2016Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answeringcited by 35
  8. 2021CSDS: A Fine-Grained Chinese Dataset for Customer Service Dialogue Summarizationcited by 27
  9. 2024FoRAG: Factuality-optimized Retrieval Augmented Generation for Web-enhanced Long-form Question Answeringcited by 18
  10. 2024Efficient Knowledge Infusion via KG-LLM Alignmentcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026CuraView: A Multi-Agent Framework for Medical Hallucination Detection with GraphRAG-Enhanced Knowledge Verificationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Osiris: A Lightweight Open-Source Hallucination Detection System
  2. 2025LUMINA: Detecting Hallucinations in RAG System with Context-Knowledge Signals
  3. 2026RAGognizer: Hallucination-Aware Fine-Tuning via Detection Head Integration
  4. 2025Real-Time Evaluation Models for RAG: Who Detects Hallucinations Best?
  5. 2025MetaRAG: Metamorphic Testing for Hallucination Detection in RAG Systems
  6. 2025Probabilistic distances-based hallucination detection in LLMs with RAG
  7. 2026Mitigating Hallucination on Hallucination in RAG via Ensemble Voting
  8. 2026Beyond Document Grounding: Span-Level Hallucination Detection over Code, Tool Output, and Documents