mlsanityData to 14 Oct 2026

SIRIN: A Unified Toolkit for Detecting Contextual Hallucinations in Retrieval-Augmented and Memory-Grounded LLM Systems

arXiv 2608.00033 ↗First posted 20 Jul 2026cs.AIcited by 0

On the map: Retrieval-augmented generation → Hallucination detection in RAG

also in 1 more
Abstract (arXiv)

SIRIN (Semantic Inconsistency Recognition and Inspection Nexus) is a unified toolkit and interactive web UI for detecting contextual hallucinations (fluent, plausible responses unsupported by the provided evidence) in retrieval-augmented, agentic, and memory-grounded LLM systems. SIRIN unifies three detector paradigms (representation probing, uncertainty estimation, and judge-style verification) and the complementary task of pre-generation query answerability under one interface, configuration system, and evaluation pipeline, supporting response- and span-level inspection in both white-box and black-box settings. The web UI enables live analysis of user-supplied context-query-answer triples through hallucination scores, unsupported-span highlighting, and side-by-side detector comparison, with a lightweight plug-in design for adding new detectors. We demonstrate SIRIN on hallucination detection, query answerability, and as a faithfulness gate within long-term memory systems. The source code is publicly available at https://github.com/sb-ai-lab/SIRIN.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation. Builds on 21 of its 32 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

21 of 32 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Data-efficient Meta-models for Evaluation of Context-based Questions and Answers in LLMs29 May
  2. 2025Benchmarking LLM Faithfulness in RAG with Evolving Leaderboards7 May
  3. 2025LettuceDetect: A Hallucination Detection Framework for RAG Applications24 Feb
  4. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models31 Dec

Other references · 17

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  4. 2019Designing and Interpreting Probes with Control Taskscited by 640
  5. 2017A Structured Self-attentive Sentence Embeddingcited by 318
  6. 2023AlignScore: Evaluating Factual Consistency with a Unified Alignment Functioncited by 288
  7. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  8. 2020Unsupervised Quality Estimation for Neural Machine Translationcited by 211
  9. 2025LightMem: Lightweight and Efficient Memory-Augmented Generationcited by 180
  10. 2026SimpleMem: Efficient Lifelong Memory for LLM Agentscited by 149
  11. 2023LM-Polygraph: Uncertainty Estimation for Language Modelscited by 131
  12. 2024LLM Internal States Reveal Hallucination Risk Faced With a Querycited by 82
5 more
  1. 2023Do Androids Know They're Only Dreaming of Electric Sheep?cited by 67
  2. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 19
  3. 2025Efficient Hallucination Detection for LLMs Using Uncertainty-Aware Attention Headscited by 16
  4. 2025When Models Lie, We Learn: Multilingual Span-Level Hallucination Detection with PsiloQAcited by 14
  5. 2025HalluEntity: Benchmarking and Understanding Entity-Level Hallucination Detectioncited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models
  2. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
  3. 2025SelfCheck-Eval: A Multi-Module Framework for Zero-Resource Hallucination Detection in Large Language Models
  4. 2025InterpDetect: Interpretable Signals for Detecting Hallucinations in Retrieval-Augmented Generation
  5. 2026ToolHACE: A Dataset for Detecting Hallucinations in Tool-Based LLM Responses
  6. 2026Detecting Hallucinations in Retrieval-Augmented Generation via Semantic-level Internal Reasoning Graph
  7. 2025ORION Grounded in Context: Retrieval-Based Method for Hallucination Detection
  8. 2026Logical Consistency as a Bridge: Improving LLM Hallucination Detection via Label Constraint Modeling between Responses and Self-Judgments