# CORTEX: Token-Level Hallucination Detection in RAG via Comparative Internal Representations

arXiv: https://arxiv.org/abs/2606.31033 · HTML: https://mlsanity.com/paper/2606.31033/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Hallucination detection in RAG, one of 15 directions in Retrieval-augmented generation.
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 30 Jun 2026.

## Placed in

- Retrieval-augmented generation / [Hallucination detection in RAG](https://mlsanity.com/rag/hallucination-detection-in-rag/)

## Builds on (most cited first, 12 of 12)

- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models](https://mlsanity.com/paper/2401.00396/) (2023)
- [HalluLens: LLM Hallucination Benchmark](https://mlsanity.com/paper/2504.17550/) (2025)
- [ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMs](https://mlsanity.com/paper/2507.16488/) (2025)
- [Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Models](https://mlsanity.com/paper/2501.09997/) (2025)
- [Zero-shot Persuasive Chatbots with LLM-Generated Strategies and Information Retrieval](https://mlsanity.com/paper/2407.03585/) (2024)
- [The HalluRAG Dataset: Detecting Closed-Domain Hallucinations in RAG Applications Using an LLM's Internal States](https://mlsanity.com/paper/2412.17056/) (2024)
- [HalluHard: A Hard Multi-Turn Hallucination Benchmark](https://mlsanity.com/paper/2602.01031/) (2026)
- [Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicity](https://mlsanity.com/paper/2602.00723/) (2026)

## Abstract (arXiv)

In this paper, we propose CORTEX, a token-level hallucination detection method for Retrieval-Augmented Generation (RAG). In long-form RAG outputs, hallucinations often arise in localized spans rather than throughout an entire response. CORTEX therefore identifies ungrounded content at the token level, enabling fine-grained localization of hallucinations. The key intuition behind CORTEX is that tokens grounded in retrieved documents should be more strongly influenced by those documents than hallucinated tokens. To capture this document-induced effect, CORTEX compares internal representations of a large language model (LLM) under two conditions: with and without the retrieved documents. Instead of relying solely on each token's immediate sensitivity to the retrieved documents, CORTEX also leverages the propagation of document-grounded information through preceding tokens, reducing false positives for tokens whose evidence has already been absorbed into the context. Finally, CORTEX applies post-processing smoothing step that models the tendency of hallucination labels to persist over contiguous spans, reducing local noise and encouraging span-consistent predictions. Experiments on two RAG benchmarks and three LLMs show that CORTEX substantially improves token-level hallucination detection, with each component consistently contributing to performance gains.