# Two-Token Features and Small-Large Ensembles for VLM Hallucination Detection

arXiv: https://arxiv.org/abs/2609.10244 · HTML: https://mlsanity.com/paper/2609.10244/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 10 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 4 of 4)

- [INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection](https://mlsanity.com/paper/2402.03744/) (2024)
- [PP-OCR: A Practical Ultra Lightweight OCR System](https://mlsanity.com/paper/2009.09941/) (2020)
- [SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakes](https://mlsanity.com/paper/2504.11975/) (2025)
- [Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarking](https://mlsanity.com/paper/2608.01021/) (2026)

## Abstract (arXiv)

We present our system for the SHROOM-Visions 2026 shared task on character-level VLM hallucination detection. A small (4B-parameter) VLM is fine-tuned as a per-token classifier reading a two-token feature from its own hidden states, and is ensembled with a ~400B zero-shot VLM judge at prediction time. Both components see off-the-shelf OCR of any visible in-image text. We use synthetic hallucination data generated by the large model as a source of ensemble diversity, and use validation to select feature layer, training data and OCR grounding. Our official entry reaches mean Cor 0.487 / Cor-lbl 0.387 on the hidden test set, placing 6th/28 (EN), 6th/21 (FR), 8th/21 (IT) and 7th/22 (ZH) on the task's primary Cor-lbl metric.