mlsanityData to 14 Oct 2026

Overview of SHROOM-Visions 2026: A Shared Task on Hallucination Detection in Large Vision-Language Models

arXiv 2608.25662 ↗First posted 26 Aug 2026cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

In 2026, we held the fourth iteration of the SHROOM Shared Task series: SHROOM-Visions (Shared-task on Hallucinations and Related Observable Overgeneration Mistakes in Vision language models), which is hosted at the UncertaiNLP Workshop co-located with EMNLP 2026. Following the success of the 2024 and 2025 tasks, this time we aim to tackle hallucinations through a model-agnostic detection task focused on large vision-language models. Building on the recently introduced SHEEP dataset, designed for long-term evaluation across model generations, the task invites participants to detect and classify fine-grained hallucination spans in image-conditioned text generation (VQA, image captioning, etc.). The evaluation uses a five-class taxonomy of hallucinations spanning four languages: Chinese, English, French, and Italian. The shared task generated strong interest in the NLP community worldwide, with 27 teams contributing 600+ system submissions. The best systems achieve average scores of 0.58 in character-level correlation, 0.46 in label-conditioned correlation, and 0.51 in intersection-over-union (IoU) across four languages, outperforming the baselines by 30-40 points.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 18 of its 45 references that are in the map. First posted on arXiv 26 Aug 2026.

Builds on

18 of 45 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs8 Dec
  2. 2025HalDec-Bench: Benchmarking Hallucination Detector in Image Captioning25 Nov
  3. 2024Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs2 Aug
  4. 2024HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoning22 Jul
  5. 2024AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models16 Jun
  6. 2024Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models24 Feb
  7. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct

Other references · 11

  1. 2023A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasetscited by 123
  2. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 105
  3. 2017FOIL it! Find One mismatch between Image and Language captioncited by 95
  4. 2024ALOHa: A New Measure for Hallucination in Captioning Modelscited by 45
  5. 2025HALoGEN: Fantastic LLM Hallucinations and Where to Find Themcited by 34
  6. 2025SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakescited by 27
  7. 2024SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakescited by 24
  8. 2024Visual Hallucination: Definition, Quantification, and Prescriptive Remediationscited by 11
  9. 2025VISaGE: Understanding Visual Generics and Exceptionscited by 4
  10. 2026Can Humans Dream of Electric Sheep? Human-Written Samples for Fine-Grained Vision-and-Language Hallucination Benchmarkingcited by 4
  11. 2026DSC2025 -- ViHallu Challenge: Detecting Hallucination in Vietnamese LLMscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SKstars at SHROOM: Visions Agreement-Guided Ensembling of Zero-Shot and LoRA-Adapted Vision--Language Models
  2. 2026SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models
  3. 2024FIHA: Autonomous Hallucination Evaluation in Vision-Language Models with Davidson Scene Graphs
  4. 2026HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models
  5. 2024Hallucination of Multimodal Large Language Models: A Survey
  6. 2025ZINA: Multimodal Fine-grained Hallucination Detection and Editing
  7. 2023Detecting and Preventing Hallucinations in Large Vision Language Models
  8. 2024Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback