mlsanityData to 7 Oct 2026

FREAK: A Fine-grained Hallucination Evaluation Benchmark for Advanced MLLMs

arXiv 2603.19765 ↗First posted 20 Mar 2026cs.CVcited by 4

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) suffer from hallucinations. Existing hallucination evaluation benchmarks are often limited by over-simplified tasks leading to saturated metrics, or insufficient diversity that fails to adequately assess the hallucination extent in state-of-the-art multimodal models. To address this gap, we propose FREAK, a comprehensive multimodal benchmark designed for fine-grained hallucination assessment in MLLMs. Through high-quality photorealistic images featuring fine-grained counter-commonsense edits, FREAK innovatively evaluates hallucination phenomena in detailed visual perception of MLLMs. Extensive experiments on FREAK show severe hallucination issues in SOTA models regarding detailed visual perception. To enable deeper investigation, we curate a controlled subset to indirectly evaluate the model's ability to perceive target detailed information. Through systematic evaluation of prevailing Chain-of-Thought (CoT) prompting techniques within this task, we reveal critical insights regarding hallucination patterns and model reasoning processes.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 42 references that are in the map. First posted on arXiv 20 Mar 2026.

Builds on

36 of 42 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM30 May
  2. 2025PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training9 Mar
  3. 2024Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models30 Oct
  4. 2024Reducing Hallucinations in Vision-Language Models via Latent Space Steering21 Oct
  5. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation15 Oct
  6. 2024Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models4 Oct
  7. 2024ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models14 Sep
  8. 2024Multi-Object Hallucination in Vision-Language Models8 Jul
  9. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  10. 2024PhD: A ChatGPT-Prompted Visual hallucination Evaluation Dataset17 Mar
  11. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  12. 2024Hal-Eval: A Universal and Fine-grained Hallucination Evaluation Framework for Large Vision Language Models24 Feb
11 more
  1. 2024Visual Hallucinations of Multi-modal Large Language Models22 Feb
  2. 2024Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models18 Feb
  3. 2024Unified Hallucination Detection for Multimodal Large Language Models5 Feb
  4. 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
  5. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  6. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  7. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  8. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
  9. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct
  10. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  11. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 13

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  6. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  7. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Modelcited by 515
  8. 2022Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Taskscited by 510
  9. 2021Unifying Vision-and-Language Tasks via Text Generationcited by 411
  10. 2024Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Toolscited by 170
  11. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 93
  12. 2023Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Imagescited by 79
1 more
  1. 2024VLind-Bench: Measuring Language Priors in Large Vision-Language Modelscited by 44

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Evidence-RL: Towards Evidence-intensive Visual Reasoningcited by 3
  2. 2026ReactBench: A Cause-Driven Benchmark for Multimodal Hallucination via Systematic Evaluationcited by 1
  3. 2026KnowHal: A Knowledge-Driven Benchmark for Comprehensive Multimodal Hallucination Evaluationcited by 1
  4. 2026Reliability-Prioritized Fine-Grained Generation in Multimodal Largecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluations
  2. 2025A Survey of Multimodal Hallucination Evaluation and Detection
  3. 2025MIHBench: Benchmarking and Mitigating Multi-Image Hallucinations in Multimodal Large Language Models
  4. 2025GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs
  5. 2025Preemptive Hallucination Reduction: An Input-Level Approach for Multimodal Language Model
  6. 2026HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models
  7. 2024Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
  8. 2025Mitigating Behavioral Hallucination in Multimodal Large Language Models for Sequential Images