mlsanityData to 7 Oct 2026

CutPaste&Find: Efficient Multimodal Hallucination Detector with Visual-aid Knowledge Base

arXiv 2502.12591 ↗First posted 18 Feb 2025cs.CV cs.CLcited by 4

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Large Vision-Language Models (LVLMs) have demonstrated impressive multimodal reasoning capabilities, but they remain susceptible to hallucination, particularly object hallucination where non-existent objects or incorrect attributes are fabricated in generated descriptions. Existing detection methods achieve strong performance but rely heavily on expensive API calls and iterative LVLM-based validation, making them impractical for large-scale or offline use. To address these limitations, we propose CutPaste&Find, a lightweight and training-free framework for detecting hallucinations in LVLM-generated outputs. Our approach leverages off-the-shelf visual and linguistic modules to perform multi-step verification efficiently without requiring LVLM inference. At the core of our framework is a Visual-aid Knowledge Base that encodes rich entity-attribute relationships and associated image representations. We introduce a scaling factor to refine similarity scores, mitigating the issue of suboptimal alignment values even for ground-truth image-text pairs. Comprehensive evaluations on benchmark datasets, including POPE and R-Bench, demonstrate that CutPaste&Find achieves competitive hallucination detection performance while being significantly more efficient and cost-effective than previous methods.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 34 of its 44 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

34 of 44 references.

References · 34

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  7. 2017Billion-scale similarity search with GPUscited by 2,477
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  9. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  10. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  11. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  12. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
22 more
  1. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  2. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 439
  3. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revisioncited by 96
  4. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  5. 2024Unified Hallucination Detection for Multimodal Large Language Modelscited by 75
  6. 2023FACTUAL: A Benchmark for Faithful and Consistent Textual Scene Graph Parsingcited by 50
  7. 2021Contrastive Learning for Neural Topic Modelcited by 48
  8. 2023Effective Neural Topic Modeling with Embedding Clustering Regularizationcited by 46
  9. 2024AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledgecited by 42
  10. 2024AKEW: Assessing Knowledge Editing in the Wildcited by 35
  11. 2024FASTopic: Pretrained Transformer is a Fast, Adaptive, Stable, and Transferable Topic Modelcited by 34
  12. 2023Zero-shot Faithful Factual Error Correctioncited by 26
  13. 2023InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modelingcited by 24
  14. 2021Enriching and Controlling Global Semantics for Text Summarizationcited by 20
  15. 2024On the Affinity, Rationality, and Diversity of Hierarchical Topic Modelingcited by 18
  16. 2024Modeling Dynamic Topics in Chain-Free Fashion by Evolution-Tracking Contrastive Learning and Unassociated Word Exclusioncited by 16
  17. 2024KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learningcited by 13
  18. 2024Topic Modeling as Multi-Objective Contrastive Optimizationcited by 12
  19. 2023DemaFormer: Damped Exponential Moving Average Transformer with Energy-Based Modeling for Temporal Language Groundingcited by 11
  20. 2024Are LLMs Good Zero-Shot Fallacy Classifiers?cited by 10
  21. 2023Expand BERT Representation with Visual Information via Grounded Language Learning with Multimodal Partial Alignmentcited by 5
  22. 2024Topic-aware Causal Intervention for Counterfactual Detectioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  2. 2025Journey Before Destination: On the importance of Visual Faithfulness in Slow Thinkingcited by 5
  3. 2025More Bias, Less Bias: BiasPrompting for Enhanced Multiple-Choice Question Answeringcited by 1
  4. 2026MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrievalcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Detecting and Preventing Hallucinations in Large Vision Language Models
  2. 2026Beyond the Global Scores: Fine-Grained Token Grounding as a Robust Detector of LVLM Hallucinations
  3. 2025Visual hallucination detection in large vision-language models via evidential conflict
  4. 2026VisER: Visual Evidence and Reliance for Object Hallucination Detection in LVLMs
  5. 2026HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models
  6. 2026TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMs
  7. 2025Seeing is Believing: Rich-Context Hallucination Detection for MLLMs via Backward Visual Grounding
  8. 2024Beyond Logit Lens: Contextual Embeddings for Robust Hallucination Detection & Grounding in VLMs