# Pelican: Correcting Hallucination in Vision-LLMs via Claim Decomposition and Program of Thought Verification

arXiv: https://arxiv.org/abs/2407.02352 · HTML: https://mlsanity.com/paper/2407.02352/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning.
- Builds on 25 of its 39 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Placed in

- LLM reasoning / [Hallucination detection and mitigation](https://mlsanity.com/lr/hallucination-detection-and-mitigation/)
- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 25 of 25)

- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Evaluating Object Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2305.10355/) (2023)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Aligning Large Multimodal Models with Factually Augmented RLHF](https://mlsanity.com/paper/2309.14525/) (2023)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback](https://mlsanity.com/paper/2312.00849/) (2023)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [Woodpecker: Hallucination Correction for Multimodal Large Language Models](https://mlsanity.com/paper/2310.16045/) (2023)
- [DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback](https://mlsanity.com/paper/2311.10081/) (2023)
- [Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision](https://mlsanity.com/paper/2311.07362/) (2023)
- [Explainable Automated Fact-Checking: A Survey](https://mlsanity.com/paper/2011.03870/) (2020)
- [CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning](https://mlsanity.com/paper/2309.02301/) (2023)
- [Language Models as Fact Checkers?](https://mlsanity.com/paper/2006.04102/) (2020)
- [Explainable Claim Verification via Knowledge-Grounded Reasoning with Large Language Models](https://mlsanity.com/paper/2310.05253/) (2023)
- [Self-Checker: Plug-and-Play Modules for Fact-Checking with Large Language Models](https://mlsanity.com/paper/2305.14623/) (2023)
- [FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback](https://mlsanity.com/paper/2404.05046/) (2024)
- [GraphReason: Enhancing Reasoning Capabilities of Large Language Models through A Graph-Based Verification Approach](https://mlsanity.com/paper/2308.09267/) (2023)

## Built on by (most cited first, 17 of 17)

- [Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding](https://mlsanity.com/paper/2505.03788/) (2025)
- [UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Models](https://mlsanity.com/paper/2503.20504/) (2025)
- [InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration](https://mlsanity.com/paper/2512.02981/) (2025)
- [mRAG: Elucidating the Design Space of Multi-modal Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.24073/) (2025)
- [Taming Object Hallucinations with Verified Atomic Confidence Estimation](https://mlsanity.com/paper/2511.09228/) (2025)
- [Pi-SQL: Enhancing Text-to-SQL with Fine-Grained Guidance from Pivot Programming Languages](https://mlsanity.com/paper/2506.00912/) (2025)
- [Grounding or Guessing? Visual Signals for Detecting Hallucinations in Sign Language Translation](https://mlsanity.com/paper/2510.18439/) (2025)
- [Empowering Multimodal LLMs with External Tools: A Comprehensive Survey](https://mlsanity.com/paper/2508.10955/) (2025)
- [V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering](https://mlsanity.com/paper/2601.18240/) (2026)
- [Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects](https://mlsanity.com/paper/2505.05318/) (2025)
- [Curing Semantic Drift: A Dynamic Approach to Grounding Generation in Large Vision-Language Models](https://mlsanity.com/paper/2506.21509/) (2025)
- [Aligning with Your Own Voice: Self-Corrected Preference Learning for Hallucination Mitigation in LVLMs](https://mlsanity.com/paper/2604.24395/) (2026)
- [Trust in Vision-Language Models: Insights from a Participatory User Workshop](https://mlsanity.com/paper/2511.13458/) (2025)
- [When Verification Fails: How Compositionally Infeasible Claims Escape Rejection](https://mlsanity.com/paper/2604.10990/) (2026)
- [HalluScope: Fine-grained Hallucination Diagnosis for Multimodal Large Language Models](https://mlsanity.com/paper/2607.21105/) (2026)
- [RECAP: Relation Evidence Calibration for Detecting Spatial Relation Hallucinations in Vision-Language Models](https://mlsanity.com/paper/2609.26093/) (2026)
- [PROVE: Proof-guided Regime-aware Operator Verification for Hallucination Detection in Medical Visual Question Answering](https://mlsanity.com/paper/2609.29604/) (2026)

## Abstract (arXiv)

Large Visual Language Models (LVLMs) struggle with hallucinations in visual instruction following task(s), limiting their trustworthiness and real-world applicability. We propose Pelican -- a novel framework designed to detect and mitigate hallucinations through claim verification. Pelican first decomposes the visual claim into a chain of sub-claims based on first-order predicates. These sub-claims consist of (predicate, question) pairs and can be conceptualized as nodes of a computational graph. We then use Program-of-Thought prompting to generate Python code for answering these questions through flexible composition of external tools. Pelican improves over prior work by introducing (1) intermediate variables for precise grounding of object instances, and (2) shared computation for answering the sub-question to enable adaptive corrections and inconsistency identification. We finally use reasoning abilities of LLMs to verify the correctness of the claim by considering the consistency and confidence of the (question, answer) pairs from each sub-claim. Our experiments reveal a drop in hallucination rate by ~ 8% - 32% across various baseline LVLMs and a 27% drop compared to approaches proposed for hallucination mitigation on MMHal-Bench. Results on two other benchmarks further corroborate our results.