mlsanityData to 7 Oct 2026

Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding

arXiv 2503.00361 ↗First posted 1 Mar 2025cs.CV cs.AIcited by 23

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Large Vision-Language Models (LVLMs) have obtained impressive performance in visual content understanding and multi-modal reasoning. Unfortunately, these large models suffer from serious hallucination problems and tend to generate fabricated responses. Recently, several Contrastive Decoding (CD) strategies have been proposed to alleviate hallucination by introducing disturbed inputs. Although great progress has been made, these CD strategies mostly apply a one-size-fits-all approach for all input conditions. In this paper, we revisit this process through extensive experiments. Related results show that hallucination causes are hybrid and each generative step faces a unique hallucination challenge. Leveraging these meaningful insights, we introduce a simple yet effective Octopus-like framework that enables the model to adaptively identify hallucination types and create a dynamic CD workflow. Our Octopus framework not only outperforms existing methods across four benchmarks but also demonstrates excellent deployability and expansibility. Code is available at https://github.com/LijunZhang01/Octopus.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 66 references that are in the map. First posted on arXiv 1 Mar 2025.

Builds on

50 of 66 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models25 Aug
  2. 2024Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)6 Aug
  3. 2024Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models28 May
  4. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  5. 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
  6. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  7. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
  8. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  9. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  10. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  11. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  12. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
4 more
  1. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  2. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  3. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  4. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 34

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2017Proximal Policy Optimization Algorithmscited by 14,441
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  8. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  9. 2023Visual Instruction Tuningcited by 7,792
  10. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
22 more
  1. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  2. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  3. 2024SimPO: Simple Preference Optimization with a Reference-Free Rewardcited by 951
  4. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  5. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  6. 2018Object Hallucination in Image Captioningcited by 661
  7. 2018A Comprehensive Survey of Deep Learning for Image Captioningcited by 213
  8. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  9. 2020Inf-Net: Automatic COVID-19 Lung Infection Segmentation from CT Imagescited by 184
  10. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 128
  11. 2023Some things are more CRINGE than others: Iterative Preference Optimization with the Pairwise Cringe Losscited by 108
  12. 2024mDPO: Conditional Preference Optimization for Multimodal Large Language Modelscited by 92
  13. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  14. 2017Near Optimal Behavior via Approximate State Abstractioncited by 63
  15. 2024Current state of LLM Risks and AI Guardrailscited by 44
  16. 2024Surgical-LVLM: Learning to Adapt Large Vision-Language Model for Grounded Visual Question Answering in Robotic Surgerycited by 39
  17. 2023Evaluation and Enhancement of Semantic Grounding in Large Vision-Language Modelscited by 13
  18. 2023S3C: Semi-Supervised VQA Natural Language Explanation via Self-Critical Learningcited by 10
  19. 2024SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Modelscited by 10
  20. 2024On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigationcited by 8
  21. 2024A Plug-and-Play Method for Rare Human-Object Interactions Detection by Bridging Domain Gapcited by 6
  22. 2024SADL: An Effective In-Context Learning Method for Compositional Visual QAcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 14.

23 papers.

Sort:

By area:

  1. 2025Mitigating Hallucination in Large Vision-Language Models via Adaptive Attention Calibrationcited by 10
  2. 2025Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layerscited by 7
  3. 2025Med-VCD: Mitigating Hallucination for Medical Large Vision Language Models through Visual Contrastive Decodingcited by 6
  4. 2025HalDec-Bench: Benchmarking Hallucination Detector in Image Captioningcited by 5
  5. 2025SAVE: Sparse Autoencoder-Driven Visual Information Enhancement for Mitigating Object Hallucinationcited by 3
  6. 2025Empowering Multimodal LLMs with External Tools: A Comprehensive Surveycited by 2
  7. 2025BIMA: Bijective Maximum Likelihood Learning Approach to Hallucination Prediction and Mitigation in Large Vision-Language Modelscited by 1
  8. 2025Mitigating Cross-Image Information Leakage in Multi-Image Understanding with Large Vision-Language Modelscited by 1
  9. 2026Context-Aware Decoding for Faithful Vision-Language Generationcited by 1
  10. 2026MACD: Model-Aware Contrastive Decoding via Counterfactual Datacited by 1
  11. 2026Scaling Test-Time Robustness of Vision-Language Models via Self-Critical Inference Frameworkcited by 1
  12. 2026Instruction-Evidence Contrastive Dual-Stream Decoding for Grounded Vision-Language Reasoningcited by 1
11 more
  1. 2026Instruction Lens Score: Your Instruction Contributes a Powerful Object Hallucination Detector for Multimodal Large Language Modelscited by 1
  2. 2025Watermarking for Factuality: Guiding Vision-Language Models Toward Truth via Tri-layer Contrastive Decodingcited by 0
  3. 2026HalDec-Bench: Benchmarking Hallucination Detector in Image Captioningcited by 0
  4. 2026BUZZY: Contrastive Scoring to Mitigate Text-Induced Bias in Multimodal Multiple-Choice QAcited by 0
  5. 2026Hallucination-aware intermediate representation edit in large vision-language modelscited by 0
  6. 2026HTDC: Hesitation-Triggered Differential Calibration for Mitigating Hallucination in Large Vision-Language Modelscited by 0
  7. 2026Online Self-Calibration Against Hallucination in Vision-Language Modelscited by 0
  8. 2026Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Modelscited by 0
  9. 2026Vision Inference Former: Sustaining Visual Consistency in Multimodal Large Language Modelscited by 0
  10. 2026Zero-MELO: Test-Time Evidence Calibration with Multimodal LLMs for Zero-Shot Micro-Gesture Recognitioncited by 0
  11. 2026Beyond Attention Imbalance: Mitigating Hallucinations via Spectral Surgerycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models
  2. 2025SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding
  3. 2025Retrieval Visual Contrastive Decoding to Mitigate Object Hallucinations in Large Vision-Language Models
  4. 2024Delve into Visual Contrastive Decoding for Hallucination Mitigation of Large Vision-Language Models
  5. 2026SDCD: Structure-Disrupted Contrastive Decoding for Mitigating Hallucinations in Large Vision-Language Models
  6. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  7. 2026CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
  8. 2025Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models -