mlsanityData to 7 Oct 2026

Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations

arXiv 2503.14895 ↗First posted 19 Mar 2025cs.CV cs.AI cs.CLcited by 7

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Recently, multimodal large language models (MLLMs) have demonstrated remarkable performance in visual-language tasks. However, the authenticity of the responses generated by MLLMs is often compromised by object hallucinations. We identify that a key cause of these hallucinations is the model's over-susceptibility to specific image frequency features in detecting objects. In this paper, we introduce Multi-Frequency Perturbations (MFP), a simple, cost-effective, and pluggable method that leverages both low-frequency and high-frequency features of images to perturb visual feature representations and explicitly suppress redundant frequency-domain features during inference, thereby mitigating hallucinations. Experimental results demonstrate that our method significantly mitigates object hallucinations across various model architectures. Furthermore, as a training-time method, MFP can be combined with inference-time methods to achieve state-of-the-art performance on the CHAIR benchmark.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 62 of its 82 references that are in the map. First posted on arXiv 19 Mar 2025.

Builds on

62 of 82 references.

Closest ancestors · 20

Same direction, newest first

  1. 2024Mitigating Object Hallucination via Concentric Causal Attention21 Oct
  2. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models4 Aug
  3. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
  4. 2024DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer21 Jul
  5. 2024Hallucination of Multimodal Large Language Models: A Survey29 Apr
  6. 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
  7. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding1 Mar
  8. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding28 Feb
  9. 2024Skip n: A Simple Method to Reduce Hallucination in Large Vision-Language Models2 Feb
  10. 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
  11. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback1 Dec
  12. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
8 more
  1. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  2. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
  3. 2023Mitigating Hallucination in Visual Language Models with Visual Supervision27 Nov
  4. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  5. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision13 Nov
  6. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
  7. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
  8. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 42

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2024GPT-4o System Cardcited by 4,467
  9. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  12. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
30 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  2. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  3. 2015VQA: Visual Question Answeringcited by 2,489
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  5. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  6. 2019Towards VQA Models That Can Readcited by 1,696
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  8. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  10. 2023SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehensioncited by 929
  11. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  12. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  13. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  14. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  15. 2018Object Hallucination in Image Captioningcited by 661
  16. 2016Visual Dialogcited by 377
  17. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  18. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  19. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  20. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 275
  21. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  22. 2020Learning in the Frequency Domaincited by 196
  23. 2022Frequency Domain Model Augmentation for Adversarial Attackcited by 128
  24. 2021FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysiscited by 60
  25. 2023Frequency Perception Network for Camouflaged Object Detectioncited by 37
  26. 2024MouSi: Poly-Visual-Expert Vision-Language Modelscited by 24
  27. 2021Backdoor Attack through Frequency Domaincited by 18
  28. 2023Mitigating Open-Vocabulary Caption Hallucinationscited by 18
  29. 2024HiPrompt: Tuning-free Higher-Resolution Generation with Hierarchical MLLM Promptscited by 14
  30. 2024Hallucination in Multimodal Foundation Models: A Survey on Causes, Corrections, and Evaluationscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 6.

7 papers.

  1. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Modelscited by 7
  2. 2026Decoding by Perturbation: Mitigating MLLM Hallucinations via Dynamic Textual Perturbationcited by 3
  3. 2026MM-THEBench: Do Reasoning MLLMs Think Reasonably?cited by 1
  4. 2026Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Rewardcited by 1
  5. 2026Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigationcited by 1
  6. 2026DO-Bench: An Attributable Benchmark for Diagnosing Object Hallucination in Vision-Language Modelscited by 0
  7. 2026The Safeguard Worked. Is the LLM System Safer?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  2. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models
  3. 2024Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention
  4. 2024NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models
  5. 2026NoLan: Mitigating Object Hallucinations in Large Vision-Language Models via Dynamic Suppression of Language Priors
  6. 2026Mitigating Object Hallucinations in Vision-Language Models through Region-Aware Attention Recalibration
  7. 2026Test-Time Hallucination Control in Large Vision-Language Models
  8. 2025HalluShift++: Bridging Language and Vision through Internal Representation Shifts for Hierarchical Hallucinations in MLLMs