mlsanityData to 7 Oct 2026

Mitigating Hallucinations on Object Attributes using Multiview Images and Negative Instructions

arXiv 2501.10011 ↗First posted 17 Jan 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Current popular Large Vision-Language Models (LVLMs) are suffering from Hallucinations on Object Attributes (HoOA), leading to incorrect determination of fine-grained attributes in the input images. Leveraging significant advancements in 3D generation from a single image, this paper proposes a novel method to mitigate HoOA in LVLMs. This method utilizes multiview images sampled from generated 3D representations as visual prompts for LVLMs, thereby providing more visual information from other viewpoints. Furthermore, we observe the input order of multiple multiview images significantly affects the performance of LVLMs. Consequently, we have devised Multiview Image Augmented VLM (MIAVLM), incorporating a Multiview Attributes Perceiver (MAP) submodule capable of simultaneously eliminating the influence of input image order and aligning visual information from multiview images with Large Language Models (LLMs). Besides, we designed and employed negative instructions to mitigate LVLMs' bias towards ``Yes" responses. Comprehensive experiments demonstrate the effectiveness of our method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 38 references that are in the map. First posted on arXiv 17 Jan 2025.

Builds on

34 of 38 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024A Survey on Hallucination in Large Vision-Language Models1 Feb
  2. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation29 Nov
  3. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation13 Nov
  4. 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct
  5. 2023CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning5 Sep
  6. 2023Detecting and Preventing Hallucinations in Large Vision Language Models11 Aug
  7. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun
  8. 2023Evaluating Object Hallucination in Large Vision-Language Models17 May

Other references · 26

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2023Segment Anythingcited by 8,158
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,122
  6. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  7. 2024Yi: Open Foundation Models by 01.AIcited by 706
  8. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 703
  9. 2023OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Modelscited by 560
  10. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  11. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 498
  12. 2023A Survey of Hallucination in Large Foundation Modelscited by 365
14 more
  1. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusioncited by 364
  2. 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 341
  3. 2024What matters when building vision-language models?cited by 321
  4. 2024LLaVA-UHD: an LMM Perceiving Any Aspect Ratio and High-Resolution Imagescited by 193
  5. 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
  6. 2024xGen-MM (BLIP-3): A Family of Open Large Multimodal Modelscited by 116
  7. 2022High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimizationcited by 65
  8. 2023Large Language Models are Visual Reasoning Coordinatorscited by 58
  9. 2023Hierarchical Open-vocabulary Universal Image Segmentationcited by 53
  10. 2022AnyFace: Free-style Text-to-Face Synthesis and Manipulationcited by 33
  11. 2023Can SAM Boost Video Super-Resolution?cited by 21
  12. 2024Order Matters: Exploring Order Sensitivity in Multimodal Large Language Modelscited by 19
  13. 2024Representing Online Handwriting for Recognition in Large Vision-Language Modelscited by 7
  14. 2023Building a Modal-balanced BlockChain with Semantic Reconstructioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  2. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models
  3. 2024VORD: Visual Ordinal Calibration for Mitigating Object Hallucinations in Large Vision-Language Models
  4. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models
  5. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding
  6. 2024VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models
  7. 2025Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models
  8. 2024Mitigating Hallucinations in Large Vision-Language Models (LVLMs) via Language-Contrastive Decoding (LCD)