mlsanityData to 14 Oct 2026

Understanding and Evaluating Hallucinations in 3D Visual Language Models

arXiv 2502.15888 ↗First posted 18 Feb 2025cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs

Abstract (arXiv)

Recently, 3D-LLMs, which combine point-cloud encoders with large models, have been proposed to tackle complex tasks in embodied intelligence and scene understanding. In addition to showing promising results on 3D tasks, we found that they are significantly affected by hallucinations. For instance, they may generate objects that do not exist in the scene or produce incorrect relationships between objects. To investigate this issue, this work presents the first systematic study of hallucinations in 3D-LLMs. We begin by quickly evaluating hallucinations in several representative 3D-LLMs and reveal that they are all significantly affected by hallucinations. We then define hallucinations in 3D scenes and, through a detailed analysis of datasets, uncover the underlying causes of these hallucinations. We find three main causes: (1) Uneven frequency distribution of objects in the dataset. (2) Strong correlations between objects. (3) Limited diversity in object attributes. Additionally, we propose new evaluation metrics for hallucinations, including Random Point Cloud Pair and Opposite Question Evaluations, to assess whether the model generates responses based on visual information and aligns it with the text's meaning.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 30 references that are in the map. First posted on arXiv 18 Feb 2025.

Builds on

21 of 30 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
  2. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Data22 Nov
  3. 2023CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuning5 Sep
  4. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning26 Jun

Other references · 17

  1. 2023Segment Anythingcited by 8,234
  2. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  3. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  4. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  5. 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 906
  6. 2018Object Hallucination in Image Captioningcited by 669
  7. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  8. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 333
  9. 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 328
  10. 2023ConceptFusion: Open-set Multimodal 3D Mappingcited by 304
  11. 2023LL3DA: Visual Interactive Instruction Tuning for Omni-3D Understanding, Reasoning, and Planningcited by 210
  12. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
5 more
  1. 2020Controlled Hallucinations: Learning to Generate Faithfully from Noisy Datacited by 84
  2. 2023Aligning and Prompting Everything All at Once for Universal Visual Perceptioncited by 60
  3. 2023Forward-Backward Reasoning in Large Language Models for Mathematical Verificationcited by 40
  4. 2024MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D Worldcited by 39
  5. 2020UniConv: A Unified Conversational Neural Architecture for Multi-domain Task-oriented Dialoguescited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 1.

5 papers.

  1. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spacescited by 36
  2. 2025The Point, the Vision and the Text: Does Point Cloud Boost Spatial Reasoning of Large Language Models? A Bias-Controlled Studycited by 5
  3. 2025LEO-VL: Efficient Scene Representation for Scalable 3D Vision-Language Learningcited by 5
  4. 20263D-VCD: Hallucination Mitigation in 3D-LLM Embodied Agents through Visual Contrastive Decodingcited by 3
  5. 2025Context-Aware Sentiment Forecasting via LLM-based Multi-Perspective Role-Playing Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models
  2. 2024From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
  3. 2024A Survey on Hallucination in Large Vision-Language Models
  4. 2024Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models
  5. 2024Multi-Object Hallucination in Vision-Language Models
  6. 2024Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens
  7. 2024AutoHallusion: Automatic Generation of Hallucination Benchmarks for Vision-Language Models
  8. 2025Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similarities