mlsanityData to 7 Oct 2026

HalluRNN: Mitigating Hallucinations via Recurrent Cross-Layer Reasoning in Large Vision-Language Models

arXiv 2506.17587 ↗First posted 21 Jun 2025cs.CV cs.AI cs.LGcited by 4

On the map: LLM reasoning → Hallucination detection and mitigation

also in 1 more
Abstract (arXiv)

Though Large Vision-Language Models (LVLMs) have achieved remarkable performance across various tasks, they are still prone to hallucinations-generating outputs that are textually plausible but visually ungrounded. While prior approaches generally address this issue through data-centric fine-tuning or innovative decoding strategies, these methods often require substantial resources or task-specific configurations. In this work, we introduce an architecture-level solution, HalluRNN, which enhances model stability through recurrent cross-layer reasoning. Specifically, we propose a novel Dual-Gated Depth Propagation Unit (DG-DPU) module, which is shared across layers and recurrently refines hidden states. This allows for the adaptive propagation of information throughout the model, enforces consistency across layers, and mitigates hallucinations caused by representational drift. By fine-tuning only the DG-DPU module, HalluRNN achieves strong and robust performance across multiple benchmarks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. Builds on 66 of its 84 references that are in the map. First posted on arXiv 21 Jun 2025.

Builds on

66 of 84 references.

References · 66

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2024The Llama 3 Herd of Modelscited by 14,192
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2023Visual Instruction Tuningcited by 7,792
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  11. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  12. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
38 more
  1. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  2. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  3. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2014On the Properties of Neural Machine Translation: Encoder-Decoder Approachescited by 1,303
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  7. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  8. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  9. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 671
  10. 2018Object Hallucination in Image Captioningcited by 661
  11. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 468
  12. 2023RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedbackcited by 404
  13. 2023DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Modelscited by 367
  14. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  15. 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 301
  16. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 254
  17. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 231
  18. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 227
  19. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 223
  20. 2024When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Methodcited by 201
  21. 2024Aligning Modalities in Vision Large Language Models via Preference Fine-tuningcited by 197
  22. 2023LLM Lies: Hallucinations are not Bugs, but Features as Adversarial Examplescited by 177
  23. 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMscited by 174
  24. 2016Analyzing the Behavior of Visual Question Answering Modelscited by 119
  25. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decodingcited by 118
  26. 2019Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editingcited by 108
  27. 2024Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Modelscited by 90
  28. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 87
  29. 2017Comparative Study of CNN and RNN for Natural Language Processingcited by 81
  30. 2024Calibrated Self-Rewarding Vision Language Modelscited by 77
  31. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 77
  32. 2021Let there be a clock on the beach: Reducing Object Hallucination in Image Captioningcited by 74
  33. 2023CIEM: Contrastive Instruction Evaluation Method for Better Instruction Tuningcited by 69
  34. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  35. 2021Advancing RNN Transducer Technology for Speech Recognitioncited by 44
  36. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  37. 2022Two-stage LLM Fine-tuning with Less Specialization and More Generalizationcited by 39
  38. 2019DIANet: Dense-and-Implicit Attention Networkcited by 37

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2026Mitigating Factual Hallucination in Large Reasoning Models via Mixed-Mode Advantage Regularizationcited by 2
  2. 2026TSRouter: Dynamic Modality-Model Selection for Time Series Reasoningcited by 2
  3. 2025MAP: Mitigating Hallucinations in Large Vision-Language Models with Map-Level Attention Processingcited by 1
  4. 2026Self-Correction Inside the Model: Leveraging Layer Attention to Mitigate Hallucinations in Large Vision Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models
  2. 2024Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
  3. 2026Residual Decoding: Mitigating Hallucinations in Large Vision-Language Models via History-Aware Residual Guidance
  4. 2026Context-Aware Decoding for Faithful Vision-Language Generation
  5. 2025VEGAS: Mitigating Hallucinations in Large Vision-Language Models via Vision-Encoder Attention Guided Adaptive Steering
  6. 2024VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding
  7. 2025ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models
  8. 2026SAKED: Mitigating Hallucination in Large Vision-Language Models via Stability-Aware Knowledge Enhanced Decoding