mlsanityData to 14 Oct 2026

Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generation

arXiv 2606.00232 ↗First posted 29 May 2026cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

We study fact-level repair for multimodal generation, where a fluent output may contain specific facts that are not supported by the input. Existing inference-time repair methods often generate feedback by jointly conditioning on the input and the current output. This design has two limitations: hallucinated claims in the output can bias the model's interpretation of the input, and free-form feedback cannot be ranked or scheduled at the fact level. We present TIGER, an inference-time framework that redesigns feedback for localized repair. TIGER independently extracts an observation graph from the input and a claim graph from the current output, then assigns each claim a graph-conditioned risk score based on support and conflict. The model repairs selected high-risk claims while keeping the backbone frozen. We provide a convergence analysis showing that the expected total risk decreases geometrically to an explicit asymptotic bound under mild assumptions. Experiments across four cross-modal paths, including image-to-text, image+text-to-text, audio-to-text, and video-to-text, show that TIGER reduces unsupported content while preserving task quality. The gains hold across multiple backbones, and a CrisisFACTS case study suggests that the same repair mechanism can improve grounding in multi-source settings.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 38 of its 51 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

38 of 51 references.

References · 38

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  4. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  5. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  6. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  7. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  8. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  9. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  10. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  11. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  12. 2019Clotho: An Audio Captioning Datasetcited by 358
26 more
  1. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  2. 2023AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluationcited by 300
  3. 2024AnyGPT: Unified Multimodal LLM with Discrete Sequence Modelingcited by 217
  4. 2025VisualPRM: An Effective Process Reward Model for Multimodal Reasoningcited by 139
  5. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revisioncited by 98
  6. 2024VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Modelscited by 90
  7. 2024EventHallusion: Diagnosing Event Hallucinations in Video LLMscited by 57
  8. 2025Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Modelscited by 44
  9. 2025Be Friendly, Not Friends: How LLM Sycophancy Shapes User Trustcited by 34
  10. 2024Knowledge Graph-Enhanced Large Language Models via Path Selectioncited by 28
  11. 2026Look Inward to Explore Outward: Learning Temperature Policy from LLM Internal States via Hierarchical RLcited by 27
  12. 2025Dropping Experts, Recombining Neurons: Retraining-Free Pruning for Sparse Mixture-of-Experts LLMscited by 25
  13. 2025Reducing Object Hallucination in Large Audio-Language Models via Audio-Aware Decodingcited by 24
  14. 2025Cycle Consistency as Reward: Learning Image-Text Alignment without Human Preferencescited by 22
  15. 2026Learning to Self-Verify Makes Language Models Better Reasonerscited by 21
  16. 2024HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipationcited by 19
  17. 2024Graph Canvas for Controllable 3D Scene Generationcited by 11
  18. 2025Attention Consistency for LLMs Explanationcited by 11
  19. 2025Evaluating Hallucination in Large Vision-Language Models based on Context-Aware Object Similaritiescited by 7
  20. 2025Same Content, Different Answers: Cross-Modal Inconsistency in MLLMscited by 7
  21. 2024What Makes a Scene ? Scene Graph-based Evaluation and Feedback for Controllable Generationcited by 6
  22. 2026One Refiner to Unlock Them All: Inference-Time Reasoning Elicitation via Reinforcement Query Refinementcited by 6
  23. 2024LLMs as Probabilistic Minimally Adequate Teachers for DFA Learningcited by 3
  24. 2024Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Trainingcited by 3
  25. 2026Provable and Practical In-Context Policy Optimization for Self-Improvementcited by 2
  26. 2026LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillationcited by 1

Built on by

Not cited yet.