mlsanityData to 14 Oct 2026

PostAlign: Multimodal Grounding as a Corrective Lens for MLLMs

arXiv 2506.17901 ↗First posted 22 Jun 2025cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Multimodal Large Language Models (MLLMs) excel in vision-language tasks, such as image captioning and visual question answering. However, they often suffer from over-reliance on spurious correlations, primarily due to linguistic priors that distract the model from leveraging actual visual information. To address these issues, we introduce MMGrounded-PostAlign, a post-multimodal alignment framework designed to enhance the visual understanding capabilities and mitigate the hallucinations of MLLMs. Our framework incorporates a multimodal grounding module for both visual grounding, which identifies the referred object in the image, and textual grounding, which generates the rationale for the final answer, ensuring that outputs are anchored in both visual and textual evidence. To mitigate the hallucinations, we introduce a negative rejection mechanism in the visual grounding module to distinguish grounded entities from non-existent objects influenced by linguistic biases. On the textual grounding side, we propose a selective reasoning mechanism that adjusts the model's reasoning strategy based on query complexity. Extensive evaluations are conducted on benchmarks such as POPE, HaloQuest, VQAv2, MME, and MMBench showing significant improvements in fine-grained visual understanding and hallucination suppression.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 62 of its 72 references that are in the map. First posted on arXiv 22 Jun 2025.

Builds on

62 of 72 references.

Closest ancestors · 9

Same direction, newest first

  1. 2024Empowering Segmentation Ability to Multi-modal Large Language Models21 Mar
  2. 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentation26 Feb
  3. 2023GSVA: Generalized Segmentation via Multimodal Large Language Models15 Dec
  4. 2023LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Models5 Dec
  5. 2023PixelLM: Pixel Reasoning with Large Multimodal Model4 Dec
  6. 2023GLaMM: Pixel Grounding Large Multimodal Model6 Nov
  7. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularity11 Oct
  8. 2023LISA: Reasoning Segmentation via Large Language Model1 Aug
  9. 2023Kosmos-2: Grounding Multimodal Large Language Models to the World26 Jun

Other references · 53

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Segment Anythingcited by 8,234
  4. 2023Visual Instruction Tuningcited by 7,869
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  7. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  11. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  12. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
38 more
  1. 2021Grounded Language-Image Pre-trainingcited by 1,156
  2. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  3. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  4. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 819
  5. 2023Multimodal Chain-of-Thought Reasoning in Language Modelscited by 752
  6. 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
  7. 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decodingcited by 688
  8. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  9. 2022Open-Vocabulary Semantic Segmentation with Mask-adapted CLIPcited by 565
  10. 2023Segment Everything Everywhere All at Oncecited by 548
  11. 2023OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocationcited by 478
  12. 2023Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuningcited by 445
  13. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  14. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  15. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 357
  16. 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Modelscited by 316
  17. 2022Generalized Decoding for Pixel, Image, and Languagecited by 294
  18. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 256
  19. 2023GRES: Generalized Referring Expression Segmentationcited by 254
  20. 2023Woodpecker: Hallucination Correction for Multimodal Large Language Modelscited by 226
  21. 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimizationcited by 225
  22. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  23. 2022Language as Queries for Referring Video Object Segmentationcited by 182
  24. 2022DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detectioncited by 176
  25. 2024HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decodingcited by 174
  26. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  27. 2023Silkie: Preference Distillation for Large Visual Language Modelscited by 130
  28. 2023HalluciDoctor: Mitigating Hallucinatory Toxicity in Visual Instruction Datacited by 126
  29. 2024IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decodingcited by 118
  30. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  31. 2023BuboGPT: Enabling Visual Grounding in Multi-Modal LLMscited by 114
  32. 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revisioncited by 98
  33. 2024MLLM can see? Dynamic Correction Decoding for Hallucination Mitigationcited by 88
  34. 2023Mitigating Fine-Grained Hallucination by Fine-Tuning Large Vision-Language Models with Caption Rewritescited by 61
  35. 2024DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLMcited by 46
  36. 2023Described Object Detection: Liberating Object Detection with Flexible Expressionscited by 40
  37. 2024HaloQuest: A Visual Hallucination Dataset for Advancing Multimodal Reasoningcited by 32
  38. 2023Towards LLM-driven Dialogue State Trackingcited by 25

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026Beyond Frame Selection: Generative Latent Evidence Aggregation for Long-Video Understandingcited by 2
  2. 2026Micro-Macro Retrieval: Reducing Long-Form Hallucination in Large Language Modelscited by 0
  3. 2026LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learningcited by 0
  4. 2026Pay More Attention To Text In High-Resolution MLLMscited by 0
  5. 2026When Words Speak Louder than Images: Towards Understanding Language Bias in Vision-Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Faithful Grounded Visual Reasoning via Learned Proxy-Tokens
  2. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluation
  3. 2026GeM-VG: Towards Generalized Multi-image Visual Grounding with Multimodal Large Language Models
  4. 2024Plug-and-Play Grounding of Reasoning in Multimodal Large Language Models
  5. 2024FiVL: A Framework for Improved Vision-Language Alignment through the Lens of Training, Evaluation and Explainability
  6. 2025Towards Understanding Visual Grounding in Visual Language Models
  7. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMs
  8. 2025Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning