mlsanityData to 14 Oct 2026

Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes

arXiv 2608.10954 ↗First posted 11 Aug 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorates significantly in complex scenes under adverse conditions. In these settings, models often rely on implicit inference without sufficient visual evidence, leading to a disconnect between perception and reasoning. Meanwhile, existing outcome-oriented benchmarks evaluate only final predictions and fail to diagnose failures in the underlying reasoning process. To address this gap, the authors propose AD2-Bench, which introduces a Hierarchical Visual Diagnosis framework that decomposes reasoning into a structured Chain of Evidence (CoE). This fine-grained diagnosis reveals that robust multimodal reasoning fundamentally depends on accurate evidence acquisition. Building on this perspective, the authors formulate reasoning from a probabilistic viewpoint and identify two primary causes of reasoning failure: Spatial Ambiguity, where models fail to distinguish target objects from background clutter, resulting in localization errors; and Semantic Uncertainty, where degraded visual features lead to incorrect semantic interpretation, resulting in understanding errors. To overcome these evidence deficiencies, they further propose Evidence-grounded Visual Reasoning (EGVOR), which replaces implicit reasoning with the explicit generation of Evidence Atoms - structured spatial-semantic triplets that enforce tight alignment between localization and semantic understanding. The model is trained through a hierarchical curriculum that progresses from reflective supervision construction to reinforcement learning, where reducing reasoning variance is explicitly rewarded. Extensive experiments demonstrate that EGVOR substantially improves reasoning stability under adverse conditions, providing a more robust framework for trustworthy multimodal cognition.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 74 of its 101 references that are in the map. First posted on arXiv 11 Aug 2026.

Builds on

74 of 101 references.

Closest ancestors · 18

Same direction, newest first

  1. 2025VGR: Visual Grounded Reasoning13 Jun
  2. 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start28 May
  3. 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning26 May
  4. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
  5. 2025GRIT: Teaching MLLMs to Think with Images21 May
  6. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  7. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  8. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  9. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  10. 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
  11. 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
  12. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
6 more
  1. 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
  2. 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models19 Mar
  3. 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning23 Jan
  4. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
  5. 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
  6. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar

Other references · 56

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2025Qwen2.5-VL Technical Reportcited by 5,587
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  11. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  12. 2015VQA: Visual Question Answeringcited by 2,506
38 more
  1. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  3. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  4. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
  5. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  6. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  7. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  8. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  9. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
  10. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  11. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  12. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
  13. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  14. 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
  15. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  16. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  17. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  18. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
  19. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  20. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  21. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  22. 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
  23. 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 291
  24. 2020Detection and Tracking Meet Drones Challengecited by 263
  25. 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
  26. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  27. 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Modelscited by 160
  28. 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 145
  29. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  30. 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
  31. 2020Explainable Object-induced Action Decision for Autonomous Vehiclescited by 88
  32. 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 86
  33. 2024Automated Evaluation of Large Vision-Language Models on Self-driving Corner Casescited by 77
  34. 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
  35. 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
  36. 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 69
  37. 2024Fusion-Mamba for Cross-modality Object Detectioncited by 67
  38. 2019AI2D-RST: A multimodal corpus of 1000 primary school science diagramscited by 60

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
  2. 2026Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
  3. 2026VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?
  4. 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
  5. 2026Beyond Accuracy: Evaluating Grounded Visual Evidence in Thinking with Images
  6. 2025Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmark
  7. 2026Don't Blink: Evidence Collapse during Multimodal Reasoning
  8. 2026ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?