Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
While Multimodal Large Language Models (MLLMs) demonstrate impressive performance in benign scenarios, their cognitive reliability deteriorates significantly in complex scenes under adverse conditions. In these settings, models often rely on implicit inference without sufficient visual evidence, leading to a disconnect between perception and reasoning. Meanwhile, existing outcome-oriented benchmarks evaluate only final predictions and fail to diagnose failures in the underlying reasoning process. To address this gap, the authors propose AD2-Bench, which introduces a Hierarchical Visual Diagnosis framework that decomposes reasoning into a structured Chain of Evidence (CoE). This fine-grained diagnosis reveals that robust multimodal reasoning fundamentally depends on accurate evidence acquisition. Building on this perspective, the authors formulate reasoning from a probabilistic viewpoint and identify two primary causes of reasoning failure: Spatial Ambiguity, where models fail to distinguish target objects from background clutter, resulting in localization errors; and Semantic Uncertainty, where degraded visual features lead to incorrect semantic interpretation, resulting in understanding errors. To overcome these evidence deficiencies, they further propose Evidence-grounded Visual Reasoning (EGVOR), which replaces implicit reasoning with the explicit generation of Evidence Atoms - structured spatial-semantic triplets that enforce tight alignment between localization and semantic understanding. The model is trained through a hierarchical curriculum that progresses from reflective supervision construction to reinforcement learning, where reducing reasoning variance is explicitly rewarded. Extensive experiments demonstrate that EGVOR substantially improves reasoning stability under adverse conditions, providing a more robust framework for trustworthy multimodal cognition.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 74 of its 101 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
74 of 101 references.
Closest ancestors · 18
Same direction, newest first
- 2025VGR: Visual Grounded Reasoning13 Jun
- 2025Advancing Multimodal Reasoning via Reinforcement Learning with Cold Start28 May
- 2025Ground-R1: Incentivizing Grounded Visual Reasoning via Reinforcement Learning26 May
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning21 May
- 2025GRIT: Teaching MLLMs to Think with Images21 May
- 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
- 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
- 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
- 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
- 2025Visual-RFT: Visual Reinforcement Fine-Tuning3 Mar
- 2025ReFocus: Visual Editing as a Chain of Thought for Structured Image Understanding9 Jan
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling6 Dec
6 more
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
- 2024Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models19 Mar
- 2024KAM-CoT: Knowledge Augmented Multimodal Chain-of-Thoughts Reasoning23 Jan
- 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts3 Oct
- 2023MMBench: Is Your Multi-modal Model an All-around Player?12 Jul
- 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning19 Mar
Other references · 56
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Visual Instruction Tuningcited by 7,869
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2015VQA: Visual Question Answeringcited by 2,506
38 more
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,729
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
- 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,050
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 735
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
- 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 490
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
- 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
- 2024MME-RealWorld: Could Your Multimodal LLM Challenge High-Resolution Real-World Scenarios that are Difficult for Humans?cited by 292
- 2024mPLUG-Owl3: Towards Long Image-Sequence Understanding in Multi-Modal Large Language Modelscited by 291
- 2020Detection and Tracking Meet Drones Challengecited by 263
- 2023NuScenes-QA: A Multi-modal Visual Question Answering Benchmark for Autonomous Driving Scenariocited by 262
- 2018Textual Explanations for Self-Driving Vehiclescited by 226
- 2023Drive Like a Human: Rethinking Autonomous Driving with Large Language Modelscited by 160
- 2023LingoQA: Visual Question Answering for Autonomous Drivingcited by 145
- 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformerscited by 96
- 2020Explainable Object-induced Action Decision for Autonomous Vehiclescited by 88
- 2024VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervisioncited by 86
- 2024Automated Evaluation of Large Vision-Language Models on Self-driving Corner Casescited by 77
- 2023Octopus: Embodied Vision-Language Programmer from Environmental Feedbackcited by 73
- 2024MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMscited by 72
- 2025Traceable Evidence Enhanced Visual Grounded Reasoning: Evaluation and Methodologycited by 69
- 2024Fusion-Mamba for Cross-modality Object Detectioncited by 67
- 2019AI2D-RST: A multimodal corpus of 1000 primary school science diagramscited by 60
50 most cited of 56 All 56 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
- 2026Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
- 2026VisualFLIP: Do Predictions Depend on Task-Critical Visual Evidence in Multimodal Reasoning?
- 2025VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity
- 2026Beyond Accuracy: Evaluating Grounded Visual Evidence in Thinking with Images
- 2025Visual Reasoning Tracer: Object-Level Grounded Reasoning Benchmark
- 2026Don't Blink: Evidence Collapse during Multimodal Reasoning
- 2026ViSTR-Bench: Can MLLMs Reason from Continuous Visual Cues in Dynamic Scenes?