mlsanityData to 14 Oct 2026

From Prompts to Tokens: Internalizing Causal Supervision in Vision-Language Model for Multi-Image Causal Reasoning

arXiv 2606.11745 ↗First posted 10 Jun 2026cs.CV cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Visual causal reasoning is essential for understanding and intervening in the physical world, requiring identification of causal variables from visual inputs and reasoning over intervention effects. Despite recent progress, large vision--language models (VLMs) remain brittle at such tasks, especially for interventional and counterfactual queries over multi-image inputs. Most existing explorations inject causal knowledge via textual prompts, leaving causal mechanisms external to model execution and limiting reliable control during inference. To address this problem, we propose BridgeVLM, which internalizes visual causal reasoning by inducing a causal graph from multi-image inputs and converting it into structured Causal Tokens executed by RAMP layers injected into the LLM decoder for causal message passing. We further introduce a unified training interface M3S for fine-grained causal supervision from different granularities (local/global level). BridgeVLM achieves 54.4% accuracy on intervention tasks on CausalVLBench (vs. 33.2% with prompt-level supervision), improves results on Causal3D from 43.6% to 49.0%, and substantially improves causal structure learning on CausalVLBench (F1: 33.4% rightarrow 75.1%).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 32 references that are in the map. First posted on arXiv 10 Jun 2026.

Builds on

13 of 32 references.

References · 13

  1. 2020End-to-End Object Detection with Transformerscited by 7,414
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  4. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  5. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  6. 2024Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoningcited by 110
  7. 2023Can large language models build causal graphs?cited by 73
  8. 2023Causal Discovery with Language Models as Imperfect Expertscited by 64
  9. 2024Efficient Causal Graph Discovery Using Large Language Modelscited by 53
  10. 2024Large Language Models for Causal Discovery: Current Landscape and Future Directionscited by 25
  11. 2024Large Language Models are Effective Priors for Causal Graph Discoverycited by 24
  12. 2025CAUSAL3D: A Comprehensive Benchmark for Causal Learning from Visual Datacited by 7
1 more
  1. 2024LLM-initialized Differentiable Causal Discoverycited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models
  2. 2025What's Missing in Vision-Language Models? Probing Their Struggles with Causal Order Reasoning
  3. 2024Multimodal Causal Reasoning Benchmark: Challenging Vision Large Language Models to Discern Causal Links Across Modalities
  4. 2026Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMs
  5. 2025CauSight: Learning to Supersense for Visual Causal Discovery
  6. 2026The Abstraction Gap in Vision-Language Causal Reasoning
  7. 2026CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Models
  8. 2026Diagnosing Causal Reasoning in Vision-Language Models via Structured Relevance Graphs