InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
On the map: World models → World models for language agents
Abstract (arXiv)
Multimodal large language models often capture visual-linguistic correlations but struggle to predict how local visual interventions propagate and affect downstream answers. We introduce InfluenceField, an intervention-aware latent field inserted between the visual encoder and language decoder. It lifts patch features into a continuous spatial representation, propagates directed influence over multiple steps, and predicts local intervention effects through a shared transition operator. Training jointly optimizes language modeling, cross-environment invariance, counterfactual rollout supervision, and structural regularization. For a nonlinear finite-basis population model, we show that target-aligned interventional supervision, together with a one-step separation condition on the transition, restricts admissible representations to within-location reparameterizations, so that the directed dependency graph of the full transition is recovered exactly. A linear specialization gives an exact partial-coverage characterization and a finite-loss stability bound, and the field analysis derives the spatial profile of coefficient interventions together with a shared-channel calibration result. On CausalVQA, InfluenceField improves overall accuracy over its backbone by 13.1 percentage points, with the largest gains on the planning and hypothetical categories. Capacity-matched baselines and structural controls attribute the gains in robustness and factual-counterfactual consistency to the causal objectives rather than to added capacity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 1 of its 47 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
1 of 47 references.
References · 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models
- 2026Beyond Language Modeling: An Exploration of Multimodal Pretraining
- 2026WOVEN: Weaving Visual World Modeling into Multimodal LLMs
- 2025CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models
- 2026From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models
- 2026Causal Probing for Internal Visual Representations in Multimodal Large Language Models
- 2025How Visual Representations Map to Language Feature Space in Multimodal LLMs
- 2025ISO-Bench: Benchmarking Multimodal Causal Reasoning in Visual-Language Models through Procedural Plans