mlsanityData to 7 Oct 2026

The Essential Role of Causality in Foundation World Models for Embodied AI

arXiv 2402.06665 ↗First posted 6 Feb 2024cs.AI cs.CL cs.LGcited by 32

On the map: World models → World models for navigation

Abstract (arXiv)

Recent advances in foundation models, especially in large multi-modal models and conversational agents, have ignited interest in the potential of generally capable embodied agents. Such agents will require the ability to perform new tasks in many different real-world environments. However, current foundation models fail to accurately model physical interactions and are therefore insufficient for Embodied AI. The study of causality lends itself to the construction of veridical world models, which are crucial for accurately predicting the outcomes of possible interactions. This paper focuses on the prospects of building foundation world models for the upcoming generation of embodied agents and presents a novel viewpoint on the significance of causality within these. We posit that integrating causal considerations is vital to facilitating meaningful physical interactions with the world. Finally, we demystify misconceptions about causality in this context and present our outlook for future research.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 120 of its 202 references that are in the map. First posted on arXiv 6 Feb 2024.

Builds on

120 of 202 references.

References · 120

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  7. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  8. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  9. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  10. 2018World Modelscited by 1,444
  11. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  12. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
38 more
  1. 2023Mastering Diverse Domains through World Modelscited by 1,193
  2. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  3. 2020Mastering Atari with Discrete World Modelscited by 990
  4. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 882
  5. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 850
  6. 2021Florence: A New Foundation Model for Computer Visioncited by 802
  7. 2022A Generalist Agentcited by 802
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  9. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  10. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  11. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 684
  12. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  13. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  14. 2015Estimation and Inference of Heterogeneous Treatment Effects using Random Forestscited by 597
  15. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  16. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 550
  17. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  18. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  19. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  20. 2020Planning to Explore via Self-Supervised World Modelscited by 365
  21. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  22. 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
  23. 2022Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoningcited by 331
  24. 2017Quasi-Oracle Estimation of Heterogeneous Treatment Effectscited by 328
  25. 2022Transformers are Sample-Efficient World Modelscited by 317
  26. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 260
  27. 2023Detecting and Preventing Hallucinations in Large Vision Language Modelscited by 254
  28. 2023Multimodal Foundation Models: From Specialists to General-Purpose Assistantscited by 247
  29. 2019Causality for Machine Learningcited by 242
  30. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 240
  31. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  32. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  33. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
  34. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
  35. 2021Offline-to-Online Reinforcement Learning via Balanced Replay and Pessimistic Q-Ensemblecited by 211
  36. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 204
  37. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  38. 2023Planning with Large Language Models for Code Generationcited by 179

50 most cited of 120 All 120 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 15; 2026: 7.

32 papers.

Sort:

By area:

  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  2. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  3. 2024BadRobot: Jailbreaking Embodied LLM Agents in the Physical Worldcited by 40
  4. 2024Artificial General Intelligence (AGI)-Native Wireless Systems: A Journey Beyond 6Gcited by 30
  5. 2025Embodied AI: From LLMs to World Modelscited by 23
  6. 2025A Survey on Large Language Models for Automated Planningcited by 17
  7. 2025I2-NeRF: Learning Neural Radiance Fields Under Physically-Grounded Media Interactionscited by 17
  8. 2025A Step Toward World Models: A Survey on Robotic Manipulationcited by 15
  9. 2025Reconstructing 4D Spatial Intelligence: A Surveycited by 13
  10. 2024CELLO: Causal Evaluation of Large Vision-Language Modelscited by 10
  11. 2025Four Principles for Physically Interpretable World Modelscited by 10
  12. 2025Position: Foundation Models Need Digital Twin Representationscited by 10
20 more
  1. 2024Language Agents Meet Causality -- Bridging LLMs and Causal World Modelscited by 8
  2. 2026What-If World: A Causal Benchmark for General World Models in Embodied Scenarioscited by 6
  3. 2024A Taxonomy of Architecture Options for Foundation Model-based Agents: Analysis and Decision Modelcited by 4
  4. 2025From 2D to 3D Cognition: A Brief Survey of General World Modelscited by 4
  5. 2026A Definition and Roadmap for World Modelscited by 3
  6. 2024CARL-GT: Evaluating Causal Reasoning Capabilities of Large Language Modelscited by 2
  7. 2025Foundation Models in Biomedical Imaging: Turning Hype into Realitycited by 2
  8. 2024Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspectivecited by 1
  9. 2025Leveraging Vision-Language Foundation Models to Reveal Hidden Image-Attribute Relationships in Medical Imagingcited by 1
  10. 2025Efficient and Generalizable Environmental Understanding for Visual Navigationcited by 1
  11. 2025Zero-shot World Models via Search in Memorycited by 1
  12. 2026Can Current Agents Close the Discovery-to-Application Gap? A Case Study in Minecraftcited by 1
  13. 2026Why LLMs Fail at Causal Discovery and How Interventional Agents Escapecited by 1
  14. 2026Autonomous Video Generation with Counterfactual Controllability for Self-Evolving World Modelscited by 1
  15. 2026Thinking in Video: Can Video Generators Really Reason About the Real World?cited by 1
  16. 2024Disentangled Representations for Causal Cognitioncited by 0
  17. 2024Adaptive World Models: Learning Behaviors by Latent Imagination Under Non-Stationaritycited by 0
  18. 2025Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systemscited by 0
  19. 2025Learning Robust Intervention Representations with Delta Embeddingscited by 0
  20. 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignmentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Physically Viable World Models: A Case for Query-Conditioned Embodied AI
  2. 2025Embodied AI Agents: Modeling the World
  3. 2026Explicit World Models for Reliable Human-Robot Collaboration
  4. 2025WoW: Towards a World omniscient World model Through Embodied Interaction
  5. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI
  6. 2026Self-Evolving Cognitive Framework via Causal World Modeling for Embodied Scientific Intelligence
  7. 2025Modeling the Mental World for Embodied AI: A Comprehensive Review
  8. 2026Planning Takes More Than Token Prediction: Causal Plan for Benchmarking and Building Physically Grounded Embodied Reasoners