mlsanityData to 14 Oct 2026

Embodied AI Agents: Modeling the World

arXiv 2506.22355 ↗First posted 27 Jun 2025cs.AIcited by 61

On the map: World models → World models for navigation

Abstract (arXiv)

This paper describes our research on AI agents embodied in visual, virtual or physical forms, enabling them to interact with both users and their environments. These agents, which include virtual avatars, wearable devices, and robots, are designed to perceive, learn and act within their surroundings, which makes them more similar to how humans learn and interact with the environments as compared to disembodied agents. We propose that the development of world models is central to reasoning and planning of embodied AI agents, allowing these agents to understand and predict their environment, to understand user intentions and social contexts, thereby enhancing their ability to perform complex tasks autonomously. World modeling encompasses the integration of multimodal perception, planning through reasoning for action and control, and memory to create a comprehensive understanding of the physical world. Beyond the physical world, we also propose to learn the mental world model of users to enable better human-agent collaboration.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 92 of its 133 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

92 of 133 references.

References · 92

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2015Continuous control with deep reinforcement learningcited by 3,407
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2019Advances and Open Problems in Federated Learningcited by 3,046
  9. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2015Variational Inference with Normalizing Flowscited by 2,170
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
38 more
  1. 2019Expressive Body Capture: 3D Hands, Face, and Body from a Single Imagecited by 1,612
  2. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  3. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,538
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  7. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  8. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  9. 2018DeepMind Control Suitecited by 956
  10. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  12. 2024Genie: Generative Interactive Environmentscited by 747
  13. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 712
  14. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  15. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 575
  16. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  17. 2017Learning multiple visual domains with residual adapterscited by 557
  18. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 530
  19. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  20. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  21. 2023VideoPoet: A Large Language Model for Zero-Shot Video Generationcited by 492
  22. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 490
  23. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 484
  24. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  25. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  26. 2022MAGVIT: Masked Generative Video Transformercited by 346
  27. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  28. 2019Learning Latent Plans from Playcited by 316
  29. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  30. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 297
  31. 2020First return, then explorecited by 266
  32. 2020ARCH: Animatable Reconstruction of Clothed Humanscited by 257
  33. 2018Stochastic Video Generation with a Learned Priorcited by 245
  34. 2017Stochastic Variational Video Predictioncited by 234
  35. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  36. 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 218
  37. 2022BlenderBot 3: a deployed conversational agent that continually learns to responsibly engagecited by 198
  38. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 195

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 20; 2026: 41.

61 papers.

Sort:

By area:

  1. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memorycited by 97
  2. 2025A Comprehensive Survey on World Models for Embodied AIcited by 68
  3. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 68
  4. 20253D and 4D World Modeling: A Surveycited by 67
  5. 2026A Survey of Agent Memory in the Second Half: Towards Self-Evolving and Long-Horizon Agentscited by 44
  6. 2025Planning with Reasoning using Vision Language World Modelcited by 36
  7. 2026Uncertainty Quantification in LLM Agents: Foundations, Emerging Challenges, and Opportunitiescited by 20
  8. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 15
  9. 2025Towards Unified World Models for Visual Navigation via Memory-Augmented Planning and Foresightcited by 14
  10. 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 13
  11. 2026Action100M: A Large-scale Video Action Datasetcited by 12
  12. 2026VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Modelcited by 9
38 more
  1. 2026VisPhyWorld: Probing Physical Reasoning via Code-Driven Video Reconstructioncited by 7
  2. 2026Graph World Models: Concepts, Taxonomy, and Future Directionscited by 6
  3. 2025Embodied AI: Emerging Risks and Opportunities for Policy Actioncited by 5
  4. 2025MoWM: Mixture-of-World-Models for Embodied Planning via Latent-to-Pixel Feature Modulationcited by 5
  5. 2025Dynamic Reflections: Probing Video Representations with Text Alignmentcited by 5
  6. 2025MindPower: Enabling Theory-of-Mind Reasoning in VLM-based Embodied Agentscited by 5
  7. 2025DynamicVerse: A Physically-Aware Multimodal Framework for 4D World Modelingcited by 5
  8. 2026Aligning Agentic World Models via Knowledgeable Experience Learningcited by 5
  9. 2026From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agentscited by 5
  10. 2026GraphThinker: Reinforcing Temporally Grounded Video Reasoning with Event Graph Thinkingcited by 5
  11. 2026SCRIPT: Scalable Diffusion Policy with Multi-stage Training for Language-driven Physics-Based Humanoid Controlcited by 5
  12. 2025PersONAL: Towards a Comprehensive Benchmark for Personalized Embodied Agentscited by 4
  13. 2025Memory Transfer Planning: LLM-driven Context-Aware Code Adaptation for Robot Manipulationcited by 4
  14. 2025ReEXplore: Improving MLLMs for Embodied Exploration with Contextualized Retrospective Experience Replaycited by 4
  15. 2026Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Jointscited by 4
  16. 2026StreamingClaw Technical Reportcited by 4
  17. 2026Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotioncited by 4
  18. 2026Sentinel-VLA: A Metacognitive VLA Model with Active Status Monitoring for Dynamic Reasoning and Error Recoverycited by 4
  19. 2026SpeechLess: Micro-utterance with Personalized Spatial Memory-aware Assistant in Everyday Augmented Realitycited by 3
  20. 2026Can Vision-Language Models Solve the Shell Game?cited by 3
  21. 2026LABSHIELD: A Multimodal Benchmark for Safety-Critical Reasoning and Planning in Scientific Laboratoriescited by 3
  22. 2026HumanOmni-Speaker: Identifying Who said What and Whencited by 3
  23. 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
  24. 2025ESearch-R1: Learning Cost-Aware MLLM Agents for Interactive Embodied Search via Reinforcement Learningcited by 2
  25. 2025Large Emotional World Modelcited by 2
  26. 2026ChronoAgentic: A Code-based Multi-Agent World Simulator for Physically Grounded Simulation Constructioncited by 2
  27. 2026Critique of Agent Modelcited by 2
  28. 2025Modeling the Mental World for Embodied AI: A Comprehensive Reviewcited by 1
  29. 2026Embodied Science: Closing the Discovery Loop with Agentic Embodied AIcited by 1
  30. 20266G Communication Networks Enabling Embodied Agents: Architecture and Prototypecited by 1
  31. 2026How to Build Digital Humans? From Priors to Photorealistic Avatarscited by 1
  32. 2025Exploring the Link Between Bayesian Inference and Embodied Intelligence: Toward Open Physical-World Embodied AI Systemscited by 0
  33. 2025Telling Speculative Stories to Help Humans Imagine the Harms of Healthcare AIcited by 0
  34. 2025Systematizing LLM Persona Design: A Four-Quadrant Technical Taxonomy for AI Companion Applicationscited by 0
  35. 2026HoverAI: An Embodied Aerial Agent for Natural Human-Drone Interactioncited by 0
  36. 2026Pretraining A Large Language Model using Distributed GPUs: A Memory-Efficient Decentralized Paradigmcited by 0
  37. 2026ICAT: Incident-Case-Grounded Adaptive Testing for Physical-Risk Prediction in Embodied World Modelscited by 0
  38. 2026Source-Modality Monitoring in Vision-Language Modelscited by 0

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI
  2. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Models
  3. 2024Agent AI: Surveying the Horizons of Multimodal Interaction
  4. 2025Embodied Web Agents: Bridging Physical-Digital Realms for Integrated Agent Intelligence
  5. 2025Large Model Empowered Embodied AI: A Survey on Decision-Making and Embodied Learning
  6. 2024Position Paper: Agent AI Towards a Holistic Intelligence
  7. 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agents
  8. 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence