mlsanityData to 14 Oct 2026

Language Models Meet World Models: Embodied Experiences Enhance Language Models

arXiv 2305.10626 ↗First posted 18 May 2023cs.CL cs.AI cs.LGcited by 124

On the map: World models → World models for language agents

Abstract (arXiv)

While large language models (LMs) have shown remarkable capabilities across numerous tasks, they often struggle with simple reasoning and planning in physical environments, such as understanding object permanence or planning household activities. The limitation arises from the fact that LMs are trained only on written text and miss essential embodied knowledge and skills. In this paper, we propose a new paradigm of enhancing LMs by finetuning them with world models, to gain diverse embodied knowledge while retaining their general language capabilities. Our approach deploys an embodied agent in a world model, particularly a simulator of the physical world (VirtualHome), and acquires a diverse set of embodied experiences through both goal-oriented planning and random exploration. These experiences are then used to finetune LMs to teach diverse abilities of reasoning and acting in the physical world, e.g., planning and completing goals, object permanence and tracking, etc. Moreover, it is desirable to preserve the generality of LMs during finetuning, which facilitates generalizing the embodied knowledge across tasks rather than being tied to specific simulations. We thus further introduce the classical (EWC) for selective weight updates, combined with low-rank adapters (LoRA) for training efficiency. Extensive experiments show our approach substantially improves base LMs on 18 downstream tasks by 64.28% on average. In particular, the small LMs (1.3B, 6B, and 13B) enhanced by our approach match or even outperform much larger LMs (e.g., ChatGPT).

Key facts. Cited by 124 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 54 of its 65 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

54 of 65 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul
  2. 2020Language Models are Few-Shot Learners28 May

Other references · 52

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  8. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  9. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  10. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 3,024
  12. 2020Learning to summarize from human feedbackcited by 2,588
38 more
  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  3. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
  8. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
  9. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  10. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  11. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 841
  13. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  14. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 611
  15. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  16. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  17. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
  18. 2015Towards AI-Complete Question Answering: A Set of Prerequisite Toy Taskscited by 314
  19. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  20. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 237
  21. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 221
  22. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 215
  23. 2022Leveraging Large Language Models for Multiple Choice Question Answeringcited by 203
  24. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  25. 2020Watch-And-Help: A Challenge for Social Perception and Human-AI Collaborationcited by 133
  26. 2018Building Generalizable Agents with a Realistic and Rich 3D Environmentcited by 109
  27. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 84
  28. 2017MINOS: Multimodal Indoor Simulator for Navigation in Complex Environmentscited by 66
  29. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 58
  30. 2022Housekeep: Tidying Virtual Households using Commonsense Reasoningcited by 57
  31. 2021PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D Worldcited by 47
  32. 2022Rainier: Reinforced Knowledge Introspector for Commonsense Question Answeringcited by 43
  33. 2018CHALET: Cornell House Agent Learning Environmentcited by 38
  34. 2019VRKitchen: an Interactive 3D Virtual Environment for Task-oriented Learningcited by 29
  35. 2021JueWu-MC: Playing Minecraft with Sample-efficient Hierarchical Reinforcement Learningcited by 28
  36. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
  37. 2020Learning to Stop: A Simple yet Effective Approach to Urban Vision-Language Navigationcited by 20
  38. 2022ASDOT: Any-Shot Data-to-Text Generation with Pretrained Language Modelscited by 14

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 13; 2024: 45; 2025: 41; 2026: 25.

124 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 706
  4. 2025SWE-smith: Scaling Data for Software Engineering Agentscited by 285
  5. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 279
  6. 2023ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddingscited by 240
  7. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  8. 2024Scene-LLM: Extending Language Model for 3D Visual Understanding and Reasoningcited by 175
  9. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  10. 2024A Survey on Large Language Model-Based Game Agentscited by 112
  11. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
  12. 2024AIOS: LLM Agent Operating Systemcited by 97
38 more
  1. 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 86
  2. 2024From Individual to Society: A Survey on Social Simulation Driven by Large Language Model-based Agentscited by 82
  3. 2024AutoAct: Automatic Agent Learning from Scratch for QA via Self-Planningcited by 81
  4. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  5. 2025RynnVLA-002: A Unified Vision-Language-Action and World Modelcited by 74
  6. 2024Agent Planning with World Knowledge Modelcited by 70
  7. 2024LLM3:Large Language Model-based Task and Motion Planning with Motion Failure Reasoningcited by 63
  8. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 60
  9. 2023Continual Learning: Applications and the Road Forwardcited by 59
  10. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 54
  11. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 54
  12. 2023Stance Detection with Collaborative Role-Infused LLM-Based Agentscited by 53
  13. 2023LangNav: Language as a Perceptual Representation for Navigationcited by 47
  14. 2024BadRobot: Jailbreaking Embodied LLM Agents in the Physical Worldcited by 43
  15. 2024WorldGPT: Empowering LLM as Multimodal World Modelcited by 42
  16. 2023Statler: State-Maintaining Language Models for Embodied Reasoningcited by 40
  17. 2024Benchmarking Agentic Workflow Generationcited by 37
  18. 2025STeCa: Step-level Trajectory Calibration for LLM Agent Learningcited by 37
  19. 2024True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learningcited by 36
  20. 2025PAN: A World Model for General, Actionable, and Long-Horizon World Simulationcited by 36
  21. 2025CityEQA: A Hierarchical LLM Agent on Embodied Question Answering Benchmark in City Spacecited by 32
  22. 2024CityGPT: Empowering Urban Spatial Cognition of Large Language Modelscited by 30
  23. 2025A Framework for Benchmarking and Aligning Task-Planning Safety in LLM-Based Embodied Agentscited by 30
  24. 2024Take Caution in Using LLMs as Human Surrogates: Scylla Ex Machinacited by 29
  25. 2024DriVLMe: Enhancing LLM-based Autonomous Driving Agents with Embodied and Social Experiencescited by 27
  26. 2024CityBench: Evaluating the Capabilities of Large Language Models for Urban Taskscited by 27
  27. 2025Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuningcited by 27
  28. 2024WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agentscited by 26
  29. 2025Critique of World Modelcited by 25
  30. 2025Embodied AI: From LLMs to World Modelscited by 24
  31. 2023Overcoming Generic Knowledge Loss with Selective Parameter Updatecited by 23
  32. 2024S-Agents: Self-organizing Agents in Open-ended Environmentscited by 22
  33. 2024Making Large Language Models into World Models with Precondition and Effect Knowledgecited by 22
  34. 2023LLaMA Rider: Spurring Large Language Models to Explore the Open Worldcited by 21
  35. 2024ControlAgent: Automating Control System Design via Novel Integration of LLM Agents and Domain Expertisecited by 21
  36. 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Drivingcited by 18
  37. 2024Language Grounded Multi-agent Reinforcement Learning with Human-interpretable Communicationcited by 17
  38. 2025Security of Internet of Agents: Attacks and Countermeasurescited by 17

50 most cited of 124 All 124 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environments
  2. 2023Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
  3. 2023Collaborating with language models for embodied reasoning
  4. 2025HELP: Hierarchical Embodied Language Planner for Household Tasks
  5. 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
  6. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models
  7. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
  8. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use