mlsanityData to 7 Oct 2026

Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modelling

arXiv 2301.12050 ↗First posted 28 Jan 2023cs.LG cs.CLcited by 88

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

Reinforcement learning (RL) agents typically learn tabula rasa, without prior knowledge of the world. However, if initialized with knowledge of high-level subgoals and transitions between subgoals, RL agents could utilize this Abstract World Model (AWM) for planning and exploration. We propose using few-shot large language models (LLMs) to hypothesize an AWM, that will be verified through world experience, to improve sample efficiency of RL agents. Our DECKARD agent applies LLM-guided exploration to item crafting in Minecraft in two phases: (1) the Dream phase where the agent uses an LLM to decompose a task into a sequence of subgoals, the hypothesized AWM; and (2) the Wake phase where the agent learns a modular policy for each subgoal and verifies or corrects the hypothesized AWM. Our method of hypothesizing an AWM with LLMs and then verifying the AWM based on agent experience not only increases sample efficiency over contemporary methods by an order of magnitude but is also robust to and corrects errors in the LLM, successfully blending noisy internet-scale information from LLMs with knowledge grounded in environment dynamics.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 41 of its 48 references that are in the map. First posted on arXiv 28 Jan 2023.

Builds on

41 of 48 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul
  2. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge17 Jun
  3. 2021Learning Knowledge Graph-based World Models of Textual Environments17 Jun
  4. 2020Language Models are Few-Shot Learners28 May

Other references · 37

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  3. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2023Mastering Diverse Domains through World Modelscited by 1,193
  6. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  8. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  9. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  10. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  11. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  12. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 401
25 more
  1. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 372
  2. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videoscited by 338
  3. 2017Embodied Question Answeringcited by 335
  4. 2020Experience Grounds Languagecited by 241
  5. 2019MineRL: A Large-Scale Dataset of Minecraft Demonstrationscited by 210
  6. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  7. 2017IQA: Visual Question Answering in Interactive Environmentscited by 154
  8. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  9. 2022Mind's Eye: Grounded Language Model Reasoning through Simulationcited by 83
  10. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  11. 2021Embodied BERT: A Transformer Model for Embodied, Language-guided Visual Task Completioncited by 57
  12. 2022Semantic Exploration from Language Abstractions and Pretrained Representationscited by 51
  13. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  14. 2021PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D Worldcited by 47
  15. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  16. 2014Learning to Win by Reading Manuals in a Monte-Carlo Frameworkcited by 39
  17. 2020How to Avoid Being Eaten by a Grue: Structured Exploration Strategies for Textual Worldscited by 38
  18. 2020Align-RUDDER: Learning From Few Demonstrations by Reward Redistributioncited by 35
  19. 2019RTFM: Generalising to Novel Environment Dynamics via Readingcited by 33
  20. 2022One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestonescited by 26
  21. 2018Interactive Grounded Language Acquisition and Generalization in a 2D Worldcited by 21
  22. 2020Playing Text-Based Games with Common Sensecited by 21
  23. 2020Compositional Networks Enable Systematic Generalization for Grounded Language Understandingcited by 18
  24. 2021Modular Framework for Visuomotor Language Groundingcited by 9
  25. 2022Learning to Query Internet Text for Informing Reinforcement Learning Agentscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 24; 2024: 32; 2025: 24; 2026: 8.

88 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  4. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  5. 2023Language Models can Solve Computer Taskscited by 371
  6. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesiscited by 350
  7. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  8. 2023Challenges and Applications of Large Language Modelscited by 251
  9. 2023Leveraging Pre-trained Large Language Models to Construct and Utilize World Models for Model-based Task Planningcited by 204
  10. 2023SwiftSage: A Generative Agent with Fast and Slow Thinking for Complex Interactive Taskscited by 166
  11. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
  12. 2023User Behavior Simulation with Large Language Model based Agentscited by 136
38 more
  1. 2023Large Language Models for Robotics: A Surveycited by 132
  2. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 130
  3. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 128
  4. 2026SoK: Agentic Skills -- Beyond Tool Use in LLM Agentscited by 120
  5. 2024Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methodscited by 114
  6. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 99
  7. 2023STEVE-1: A Generative Model for Text-to-Behavior in Minecraftcited by 95
  8. 2025Lifelong Learning of Large Language Model based Agents: A Roadmapcited by 78
  9. 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directionscited by 74
  10. 2024Scaling Instructable Agents Across Many Simulated Worldscited by 67
  11. 2023See and Think: Embodied Agent in Virtual Environmentcited by 54
  12. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  13. 2024Can Language Models Serve as Text-Based World Simulators?cited by 50
  14. 2024Project Sid: Many-agent simulations toward AI civilizationcited by 46
  15. 2024Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Modelscited by 45
  16. 2024EmbodiedCity: A Benchmark Platform for Embodied Agent in Real-world City Environmentcited by 42
  17. 2024Vision-Language Models Provide Promptable Representations for Reinforcement Learningcited by 41
  18. 2024On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)cited by 40
  19. 2023Learning adaptive planning representations with natural language guidancecited by 36
  20. 2025Multi-agent Embodied AI: Advances and Future Directionscited by 36
  21. 2023Skill Reinforcement Learning and Planning for Open-World Long-Horizon Taskscited by 35
  22. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  23. 2024Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skillscited by 24
  24. 2023Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Webcited by 23
  25. 2024AgentLens: Visual Analysis for Agent Behaviors in LLM-based Autonomous Systemscited by 23
  26. 2024Making Large Language Models into World Models with Precondition and Effect Knowledgecited by 21
  27. 2023Can ChatGPT Enable ITS? The Case of Mixed Traffic Control via Reinforcement Learningcited by 18
  28. 2024MobileAgent: enhancing mobile control via human-machine interaction and SOP integrationcited by 18
  29. 2024From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systemscited by 17
  30. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  31. 2023Reinforcement Learning Friendly Vision-Language Model for Minecraftcited by 14
  32. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  33. 2024Open-World Reinforcement Learning over Long Short-Term Imaginationcited by 13
  34. 2024ADAM: An Embodied Causal Agent in Open-World Environmentscited by 13
  35. 2024Few-shot Dialogue Strategy Learning for Motivational Interviewing via Inductive Reasoningcited by 11
  36. 2023AdaRefiner: Refining Decisions of Language Models with Adaptive Feedbackcited by 10
  37. 2025DMWM: Dual-Mind World Model with Long-Term Imaginationcited by 10
  38. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Training Agents Inside of Scalable World Models
  2. 2024LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence
  3. 2026MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraft
  4. 2025From Word to World: Can Large Language Models be Implicit Text-based World Models?
  5. 2024World Models with Hints of Large Language Models for Goal Achieving
  6. 2025Optimus-2: Multimodal Minecraft Agent with Goal-Observation-Action Conditioned Policy
  7. 2025OpenHA: A Series of Open-Source Hierarchical Agentic Models in Minecraft
  8. 2025WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-making