mlsanityData to 14 Oct 2026

Collaborating with language models for embodied reasoning

arXiv 2302.00763 ↗First posted 1 Feb 2023cs.LG cs.AI cs.CLcited by 61

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Reasoning in a complex and ambiguous environment is a key goal for Reinforcement Learning (RL) agents. While some sophisticated RL agents can successfully solve difficult tasks, they require a large amount of training data and often struggle to generalize to new unseen environments and new tasks. On the other hand, Large Scale Language Models (LSLMs) have exhibited strong reasoning ability and the ability to to adapt to new tasks through in-context learning. However, LSLMs do not inherently have the ability to interrogate or intervene on the environment. In this work, we investigate how to combine these complementary abilities in a single system consisting of three parts: a Planner, an Actor, and a Reporter. The Planner is a pre-trained language model that can issue commands to a simple embodied agent (the Actor), while the Reporter communicates with the Planner to inform its next command. We present a set of tasks that require reasoning, test this system's ability to generalize zero-shot and investigate failure cases, and demonstrate how components of this system can be trained with reinforcement-learning to improve performance.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 6 of its 11 references that are in the map. First posted on arXiv 1 Feb 2023.

Builds on

6 of 11 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Models12 Jul

Other references · 5

  1. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  2. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,202
  3. 2018IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architecturescited by 883
  4. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  5. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 25; 2024: 21; 2025: 13; 2026: 2.

61 papers.

Sort:

By area:

  1. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  2. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  3. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challengescited by 769
  4. 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
  5. 2023Cognitive Architectures for Language Agentscited by 429
  6. 2023Augmented Language Models: a Surveycited by 403
  7. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 398
  8. 2023Language Models can Solve Computer Taskscited by 372
  9. 2023Grounding Large Language Models in Interactive Environments with Online Reinforcement Learningcited by 195
  10. 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 171
  11. 2024Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoningcited by 136
  12. 2023Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Surveycited by 135
38 more
  1. 2024Exploring Large Language Model based Intelligent Agents: Definitions, Methods, and Prospectscited by 132
  2. 2025Improving Vision-Language-Action Model with Online Reinforcement Learningcited by 116
  3. 2024A Review of Prominent Paradigms for LLM-Based Agents: Tool Use (Including RAG), Planning, and Feedback Learningcited by 74
  4. 2023Learning to Model the World with Languagecited by 68
  5. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 60
  6. 2024A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challengescited by 55
  7. 2023SPOC: Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real Worldcited by 48
  8. 2023Towards A Unified Agent with Foundation Modelscited by 47
  9. 2024Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)cited by 46
  10. 2025Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Taskscited by 44
  11. 2024True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learningcited by 36
  12. 2025LLM-Powered GUI Agents in Phone Automation: Surveying Progress and Prospectscited by 32
  13. 2023Augmenting Autotelic Agents with Large Language Modelscited by 30
  14. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  15. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approachcited by 28
  16. 2023Are Large Language Models Temporally Grounded?cited by 27
  17. 2025Large Language Models Miss the Multi-Agent Markcited by 26
  18. 2023LLaMA Rider: Spurring Large Language Models to Explore the Open Worldcited by 21
  19. 2024Depending on yourself when you should: Mentoring LLM with RL agents to become the master in cybersecurity gamescited by 20
  20. 2024MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulationcited by 19
  21. 2023Semantic HELM: A Human-Readable Memory for Reinforcement Learningcited by 17
  22. 2023LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testingcited by 17
  23. 2023LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulationcited by 17
  24. 2024The RL/LLM Taxonomy Tree: Reviewing Synergies Between Reinforcement Learning and Large Language Modelscited by 15
  25. 2025Multi-Agent Systems Powered by Large Language Models: Applications in Swarm Intelligencecited by 14
  26. 2024A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Modelscited by 12
  27. 2024Embodied CoT Distillation From LLM To Off-the-shelf Agentscited by 9
  28. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  29. 2023Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learningcited by 8
  30. 2024Verco: Learning Coordinated Verbal Communication for Multi-agent Reinforcement Learningcited by 7
  31. 2024Assessing the Zero-Shot Capabilities of LLMs for Action Evaluation in RLcited by 7
  32. 2024Multi-granularity Knowledge Transfer for Continual Reinforcement Learningcited by 6
  33. 2024LangSuitE: Planning, Controlling and Interacting with Large Language Models in Embodied Text Environmentscited by 6
  34. 2025Preacher: Paper-to-Video Agentic Systemcited by 6
  35. 2024Counterfactual Effect Decomposition in Multi-Agent Sequential Decision Makingcited by 4
  36. 2025MAPLE: Multi-Agent Adaptive Planning with Long-Term Memory for Table Reasoningcited by 4
  37. 2025Chain-of-Trigger: An Agentic Backdoor that Paradoxically Enhances Agentic Robustnesscited by 4
  38. 2025Fetch.ai: An Architecture for Modern Multi-Agent Systemscited by 4

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents
  2. 2023Language Models Meet World Models: Embodied Experiences Enhance Language Models
  3. 2025RobotxR1: Enabling Embodied Robotic Intelligence on Large Language Models through Closed-Loop Reinforcement Learning
  4. 2024Integrating Large Language Models and Reinforcement Learning for Non-Linear Reasoning
  5. 2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agents
  6. 2025A Survey of Reinforcement Learning for Large Reasoning Models
  7. 2024Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use
  8. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents