mlsanityData to 7 Oct 2026

Inner Monologue: Embodied Reasoning through Planning with Language Models

arXiv 2207.05608 ↗First posted 12 Jul 2022cs.RO cs.AI cs.CLcited by 1,057

On the map: World models → World models for language agents

also in 1 more
Abstract (arXiv)

Recent works have shown how the reasoning capabilities of Large Language Models (LLMs) can be applied to domains beyond natural language processing, such as planning and interaction for robots. These embodied problems require an agent to understand many semantic aspects of the world: the repertoire of skills available, how these skills influence the world, and how changes to the world map back to the language. LLMs planning in embodied environments need to consider not just what skills to do, but also how and when to do them - answers that change over time in response to the agent's own choices. In this work, we investigate to what extent LLMs used in such embodied contexts can reason over sources of feedback provided through natural language, without any additional training. We propose that by leveraging environment feedback, LLMs are able to form an inner monologue that allows them to more richly process and plan in robotic control scenarios. We investigate a variety of sources of feedback, such as success detection, scene description, and human interaction. We find that closed-loop language feedback significantly improves high-level instruction completion on three domains, including simulated and real table top rearrangement tasks and long-horizon mobile manipulation tasks in a kitchen environment in the real world.

Key facts. Cited by 1,057 papers in the map (data through 7 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 71 of its 111 references that are in the map. First posted on arXiv 12 Jul 2022.

Builds on

71 of 111 references.

Closest ancestors · 1

Same direction, newest first

  1. 2020Language Models are Few-Shot Learners28 May

Other references · 70

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  6. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  7. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  8. 2021Evaluating Large Language Models Trained on Codecited by 8,053
  9. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  10. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  11. 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
  12. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
38 more
  1. 2015VQA: Visual Question Answeringcited by 2,489
  2. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  3. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  4. 2019Language Models as Knowledge Bases?cited by 1,922
  5. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  6. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  7. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  8. 2019How Can We Know What Language Models Know?cited by 953
  9. 2021Open-vocabulary Object Detection via Vision and Language Knowledge Distillationcited by 903
  10. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 871
  11. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  12. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  13. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 713
  14. 2021SimVLM: Simple Visual Language Model Pretraining with Weak Supervisioncited by 645
  15. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  16. 2019Object Detection in 20 Years: A Surveycited by 528
  17. 2022Socratic Models: Composing Zero-Shot Multimodal Reasoning with Languagecited by 513
  18. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  19. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  20. 2022Can language models learn from explanations in context?cited by 248
  21. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  22. 2022Pre-Trained Language Models for Interactive Decision-Makingcited by 220
  23. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 208
  24. 2021Simple but Effective: CLIP Embeddings for Embodied AIcited by 191
  25. 2019Commonsense Knowledge Mining from Pretrained Modelscited by 189
  26. 2019Search on the Replay Buffer: Bridging Planning and Reinforcement Learningcited by 188
  27. 2019oLMpics -- On what Language Model Pre-training Capturescited by 161
  28. 2020Unsupervised Commonsense Question Answering with Self-Talkcited by 161
  29. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  30. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 137
  31. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  32. 2013Deep Learning for Detecting Robotic Graspscited by 114
  33. 2021Skill Induction and Planning with Latent Languagecited by 99
  34. 2022Deep Hierarchical Planning from Pixelscited by 96
  35. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  36. 2017Neural Task Programming: Learning to Generalize Across Hierarchical Taskscited by 75
  37. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  38. 2022Correcting Robot Plans with Natural Language Feedbackcited by 68

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 25; 2023: 249; 2024: 276; 2025: 223; 2026: 284.

1,057 papers.

Sort:

By area:

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  8. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  9. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,348
  10. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  11. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  12. 2023Reasoning with Language Model is Planning with World Modelcited by 846
38 more
  1. 2023The Flan Collection: Designing Data and Methods for Effective Instruction Tuningcited by 734
  2. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  3. 2023A Comprehensive Overview of Large Language Modelscited by 639
  4. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  5. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
  6. 2022Towards Reasoning in Large Language Models: A Surveycited by 593
  7. 2023Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Modelscited by 551
  8. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  9. 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
  10. 2024Large Language Models: A Surveycited by 506
  11. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 496
  12. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  13. 2023Cognitive Architectures for Language Agentscited by 422
  14. 2026World Action Models are Zero-shot Policiescited by 411
  15. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 403
  16. 2023Augmented Language Models: a Surveycited by 398
  17. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  18. 2023Language Models can Solve Computer Taskscited by 371
  19. 2022Visual Language Maps for Robot Navigationcited by 362
  20. 2023An Embodied Generalist Agent in 3D Worldcited by 355
  21. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
  22. 2023Chameleon: Plug-and-Play Compositional Reasoning with Large Language Modelscited by 352
  23. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  24. 2023GPT-Driver: Learning to Drive with GPTcited by 344
  25. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  26. 2023Language to Rewards for Robotic Skill Synthesiscited by 307
  27. 2023Tool Learning with Foundation Modelscited by 302
  28. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 298
  29. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 296
  30. 2023SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planningcited by 289
  31. 2023Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memorycited by 288
  32. 2023A Survey on Multimodal Large Language Models for Autonomous Drivingcited by 288
  33. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  34. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  35. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
  36. 2023Challenges and Applications of Large Language Modelscited by 251
  37. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  38. 2023Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaborationcited by 246

50 most cited of 1,057 All 1,057 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
  2. 2025Think, Act, Learn: A Framework for Autonomous Robotic Agents using Closed-Loop Large Language Models
  3. 2026PLanAR: Planning-Language-Grounded Agentic Reasoning for Robot Manipulation
  4. 2025Plan Verification for LLM-Based Embodied Task Completion Agents
  5. 2023Dynamic Planning with a LLM
  6. 2024EnvBridge: Bridging Diverse Environments with Cross-Environment Knowledge Transfer for Embodied AI
  7. 2024Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making
  8. 2024Grounding LLMs For Robot Task Planning Using Closed-loop State Feedback