mlsanityData to 7 Oct 2026

Prospection: Interpretable Plans From Language By Predicting the Future

arXiv 1903.08309 ↗First posted 20 Mar 2019cs.AI cs.CL cs.LGcited by 21

Not on the map. Why?

Abstract (arXiv)

High-level human instructions often correspond to behaviors with multiple implicit steps. In order for robots to be useful in the real world, they must be able to to reason over both motions and intermediate goals implied by human instructions. In this work, we propose a framework for learning representations that convert from a natural-language command to a sequence of intermediate goals for execution on a robot. A key feature of this framework is prospection, training an agent not just to correctly execute the prescribed command, but to predict a horizon of consequences of an action before taking it. We demonstrate the fidelity of plans generated by our framework when interpreting real, crowd-sourced natural language commands for a robot in simulated scenes.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 6; 2023: 4; 2024: 1; 2025: 1; 2026: 1.

21 papers.

Sort:

By area:

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  3. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  4. 2021A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Executioncited by 91
  5. 2022Correcting Robot Plans with Natural Language Feedbackcited by 68
  6. 2021StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objectscited by 57
  7. 2021SORNet: Spatial Object-Centric Representations for Sequential Manipulationcited by 48
  8. 2023Distilling Script Knowledge from Large Language Models for Constrained Language Planningcited by 45
  9. 2021Language Grounding with 3D Objectscited by 43
  10. 2021Latent Execution for Neural Program Synthesiscited by 35
  11. 2022LISA: Learning Interpretable Skill Abstractions from Languagecited by 30
  12. 2023Vision-Language Interpreter for Robot Task Planningcited by 29
9 more
  1. 2021Trajectory Prediction with Linguistic Representationscited by 13
  2. 2022Policy Adaptation from Foundation Model Feedbackcited by 10
  3. 2022Learning Neuro-symbolic Programs for Language Guided Robot Manipulationcited by 9
  4. 2024Robi Butler: Multimodal Remote Interaction with a Household Robot Assistantcited by 6
  5. 2023Spatial-Language Attention Policies for Efficient Robot Learningcited by 3
  6. 2021DASH: Modularized Human Manipulation Simulation with Vision and Language for Embodied AIcited by 2
  7. 2023tagE: Enabling an Embodied Agent to Understand Human Instructionscited by 1
  8. 2025Automating Complex Document Workflows via Stepwise and Rollback-Enabled Operation Orchestrationcited by 0
  9. 2026AI-IoT-Robotics Integration: Survey of Frameworks, Emerging Trends, and the Path Toward Connected Roboticscited by 0