mlsanityData to 7 Oct 2026

Improving Long-Horizon Imitation Through Instruction Prediction

arXiv 2306.12554 ↗First posted 21 Jun 2023cs.LG cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

Complex, long-horizon planning and its combinatorial nature pose steep challenges for learning-based agents. Difficulties in such settings are exacerbated in low data regimes where over-fitting stifles generalization and compounding errors hurt accuracy. In this work, we explore the use of an often unused source of auxiliary supervision: language. Inspired by recent advances in transformer-based models, we train agents with an instruction prediction loss that encourages learning temporally extended representations that operate at a high level of abstraction. Concretely, we demonstrate that instruction modeling significantly improves performance in planning environments when training with a limited number of demonstrations on the BabyAI and Crafter benchmarks. In further analysis we find that instruction modeling is most important for tasks that require complex reasoning, while understandably offering smaller gains in environments that require simple plans. More details and code can be found at https://github.com/jhejna/instruction-prediction.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 21 Jun 2023.

Builds on

42 of 60 references.

References · 42

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  3. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,143
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  5. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 773
  6. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  7. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  8. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 429
  9. 2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 400
  10. 2019DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Framescited by 352
  11. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  12. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
30 more
  1. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  2. 2019Stabilizing Transformers for Reinforcement Learningcited by 262
  3. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 238
  4. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  5. 2021Episodic Transformer for Vision-and-Language Navigationcited by 155
  6. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  7. 2019Unsupervised State Representation Learning in Ataricited by 144
  8. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  9. 2020Language-Conditioned Imitation Learning for Robot Manipulation Taskscited by 136
  10. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 117
  11. 2020Topological Planning with Transformers for Vision-and-Language Navigationcited by 115
  12. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  13. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  14. 2017Zero-Shot Task Generalization with Multi-Task Deep Reinforcement Learningcited by 97
  15. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  16. 2017Hierarchical and Interpretable Skill Acquisition in Multi-task Reinforcement Learningcited by 77
  17. 2017Learning with Latent Languagecited by 75
  18. 2017Gated-Attention Architectures for Task-Oriented Language Groundingcited by 72
  19. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Followingcited by 67
  20. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  21. 2021Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learningcited by 45
  22. 2021VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputscited by 42
  23. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  24. 2020Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learningcited by 32
  25. 2019Hierarchical Decision Making by Generating and Following Natural Language Instructionscited by 30
  26. 2019ACTRCE: Augmenting Experience via Teacher's Advice For Multi-Goal Reinforcement Learningcited by 17
  27. 2021Zero-shot Task Adaptation using Natural Languagecited by 15
  28. 2022An Empirical Investigation of Representation Learning for Imitationcited by 15
  29. 2018Translating Navigation Instructions in Natural Language to a High-Level Plan for Behavioral Robot Navigationcited by 9
  30. 2020Following Instructions by Imagining and Reaching Visual Goalscited by 6

Built on by

Cited by year

Citing papers per year: 2024: 4; 2025: 4.

8 papers.

  1. 2024RT-H: Action Hierarchies Using Languagecited by 204
  2. 2024FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learningcited by 32
  3. 2025RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Codecited by 28
  4. 2024How Can LLM Guide RL? A Value-Based Approachcited by 13
  5. 2025Embodied Navigation with Auxiliary Task of Action Description Predictioncited by 3
  6. 2024SEAL: SEmantic-Augmented Imitation Learning via Language Modelcited by 1
  7. 2025Inductive Biases for Zero-shot Systematic Generalization in Language-informed Reinforcement Learningcited by 0
  8. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advancescited by 0