mlsanityData to 14 Oct 2026

From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following

arXiv 1902.07742 ↗First posted 20 Feb 2019cs.LG stat.MLcited by 67

On the map: Offline RL and imitation learning from data → Language-conditioned policies and VLAs

Abstract (arXiv)

Reinforcement learning is a promising framework for solving control problems, but its use in practical situations is hampered by the fact that reward functions are often difficult to engineer. Specifying goals and tasks for autonomous machines, such as robots, is a significant challenge: conventionally, reward functions and goal states have been used to communicate objectives. But people can communicate objectives to each other simply by describing or demonstrating them. How can we build learning algorithms that will allow us to tell machines what we want them to do? In this work, we investigate the problem of grounding language commands as reward functions using inverse reinforcement learning, and argue that language-conditioned rewards are more transferable than language-conditioned policies to new environments. We propose language-conditioned reward learning (LC-RL), which grounds language commands as a reward function represented by a deep neural network. We demonstrate that our model learns rewards that transfer to novel tasks and environments on realistic, high-dimensional visual environments with natural language commands, whereas directly learning a language-conditioned policy leads to poor performance.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data. Builds on 15 of its 29 references that are in the map. First posted on arXiv 20 Feb 2019.

Builds on

15 of 29 references.

References · 15

  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2016Semantic Scene Completion from a Single Depth Imagecited by 472
  7. 2017Embodied Question Answeringcited by 336
  8. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  9. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  10. 2017Mapping Instructions and Visual Observations to Actions with Reinforcement Learningcited by 81
  11. 2015Listen, Attend, and Walk: Neural Mapping of Navigational Instructions to Action Sequencescited by 50
  12. 2018Reward Learning from Narrated Demonstrationscited by 13
3 more
  1. 2018FollowNet: Robot Navigation by Following Natural Language Directions with Deep Reinforcement Learningcited by 10
  2. 2017Meta Inverse Reinforcement Learning via Maximum Reward Sharing for Human Motion Analysiscited by 4
  3. 2015Robobarista: Object Part based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointcloudscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 2; 2020: 3; 2021: 12; 2022: 13; 2023: 12; 2024: 12; 2025: 9; 2026: 4.

67 papers.

Sort:

By area:

  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
  3. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  4. 2023ChatGPT for Robotics: Design Principles and Model Abilitiescited by 299
  5. 2021BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environmentscited by 165
  6. 2024AgentScope: A Flexible yet Robust Multi-Agent Platformcited by 121
  7. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  8. 2022Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directionscited by 117
  9. 2023Preference Transformer: Modeling Human Preferences using Transformers for RLcited by 79
  10. 2022Robotic Skill Acquisition via Instruction Augmentation with Vision-Language Modelscited by 58
  11. 2024VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thoughtcited by 55
  12. 2022Inferring Rewards from Language in Contextcited by 50
38 more
  1. 2022Improving Intrinsic Exploration with Language Abstractionscited by 49
  2. 2021ELLA: Exploration through Learned Language Abstractioncited by 42
  3. 2020Learning Rewards from Linguistic Feedbackcited by 41
  4. 2022LATTE: LAnguage Trajectory TransformErcited by 40
  5. 2021Vision-Language Navigation: A Survey and Taxonomycited by 34
  6. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  7. 2021LILA: Language-Informed Latent Actionscited by 27
  8. 2021Interactive Learning from Activity Descriptioncited by 26
  9. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  10. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  11. 2024Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferencescited by 23
  12. 2022How to talk so AI will learn: Instructions, descriptions, and autonomycited by 22
  13. 2020Inverse Reinforcement Learning with Natural Language Goalscited by 20
  14. 2022Ground then Navigate: Language-guided Navigation in Dynamic Scenescited by 20
  15. 2022Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformerscited by 17
  16. 2023Inferring the Goals of Communicating Agents from Actions and Instructionscited by 17
  17. 2019Zero-shot Imitation Learning from Demonstrations for Legged Robot Visual Navigationcited by 13
  18. 2019Inverse Reinforcement Learning in Contextual MDPscited by 12
  19. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  20. 2021Generalization in Text-based Games via Hierarchical Reinforcement Learningcited by 11
  21. 2022Discrete Factorial Representations as an Abstraction for Goal Conditioned Reinforcement Learningcited by 11
  22. 2024Problem Solving Through Human-AI Preference-Based Cooperationcited by 11
  23. 2024Vision-Language Navigation with Embodied Intelligence: A Surveycited by 9
  24. 2023Improving Long-Horizon Imitation Through Instruction Predictioncited by 8
  25. 2023Designing Fiduciary Artificial Intelligencecited by 8
  26. 2021SocialAI: Benchmarking Socio-Cognitive Abilities in Deep Reinforcement Learning Agentscited by 7
  27. 2022Linguistic communication as (inverse) reward designcited by 6
  28. 2024Grounded Answers for Multi-agent Decision-making Problem through Generative World Modelcited by 6
  29. 2024Adaptive Reward Design for Reinforcement Learningcited by 6
  30. 2025The Safety Challenge of World Models for Embodied AI Agents: A Reviewcited by 6
  31. 2021Assisted Robust Reward Designcited by 5
  32. 2020Combating False Negatives in Adversarial Imitation Learningcited by 4
  33. 2024Few-Shot Task Learning through Inverse Generative Modelingcited by 4
  34. 2025Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Languagecited by 4
  35. 2023Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learningcited by 3
  36. 2021Mapping Language to Programs using Multiple Reward Components with Inverse Reinforcement Learningcited by 2
  37. 2023A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agentscited by 2
  38. 2025LaGEA: Language Guided Embodied Agents for Robotic Manipulationcited by 2

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics
  2. 2025GoalLadder: Incremental Goal Discovery with Vision-Language Models
  3. 2026From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Models
  4. 2018Learning to Understand Goal Specifications by Modelling Reward
  5. 2023Language Reward Modulation for Pretraining Reinforcement Learning
  6. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedback
  7. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning
  8. 2024RLZero: Direct Policy Inference from Language Without In-Domain Supervision