mlsanityData to 7 Oct 2026

Inverse Reinforcement Learning with Natural Language Goals

arXiv 2008.06924 ↗First posted 16 Aug 2020cs.LG cs.AIcited by 20

On the map: Offline RL and imitation learning from data → Language-conditioned policies and VLAs

Abstract (arXiv)

Humans generally use natural language to communicate task requirements to each other. Ideally, natural language should also be usable for communicating goals to autonomous machines (e.g., robots) to minimize friction in task specification. However, understanding and mapping natural language goals to sequences of states and actions is challenging. Specifically, existing work along these lines has encountered difficulty in generalizing learned policies to new natural language goals and environments. In this paper, we propose a novel adversarial inverse reinforcement learning algorithm to learn a language-conditioned policy and reward function. To improve generalization of the learned policy and reward function, we use a variational goal generator to relabel trajectories and sample diverse goals during training. Our algorithm outperforms multiple baselines by a large margin on a vision-based natural language instruction following dataset (Room-2-Room), demonstrating a promising advance in enabling the use of natural language instructions in specifying agent goals.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data. Builds on 30 of its 43 references that are in the map. First posted on arXiv 16 Aug 2020.

Builds on

30 of 43 references.

Closest ancestors · 2

Same direction, newest first

  1. 2019From Language to Goals: Inverse Reinforcement Learning for Vision-Based Instruction Following20 Feb
  2. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation25 Nov

Other references · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2015Trust Region Policy Optimizationcited by 2,503
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  7. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,013
  10. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  11. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  12. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
16 more
  1. 2018Speaker-Follower Models for Vision-and-Language Navigationcited by 337
  2. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  3. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  4. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  5. 2019Learning to Navigate Unseen Environments: Back Translation with Environmental Dropoutcited by 241
  6. 2019Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Taskscited by 177
  7. 2019Goal-conditioned Imitation Learningcited by 159
  8. 2019A Survey of Reinforcement Learning Informed by Natural Languagecited by 154
  9. 2019Language as an Abstraction for Hierarchical Deep Reinforcement Learningcited by 138
  10. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  11. 2017Imitating Driver Behavior with Generative Adversarial Networkscited by 92
  12. 2018Learning to Understand Goal Specifications by Modelling Rewardcited by 80
  13. 2016First-Person Activity Forecasting with Online Inverse Reinforcement Learningcited by 40
  14. 2019rlpyt: A Research Code Base for Deep Reinforcement Learning in PyTorchcited by 38
  15. 2018Inverse reinforcement learning for video gamescited by 26
  16. 2019HIGhER : Improving instruction following with Hindsight Generation for Experience Replaycited by 13

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 6; 2022: 1; 2023: 5; 2024: 5; 2025: 2.

20 papers.

Sort:

By area:

  1. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  2. 2024LLM-based Multi-Agent Reinforcement Learning: Current and Future Directionscited by 67
  3. 2020Learning Rewards from Linguistic Feedbackcited by 41
  4. 2021Vision-Language Navigation: A Survey and Taxonomycited by 33
  5. 2021Interactive Learning from Activity Descriptioncited by 26
  6. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  7. 2024Direct Preference Optimization With Unobserved Preference Heterogeneity: The Necessity of Ternary Preferencescited by 23
  8. 2022Language and Culture Internalisation for Human-Like Autotelic AIcited by 14
  9. 2021A Framework for Learning to Request Rich and Contextually Useful Information from Humanscited by 12
  10. 2021Grounding Spatio-Temporal Language with Transformerscited by 7
  11. 2024Grounded Answers for Multi-agent Decision-making Problem through Generative World Modelcited by 6
  12. 2023Curricular Subgoals for Inverse Reinforcement Learningcited by 5
8 more
  1. 2021Combining Learning from Human Feedback and Knowledge Engineering to Solve Hierarchical Tasks in Minecraftcited by 3
  2. 2021Towards Teachable Autotelic Agentscited by 2
  3. 2023Natural Language Robot Programming: NLP integrated with autonomous robotic graspingcited by 2
  4. 2023Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learningcited by 1
  5. 2025Direct Preference Optimization with Unobserved Preference Heterogeneity: The Necessity of Ternary Preferencescited by 1
  6. 2024Interpretable Robotic Manipulation from Languagecited by 0
  7. 2024OCALM: Object-Centric Assessment with Language Modelscited by 0
  8. 2025NatSGLD: A Dataset with Speech, Gesture, Logic, and Demonstration for Robot Learning in Natural Human-Robot Interactioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunities
  2. 2024Natural Language Reinforcement Learning
  3. 2026Inverse RL Helps Align AI by Imitating Humans
  4. 2025GoalLadder: Incremental Goal Discovery with Vision-Language Models
  5. 2026ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer
  6. 2025Masked IRL: LLM-Guided Reward Disambiguation from Demonstrations and Language
  7. 2023Is Feedback All You Need? Leveraging Natural Language Feedback in Goal-Conditioned Reinforcement Learning
  8. 2024Inverse-Q*: Token Level Reinforcement Learning for Aligning Large Language Models Without Preference Data