# Guiding Policies with Language via Meta-Learning

arXiv: https://arxiv.org/abs/1811.07882 · HTML: https://mlsanity.com/paper/1811.07882/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 19 Nov 2018.

## Placed in

- Offline RL and imitation learning from data / [Language-conditioned policies and VLAs](https://mlsanity.com/offlinerl/language-conditioned-policies-and-vlas/)

## Built on by (most cited first, 30 of 35)

- [Interactive Language: Talking to Robots in Real Time](https://mlsanity.com/paper/2210.06407/) (2022)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotation](https://mlsanity.com/paper/2109.01115/) (2021)
- [Few-Shot Preference Learning for Human-in-the-Loop RL](https://mlsanity.com/paper/2212.03363/) (2022)
- [Vision-based Navigation with Language-based Assistance via Imitation Learning with Indirect Intervention](https://mlsanity.com/paper/1812.04155/) (2018)
- [When Can Models Learn From Explanations? A Formal Framework for Understanding the Roles of Explanation Data](https://mlsanity.com/paper/2102.02201/) (2021)
- [Correcting Robot Plans with Natural Language Feedback](https://mlsanity.com/paper/2204.05186/) (2022)
- ["No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy](https://mlsanity.com/paper/2301.02555/) (2023)
- [Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning](https://mlsanity.com/paper/2101.07393/) (2021)
- [Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections](https://mlsanity.com/paper/2311.10678/) (2023)
- [Fixing Model Bugs with Natural Language Patches](https://mlsanity.com/paper/2211.03318/) (2022)
- [Distilling Internet-Scale Vision-Language Models into Embodied Agents](https://mlsanity.com/paper/2301.12507/) (2023)
- [LILA: Language-Informed Latent Actions](https://mlsanity.com/paper/2111.03205/) (2021)
- [Leveraging Language for Accelerated Learning of Tool Manipulation](https://mlsanity.com/paper/2206.13074/) (2022)
- [Dexterous Manipulation from Images: Autonomous Real-World RL via Substep Guidance](https://mlsanity.com/paper/2212.09902/) (2022)
- [Language-Conditioned Imitation Learning with Base Skill Priors under Unstructured Data](https://mlsanity.com/paper/2305.19075/) (2023)
- [REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulation](https://mlsanity.com/paper/2309.03322/) (2023)
- [Preference-Conditioned Language-Guided Abstraction](https://mlsanity.com/paper/2402.03081/) (2024)
- [SILG: The Multi-environment Symbolic Interactive Language Grounding Benchmark](https://mlsanity.com/paper/2110.10661/) (2021)
- [Learning with Language-Guided State Abstractions](https://mlsanity.com/paper/2402.18759/) (2024)
- [Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty](https://mlsanity.com/paper/2312.01097/) (2023)
- [Learning from Symmetry: Meta-Reinforcement Learning with Symmetrical Behaviors and Language Instructions](https://mlsanity.com/paper/2209.10656/) (2022)
- [SalKG: Learning From Knowledge Graph Explanations for Commonsense Reasoning](https://mlsanity.com/paper/2104.08793/) (2021)
- [Growing with Your Embodied Agent: A Human-in-the-Loop Lifelong Code Generation Framework for Long-Horizon Manipulation Skills](https://mlsanity.com/paper/2509.18597/) (2025)
- [Targeted Data Acquisition for Evolving Negotiation Agents](https://mlsanity.com/paper/2106.07728/) (2021)
- [lilGym: Natural Language Visual Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2211.01994/) (2022)
- [Teachable Reinforcement Learning via Advice Distillation](https://mlsanity.com/paper/2203.11197/) (2022)
- [Entity Divider with Language Grounding in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2210.13942/) (2022)
- [Yes, this Way! Learning to Ground Referring Expressions into Actions with Intra-episodic Feedback from Supportive Teachers](https://mlsanity.com/paper/2305.12880/) (2023)
- [Ask Before You Act: Generalising to Novel Environments by Asking Questions](https://mlsanity.com/paper/2209.04665/) (2022)

## Abstract (arXiv)

Behavioral skills or policies for autonomous agents are conventionally learned from reward functions, via reinforcement learning, or from demonstrations, via imitation learning. However, both modes of task specification have their disadvantages: reward functions require manual engineering, while demonstrations require a human expert to be able to actually perform the task in order to generate the demonstration. Instruction following from natural language instructions provides an appealing alternative: in the same way that we can specify goals to other humans simply by speaking or writing, we would like to be able to specify tasks for our machines. However, a single instruction may be insufficient to fully communicate our intent or, even if it is, may be insufficient for an autonomous agent to actually understand how to perform the desired task. In this work, we propose an interactive formulation of the task specification problem, where iterative language corrections are provided to an autonomous agent, guiding it in acquiring the desired skill. Our proposed language-guided policy learning algorithm can integrate an instruction and a sequence of corrections to acquire new skills very quickly. In our experiments, we show that this method can enable a policy to follow instructions and corrections for simulated navigation and manipulation tasks, substantially outperforming direct, non-interactive instruction following.