# Interactive Robot Learning from Verbal Correction

arXiv: https://arxiv.org/abs/2310.17555 · HTML: https://mlsanity.com/paper/2310.17555/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 26 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes](https://mlsanity.com/paper/2305.02301/) (2023)
- [What Matters in Learning from Offline Human Demonstrations for Robot Manipulation](https://mlsanity.com/paper/2108.03298/) (2021)
- [ProgPrompt: Generating Situated Robot Task Plans using Large Language Models](https://mlsanity.com/paper/2209.11302/) (2022)
- [An Algorithmic Perspective on Imitation Learning](https://mlsanity.com/paper/1811.06711/) (2018)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners](https://mlsanity.com/paper/2307.01928/) (2023)
- [ChatGPT for Robotics: Design Principles and Model Abilities](https://mlsanity.com/paper/2306.17582/) (2023)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [HG-DAgger: Interactive Imitation Learning with Human Experts](https://mlsanity.com/paper/1810.02890/) (2018)
- [Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects](https://mlsanity.com/paper/1809.10790/) (2018)
- [Interactive Language: Talking to Robots in Real Time](https://mlsanity.com/paper/2210.06407/) (2022)
- [AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators](https://mlsanity.com/paper/2303.16854/) (2023)
- [Human-in-the-Loop Imitation Learning using Remote Teleoperation](https://mlsanity.com/paper/2012.06733/) (2020)
- [ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learning](https://mlsanity.com/paper/2109.08273/) (2021)
- [Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deployment](https://mlsanity.com/paper/2211.08416/) (2022)
- [Correcting Robot Plans with Natural Language Feedback](https://mlsanity.com/paper/2204.05186/) (2022)
- [Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation](https://mlsanity.com/paper/2112.05251/) (2021)
- [LazyDAgger: Reducing Context Switching in Interactive Imitation Learning](https://mlsanity.com/paper/2104.00053/) (2021)
- [Learning to Generate Better Than Your LLM](https://mlsanity.com/paper/2306.11816/) (2023)
- [KITE: Keypoint-Conditioned Policies for Semantic Manipulation](https://mlsanity.com/paper/2306.16605/) (2023)
- [PATO: Policy Assisted TeleOperation for Scalable Robot Data Collection](https://mlsanity.com/paper/2212.04708/) (2022)
- [Comparing Human-Centric and Robot-Centric Sampling for Robot Deep Learning from Demonstrations](https://mlsanity.com/paper/1610.00850/) (2016)
- [Using Both Demonstrations and Language Instructions to Efficiently Learn Robotic Tasks](https://mlsanity.com/paper/2210.04476/) (2022)

## Built on by (most cited first, 29 of 29)

- [Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Models](https://mlsanity.com/paper/2502.19417/) (2025)
- [Agent AI: Surveying the Horizons of Multimodal Interaction](https://mlsanity.com/paper/2401.03568/) (2024)
- [Yell At Your Robot: Improving On-the-Fly from Language Corrections](https://mlsanity.com/paper/2403.12910/) (2024)
- [VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought](https://mlsanity.com/paper/2406.14596/) (2024)
- [Expanding the Capabilities of Reinforcement Learning via Text Feedback](https://mlsanity.com/paper/2602.02482/) (2026)
- [LLF-Bench: Benchmark for Interactive Learning from Language Feedback](https://mlsanity.com/paper/2312.06853/) (2023)
- [Autoregressive Action Sequence Learning for Robotic Manipulation](https://mlsanity.com/paper/2410.03132/) (2024)
- [Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions](https://mlsanity.com/paper/2410.12773/) (2024)
- [Robix: A Unified Model for Robot Interaction, Reasoning and Planning](https://mlsanity.com/paper/2509.01106/) (2025)
- [Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols](https://mlsanity.com/paper/2512.02787/) (2025)
- [Formalizing Learning from Language Feedback with Provable Guarantees](https://mlsanity.com/paper/2506.10341/) (2025)
- [See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation](https://mlsanity.com/paper/2603.09292/) (2026)
- [Mapping out the Space of Human Feedback for Reinforcement Learning: A Conceptual Framework](https://mlsanity.com/paper/2411.11761/) (2024)
- [Scene Graph-Guided Proactive Replanning for Failure-Resilient Embodied Agent](https://mlsanity.com/paper/2508.11286/) (2025)
- [Ask-to-Clarify: Resolving Instruction Ambiguity through Multi-turn Dialogue](https://mlsanity.com/paper/2509.15061/) (2025)
- [Human-like Navigation in a World Built for Humans](https://mlsanity.com/paper/2509.21189/) (2025)
- [From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledge](https://mlsanity.com/paper/2410.01458/) (2024)
- [Enhancing Q-Learning with Large Language Model Heuristics](https://mlsanity.com/paper/2405.03341/) (2024)
- [LLM-based Interactive Imitation Learning for Robotic Manipulation](https://mlsanity.com/paper/2504.21769/) (2025)
- [Failure Forecasting Boosts Robustness of Sim2Real Rhythmic Insertion Policies](https://mlsanity.com/paper/2507.06519/) (2025)
- [Benchmarking Affordance Generalization with BusyBox](https://mlsanity.com/paper/2602.05441/) (2026)
- [IROSA: Interactive Robot Skill Adaptation using Natural Language](https://mlsanity.com/paper/2603.03897/) (2026)
- [ORPA: Online Residual Policy Adaptation for Robot Manipulation Control with Human Feedback](https://mlsanity.com/paper/2608.17323/) (2026)
- [Ludiscriptscriptstyle 0.1: An Agentic System for Socially Intelligent Robots](https://mlsanity.com/paper/2608.22035/) (2026)
- [The Power of Combined Modalities in Interactive Robot Learning](https://mlsanity.com/paper/2405.07817/) (2024)
- [CueLearner: Bootstrapping and local policy adaptation from relative feedback](https://mlsanity.com/paper/2507.04730/) (2025)
- [Off-Policy Actor-Critic with Sigmoid-Bounded Entropy for Real-World Robot Learning](https://mlsanity.com/paper/2601.15761/) (2026)
- [Robust Instruction Compliance in Cooperative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2605.12655/) (2026)
- [Future Anchored Verification and Online Recovery for World Action Models](https://mlsanity.com/paper/2610.06280/) (2026)

## Abstract (arXiv)

The ability to learn and refine behavior after deployment has become ever more important for robots as we design them to operate in unstructured environments like households. In this work, we design a new learning system based on large language model (LLM), OLAF, that allows everyday users to teach a robot using verbal corrections when the robot makes mistakes, e.g., by saying "Stop what you're doing. You should move closer to the cup." A key feature of OLAF is its ability to update the robot's visuomotor neural policy based on the verbal feedback to avoid repeating mistakes in the future. This is in contrast to existing LLM-based robotic systems, which only follow verbal commands or corrections but not learn from them. We demonstrate the efficacy of our design in experiments where a user teaches a robot to perform long-horizon manipulation tasks both in simulation and on physical hardware, achieving on average 20.0% improvement in policy success rate. Videos and more results are at https://ut-austin-rpl.github.io/olaf/