mlsanityData to 7 Oct 2026

RoboCLIP: One Demonstration is Enough to Learn Robot Policies

arXiv 2310.07899 ↗First posted 11 Oct 2023cs.AI cs.ROcited by 128

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Reward specification is a notoriously difficult problem in reinforcement learning, requiring extensive expert supervision to design robust reward functions. Imitation learning (IL) methods attempt to circumvent these problems by utilizing expert demonstrations but typically require a large number of in-domain expert demonstrations. Inspired by advances in the field of Video-and-Language Models (VLMs), we present RoboCLIP, an online imitation learning method that uses a single demonstration (overcoming the large data requirement) in the form of a video demonstration or a textual description of the task to generate rewards without manual reward function design. Additionally, RoboCLIP can also utilize out-of-domain demonstrations, like videos of humans solving the task for reward generation, circumventing the need to have the same demonstration and deployment domains. RoboCLIP utilizes pretrained VLMs without any finetuning for reward generation. Reinforcement learning agents trained with RoboCLIP rewards demonstrate 2-3 times higher zero-shot performance than competing imitation learning methods on downstream robot manipulation tasks, doing so using only one video/text demonstration.

Key facts. Cited by 128 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 39 of its 46 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

39 of 46 references.

References · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  4. 2017Deep reinforcement learning from human preferencescited by 3,846
  5. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2016Generative Adversarial Imitation Learningcited by 1,576
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  9. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  10. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  11. 2017Deep Image Priorcited by 1,097
  12. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
27 more
  1. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  2. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  3. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  4. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  5. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  6. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  7. 2022Expanding Language-Image Pretrained Models for General Video Recognitioncited by 320
  8. 2021PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-trainingcited by 298
  9. 2019RoboNet: Large-Scale Multi-Robot Learningcited by 268
  10. 2023Reward Design with Language Modelscited by 257
  11. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  12. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  13. 2017Inverse Reward Designcited by 246
  14. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  15. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  16. 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 107
  17. 2023Do Embodied Agents Dream of Pixelated Sheep: Embodied Decision Making using Language Guided World Modellingcited by 88
  18. 2022When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?cited by 86
  19. 2022Few-Shot Preference Learning for Human-in-the-Loop RLcited by 85
  20. 2023Teach a Robot to FISH: Versatile Imitation from One Minute of Demonstrationscited by 74
  21. 2020Learning Reward Functions from Diverse Sources of Human Feedback: Optimally Integrating Demonstrations and Preferencescited by 71
  22. 2020Active Preference-Based Gaussian Process Regression for Reward Learningcited by 70
  23. 2019Asking Easy Questions: A User-Friendly Approach to Active Reward Learningcited by 67
  24. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 66
  25. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 61
  26. 2021Learning Multimodal Rewards from Rankingscited by 41
  27. 2022Policy Adaptation from Foundation Model Feedbackcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 41; 2025: 41; 2026: 41.

128 papers.

Sort:

By area:

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  3. 2024Agent AI: Surveying the Horizons of Multimodal Interactioncited by 167
  4. 2025AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challengescited by 167
  5. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learningcited by 155
  6. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 147
  7. 2023Large Language Models for Robotics: A Surveycited by 132
  8. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  9. 2024Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequencescited by 93
  10. 2023Vision-Language Models as a Source of Rewardscited by 65
  11. 2024RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulationcited by 55
  12. 2024Sora as a World Model? A Complete Survey on Text-to-Video Generationcited by 53
38 more
  1. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 51
  2. 2024VLM-RL: A Unified Vision Language Models and Reinforcement Learning Framework for Safe Autonomous Drivingcited by 49
  3. 2024Towards Generalist Robot Learning from Internet Video: A Surveycited by 40
  4. 2024Code as Reward: Empowering Reinforcement Learning with VLMscited by 38
  5. 2024FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learningcited by 32
  6. 2024LLM-Empowered State Representation for Reinforcement Learningcited by 28
  7. 2024EPO: Hierarchical LLM Agents with Environment Preference Optimizationcited by 28
  8. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  9. 2025SIMA 2: A Generalist Embodied Agent for Virtual Worldscited by 26
  10. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 24
  11. 2024Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulationcited by 22
  12. 2024Real-World Offline Reinforcement Learning from Vision Language Model Feedbackcited by 20
  13. 2025EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Modelscited by 20
  14. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 18
  15. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Roboticscited by 17
  16. 2024ExpertAF: Expert Actionable Feedback from Videocited by 17
  17. 2025Enhancing Rating-Based Reinforcement Learning to Effectively Leverage Feedback from Large Vision-Language Modelscited by 15
  18. 2023LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teacherscited by 14
  19. 2024Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Modelscited by 14
  20. 2024EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Datacited by 13
  21. 2025Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Modelscited by 13
  22. 2026Towards Long-Lived Robots: Continual Learning VLA Models via Reinforcement Fine-Tuningcited by 11
  23. 2026ARM: Advantage Reward Modeling for Long-Horizon Manipulationcited by 10
  24. 2025SENSEI: Semantic Exploration Guided by Foundation Models to Learn Versatile World Modelscited by 9
  25. 2024Robot Policy Learning with Temporal Optimal Transport Rewardcited by 8
  26. 2024ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Roboticscited by 8
  27. 2025VLP: Vision-Language Preference Learning for Embodied Manipulationcited by 8
  28. 2025FOUNDER: Grounding Foundation Models in World Models for Open-Ended Embodied Decision Makingcited by 8
  29. 2025Reflection-Based Task Adaptation for Self-Improving VLAcited by 8
  30. 2025LLM-Driven Stationarity-Aware Expert Demonstrations for Multi-Agent Reinforcement Learning in Mobile Systemscited by 8
  31. 2024Text-Aware Diffusion for Policy Learningcited by 7
  32. 2024RLZero: Direct Policy Inference from Language Without In-Domain Supervisioncited by 7
  33. 2024Online Preference-based Reinforcement Learning with Self-augmented Feedback from Large Language Modelcited by 7
  34. 2025Imitation Learning from a Single Temporally Misaligned Videocited by 7
  35. 2025MILE: Model-based Intervention Learningcited by 7
  36. 2025TeViR: Text-to-Video Reward with Diffusion Models for Efficient Reinforcement Learningcited by 7
  37. 2024LORD: Large Models based Opposite Reward Design for Autonomous Drivingcited by 6
  38. 2024KOI: Accelerating Online Imitation Learning via Hybrid Key-state Guidancecited by 6

50 most cited of 128 All 128 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Reward
  2. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models
  3. 2023A Survey of Demonstration Learning
  4. 2019SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
  5. 2022Reinforcement learning with Demonstrations from Mismatched Task under Sparse Reward
  6. 2024Learning Manipulation Skills through Robot Chain-of-Thought with Sparse Failure Guidance
  7. 2022STIR2: Reward Relabelling for combined Reinforcement and Imitation Learning on sparse-reward tasks
  8. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges