mlsanityData to 14 Oct 2026

Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets

arXiv 2304.08742 ↗First posted 18 Apr 2023cs.RO cs.AI cs.LGcited by 58

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Enabling robots to learn novel visuomotor skills in a data-efficient manner remains an unsolved problem with myriad challenges. A popular paradigm for tackling this problem is through leveraging large unlabeled datasets that have many behaviors in them and then adapting a policy to a specific task using a small amount of task-specific human supervision (i.e. interventions or demonstrations). However, how best to leverage the narrow task-specific supervision and balance it with offline data remains an open question. Our key insight in this work is that task-specific data not only provides new data for an agent to train on but can also inform the type of prior data the agent should use for learning. Concretely, we propose a simple approach that uses a small amount of downstream expert data to selectively query relevant behaviors from an offline, unlabeled dataset (including many sub-optimal behaviors). The agent is then jointly trained on the expert and queried data. We observe that our method learns to query only the relevant transitions to the task, filtering out sub-optimal or task-irrelevant data. By doing so, it is able to learn more effectively from the mix of task-specific and offline data compared to naively mixing the data or only using the task-specific data. Furthermore, we find that our simple querying approach outperforms more complex goal-conditioned methods by 20% across simulated and real robotic manipulation tasks from images. See https://sites.google.com/view/behaviorretrieval for videos and code.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 38 of its 48 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

38 of 48 references.

Closest ancestors · 9

Same direction, newest first

  1. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learning20 Oct
  2. 2022Eliciting Compatible Demonstrations for Multi-Human Imitation Learning14 Oct
  3. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Training30 Sep
  4. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb
  5. 2021Towards More Generalizable One-shot Visual Imitation Learning26 Oct
  6. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstration13 May
  7. 2020Human-in-the-Loop Imitation Learning using Remote Teleoperation12 Dec
  8. 2018One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks25 Oct
  9. 2018Task-Embedded Control Networks for Few-Shot Imitation Learning8 Oct

Other references · 29

  1. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  3. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  4. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  5. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  6. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  7. 2019Learning Latent Plans from Playcited by 316
  8. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 266
  9. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  10. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  11. 2022Masked Visual Pre-training for Motor Controlcited by 248
  12. 2022Real-World Robot Learning with Masked Visual Pre-trainingcited by 245
17 more
  1. 2017One-Shot Imitation Learningcited by 233
  2. 2018Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Controlcited by 230
  3. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  4. 2017DART: Noise Injection for Robust Imitation Learningcited by 191
  5. 2019Goal-conditioned Imitation Learningcited by 162
  6. 2022The Unsurprising Effectiveness of Pre-Trained Vision Models for Controlcited by 155
  7. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 138
  8. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  9. 2022Pre-Training for Robots: Offline RL Enables Learning New Tasks from a Handful of Trialscited by 78
  10. 2022Interactive Imitation Learning in Robotics: A Surveycited by 55
  11. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  12. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
  13. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  14. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 28
  15. 2022PLATO: Predicting Latent Affordances Through Object-Centric Playcited by 13
  16. 2022Coarse-to-fine Q-attention with Tree Expansioncited by 9
  17. 2022ReIL: A Framework for Reinforced Intervention-based Imitation Learningcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 6; 2024: 12; 2025: 18; 2026: 22.

58 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  3. 2023Data Quality in Imitation Learningcited by 75
  4. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  5. 2024RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulationcited by 58
  6. 2024Re-Mix: Optimizing Data Mixtures for Large Scale Imitation Learningcited by 47
  7. 2024DINOBot: Robot Manipulation via Retrieval and Alignment with Vision Foundation Modelscited by 44
  8. 2024STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learningcited by 44
  9. 2024Surgical Robot Transformer (SRT): Imitation Learning for Surgical Taskscited by 40
  10. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
  11. 2024FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learningcited by 33
  12. 2025DataMIL: Selecting Data for Robot Imitation Learning with Datamodelscited by 28
38 more
  1. 2025RICL: Adding In-Context Adaptability to Pre-Trained Vision-Language-Action Modelscited by 22
  2. 2025What Matters in Learning from Large-Scale Datasets for Robot Manipulationcited by 20
  3. 2025Curating Demonstrations using Online Experiencecited by 18
  4. 2023Learning to Discern: Imitating Heterogeneous Human Demonstrations with Preference and Representation Learningcited by 15
  5. 2023On the Effectiveness of Retrieval, Alignment, and Replay in Manipulationcited by 15
  6. 2025Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrationscited by 15
  7. 2024Learning from Imperfect Demonstrations with Self-Supervision for Robotic Manipulationcited by 14
  8. 2024Offline Imitation Learning Through Graph Search and Retrievalcited by 14
  9. 2023Selective Sampling and Imitation Learning via Online Regressioncited by 12
  10. 2025Guiding Data Collection via Factored Scaling Curvescited by 12
  11. 2025Data Retrieval with Importance Weights for Few-Shot Imitation Learningcited by 10
  12. 2026Uni-Skill: Building Self-Evolving Skill Repository for Generalizable Robotic Manipulationcited by 9
  13. 2023NOIR: Neural Signal Operated Intelligent Robots for Everyday Activitiescited by 8
  14. 2025cVLA: Towards Efficient Camera-Space VLAscited by 8
  15. 2025RT-Cache: Training-Free Retrieval for Real-Time Manipulationcited by 6
  16. 2025COLLAGE: Adaptive Fusion-based Retrieval for Augmented Policy Learningcited by 5
  17. 2026Retrieve, Don't Retrain: Extending Vision Language Action Models to New Tasks at Test Timecited by 5
  18. 2025Factorizing Diffusion Policies for Observation Modality Prioritizationcited by 3
  19. 2025Policy Compatible Skill Incremental Learning via Lazy Learning Interfacecited by 3
  20. 2026Learning from the Best: Smoothness-Driven Metrics for Data Quality in Imitation Learningcited by 3
  21. 2026SiMDex: Mining Similar Egocentric Videos for Cross-Embodiment Dexterous Manipulationcited by 3
  22. 2025Robotic Paper Wrapping by Learning Force Controlcited by 2
  23. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learningcited by 2
  24. 2025Beyond Success: Refining Elegant Robot Manipulation from Mixed-Quality Data via Just-in-Time Interventioncited by 2
  25. 2026Tether: Autonomous Functional Play with Correspondence-Driven Trajectory Warpingcited by 2
  26. 2026Grounding Robot Generalization in Training Data via Retrieval-Augmented VLMscited by 2
  27. 2024ReMoBot: Retrieval-Based Few-Shot Imitation Learning for Mobile Manipulation with Vision Foundation Modelscited by 1
  28. 2024Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptationcited by 1
  29. 2025Robust Offline Imitation Learning Through State-level Trajectory Stitchingcited by 1
  30. 2025Retrieve-Augmented Generation for Speeding up Diffusion Policy without Additional Trainingcited by 1
  31. 2026Quality over Quantity: Demonstration Curation via Influence Functions for Data-Centric Robot Learningcited by 1
  32. 2026ReGIL: Retrieval-Guided Imitation Learning from a Single Demonstrationcited by 1
  33. 2026Difference-Aware Retrieval Policies for Imitation Learningcited by 1
  34. 2026ATHENA: Accelerated Multi-Task Heterogeneous Influence Functions for Robot Data Curationcited by 1
  35. 2026Retrieve in Time, Correct in Frequencycited by 1
  36. 2026RoboTok: A Scalable Data Engine for Internet Demonstration Video Retrieval and Dexterous Manipulation Learningcited by 1
  37. 2025NOIR 2.0: Neural Signal Operated Intelligent Robots for Everyday Activitiescited by 0
  38. 2026Tactile Memory with Soft Robot: Robust Object Insertion via Masked Encoding and Soft Wristcited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  2. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learning
  3. 2020Offline Learning from Demonstrations and Unlabeled Experience
  4. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  5. 2022Generalization with Lossy Affordances: Leveraging Broad Offline Data for Learning Visuomotor Tasks
  6. 2025SCIZOR: A Self-Supervised Approach to Data Curation for Large-Scale Imitation Learning
  7. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos
  8. 2021Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage