mlsanityData to 7 Oct 2026

Hierarchical Few-Shot Imitation with Skill Transition Models

arXiv 2107.08981 ↗First posted 19 Jul 2021cs.LG cs.AI cs.ROcited by 37

Not on the map. Why?

Abstract (arXiv)

A desirable property of autonomous agents is the ability to both solve long-horizon problems and generalize to unseen tasks. Recent advances in data-driven skill learning have shown that extracting behavioral priors from offline data can enable agents to solve challenging long-horizon tasks with reinforcement learning. However, generalization to tasks unseen during behavioral prior training remains an outstanding challenge. To this end, we present Few-shot Imitation with Skill Transition Models (FIST), an algorithm that extracts skills from offline data and utilizes them to generalize to unseen tasks given a few downstream demonstrations. FIST learns an inverse skill dynamics model, a distance function, and utilizes a semi-parametric approach for imitation. We show that FIST is capable of generalizing to new tasks and substantially outperforms prior baselines in navigation experiments requiring traversing unseen parts of a large maze and 7-DoF robotic arm experiments requiring manipulating previously unseen objects in a kitchen.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Builds on 30 of its 43 references that are in the map. First posted on arXiv 19 Jul 2021.

Builds on

30 of 43 references.

References · 30

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  8. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  9. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  10. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  11. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  12. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
18 more
  1. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  2. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  3. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 380
  4. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
  5. 2020Decoupling Representation Learning from Reinforcement Learningcited by 263
  6. 2017One-Shot Imitation Learningcited by 232
  7. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  8. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 220
  9. 2014Robots that can adapt like animalscited by 203
  10. 2018Semi-parametric Topological Memory for Navigationcited by 203
  11. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learningcited by 164
  12. 2020Parrot: Data-Driven Behavioral Priors for Reinforcement Learningcited by 136
  13. 2020ViNG: Learning Open-World Navigation with Visual Goalscited by 77
  14. 2020Neural Dynamic Policies for End-to-End Sensorimotor Learningcited by 56
  15. 2018A survey on policy search algorithms for learning robot controllers in a handful of trialscited by 33
  16. 2020Sparse Graphical Memory for Robust Planningcited by 30
  17. 2020Hallucinative Topological Memory for Zero-Shot Visual Planningcited by 25
  18. 2017Bayesian Optimization with Automatic Prior Selection for Data-Efficient Direct Policy Searchcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 10; 2023: 8; 2024: 9; 2025: 8; 2026: 1.

37 papers.

Sort:

By area:

  1. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  2. 2024RT-H: Action Hierarchies Using Languagecited by 204
  3. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 104
  4. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  5. 2024SkillMimicGen: Automated Demonstration Generation for Efficient Skill Learning and Deploymentcited by 48
  6. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  7. 2025Sim-and-Real Co-Training: A Simple Recipe for Vision-Based Robotic Manipulationcited by 42
  8. 2021TRAIL: Near-Optimal Imitation Learning with Suboptimal Datacited by 39
  9. 2022Chain of Thought Imitation with Procedure Cloningcited by 30
  10. 2023Demonstration-Guided Reinforcement Learning with Efficient Exploration for Task Automation of Surgical Robotcited by 21
  11. 2024Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulationcited by 19
  12. 2024Incremental Learning of Retrievable Skills For Efficient Continual Task Adaptationcited by 19
25 more
  1. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  2. 2022Cross-Domain Transfer via Semantic Skill Imitationcited by 17
  3. 2023Skill Transformer: A Monolithic Policy for Mobile Manipulationcited by 15
  4. 2022Towards Informed Design and Validation Assistance in Computer Games Using Imitation Learningcited by 13
  5. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learningcited by 13
  6. 2023PLEX: Making the Most of the Available Data for Robotic Manipulation Pretrainingcited by 9
  7. 2023Learning to Discover Skills through Guidancecited by 8
  8. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  9. 2024Robust Policy Learning via Offline Skill Diffusioncited by 7
  10. 2022Intra-agent speech permits zero-shot task acquisitioncited by 5
  11. 2023Skill Disentanglement for Imitation Learning from Suboptimal Demonstrationscited by 5
  12. 2023Offline Imitation from Observation via Primal Wasserstein State Occupancy Matchingcited by 5
  13. 2022CEIP: Combining Explicit and Implicit Priors for Reinforcement Learning with Demonstrationscited by 4
  14. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectoriescited by 3
  15. 2024Meta-Controller: Few-Shot Imitation of Unseen Embodiments and Tasks in Continuous Controlcited by 3
  16. 2025Bridging Perception and Action: Spatially-Grounded Mid-Level Representations for Robot Generalizationcited by 2
  17. 2025Goal-Oriented Skill Abstraction for Offline Multi-Task Reinforcement Learningcited by 2
  18. 2025Policy Compatible Skill Incremental Learning via Lazy Learning Interfacecited by 2
  19. 2024Skill-Enhanced Reinforcement Learning Acceleration from Heterogeneous Demonstrationscited by 1
  20. 2025NBDI: A Simple and Effective Termination Condition for Skill Extraction from Task-Agnostic Demonstrationscited by 1
  21. 2025In-Context Policy Adaptation via Cross-Domain Skill Diffusioncited by 1
  22. 2022A Versatile Agent for Fast Learning from Human Instructorscited by 0
  23. 2024Interpretable Robotic Manipulation from Languagecited by 0
  24. 2025Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Datacited by 0
  25. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learningcited by 0