mlsanityData to 7 Oct 2026

Watch, Try, Learn: Meta-Learning from Demonstrations and Reward

arXiv 1906.03352 ↗First posted 7 Jun 2019cs.LG cs.AI stat.MLcited by 26

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning allows agents to learn complex behaviors from demonstrations. However, learning a complex vision-based task may require an impractical number of demonstrations. Meta-imitation learning is a promising approach towards enabling agents to learn a new task from one or a few demonstrations by leveraging experience from learning similar tasks. In the presence of task ambiguity or unobserved dynamics, demonstrations alone may not provide enough information; an agent must also try the task to successfully infer a policy. In this work, we propose a method that can learn to learn from both demonstrations and trial-and-error experience with sparse reward feedback. In comparison to meta-imitation, this approach enables the agent to effectively and efficiently improve itself autonomously beyond the demonstration data. In comparison to meta-reinforcement learning, we can scale to substantially broader distributions of tasks, as the demonstration reduces the burden of exploration. Our experiments show that our method significantly outperforms prior approaches on a set of challenging, vision-based control tasks.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 24 of its 43 references that are in the map. First posted on arXiv 7 Jun 2019.

Builds on

24 of 43 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Truncated Horizon Policy Search: Combining Reinforcement Learning & Imitation Learning29 May

Other references · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  4. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  5. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  6. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  7. 2017A Tutorial on Thompson Samplingcited by 420
  8. 2019Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variablescited by 372
  9. 2016Learning to reinforcement learncited by 369
  10. 2017Overcoming Exploration in Reinforcement Learning with Demonstrationscited by 322
  11. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  12. 2017One-Shot Imitation Learningcited by 232
11 more
  1. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  2. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 149
  3. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  4. 2018Meta-Reinforcement Learning of Structured Exploration Strategiescited by 128
  5. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  6. 2018Evolved Policy Gradientscited by 86
  7. 2018Task-Embedded Control Networks for Few-Shot Imitation Learningcited by 65
  8. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  9. 2018Some Considerations on Learning to Explore via Meta-Reinforcement Learningcited by 43
  10. 2017Learning to Learn: Meta-Critic Networks for Sample Efficient Learningcited by 28
  11. 2018One-Shot High-Fidelity Imitation: Training Large-Scale Deep Nets with RLcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 1; 2021: 5; 2022: 7; 2023: 8; 2024: 2; 2025: 1; 2026: 2.

26 papers.

Sort:

By area:

  1. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  2. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  3. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  4. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 56
  5. 2021Towards More Generalizable One-shot Visual Imitation Learningcited by 48
  6. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  7. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  8. 2023EC^2: Emergent Communication for Embodied Controlcited by 10
  9. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  10. 2023PLEX: Making the Most of the Available Data for Robotic Manipulation Pretrainingcited by 9
  11. 2022Learning from Symmetry: Meta-Reinforcement Learning with Symmetrical Behaviors and Language Instructionscited by 8
  12. 2022One-Shot Learning from a Demonstration with Hierarchical Latent Languagecited by 6
14 more
  1. 2022Intra-agent speech permits zero-shot task acquisitioncited by 5
  2. 2023Continual Robot Learning using Self-Supervised Task Inferencecited by 5
  3. 2022Abstract-to-Executable Trajectory Translation for One-Shot Task Generalizationcited by 2
  4. 2021Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Modulecited by 1
  5. 2022Enhanced Meta Reinforcement Learning using Demonstrations in Sparse Reward Environmentscited by 1
  6. 2023Robot Skill Generalization via Keypoint Integrated Soft Actor-Critic Gaussian Mixture Modelscited by 1
  7. 2024Sequential Decision Making with Expert Demonstrations under Unobserved Heterogeneitycited by 1
  8. 2025Balance Equation-based Distributionally Robust Offline Imitation Learningcited by 1
  9. 2021Curiosity-driven Intuitive Physics Learningcited by 0
  10. 2021Visual Goal-Directed Meta-Learning with Contextual Planning Networkscited by 0
  11. 2021Probability Density Estimation Based Imitation Learningcited by 0
  12. 2023Policy Optimization with Smooth Guidance Learned from State-Only Demonstrationscited by 0
  13. 2026Match or Replay: Self Imitating Proximal Policy Optimizationcited by 0
  14. 2026When Dynamics Shift, Robust Task Inference Wins: Offline Imitation Learning with Behavior Foundation Models Revisitedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Imitation Learning from Suboptimal Demonstrations via Meta-Learning An Action Ranker
  2. 2022Imitating, Fast and Slow: Robust learning from demonstrations via decision-time planning
  3. 2022Self-Imitation Learning from Demonstrations
  4. 2023A Survey of Demonstration Learning
  5. 2021Learning from Imperfect Demonstrations from Agents with Varying Dynamics
  6. 2019Reinforced Imitation in Heterogeneous Action Space
  7. 2021Imitation Learning: Progress, Taxonomies and Challenges
  8. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning