mlsanityData to 7 Oct 2026

Task-Embedded Control Networks for Few-Shot Imitation Learning

arXiv 1810.03237 ↗First posted 8 Oct 2018cs.RO cs.AI cs.CVcited by 65

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Much like humans, robots should have the ability to leverage knowledge from previously learned tasks in order to learn new tasks quickly in new and unfamiliar environments. Despite this, most robot learning approaches have focused on learning a single task, from scratch, with a limited notion of generalisation, and no way of leveraging the knowledge to learn other tasks more efficiently. One possible solution is meta-learning, but many of the related approaches are limited in their ability to scale to a large number of tasks and to learn further tasks without forgetting previously learned ones. With this in mind, we introduce Task-Embedded Control Networks, which employ ideas from metric learning in order to create a task embedding that can be used by a robot to learn new tasks from one or more demonstrations. In the area of visually-guided manipulation, we present simulation results in which we surpass the performance of a state-of-the-art method when using only visual information from each demonstration. Additionally, we demonstrate that our approach can also be used in conjunction with domain randomisation to train our few-shot learning ability in simulation and then deploy in the real world without any additional training. Once deployed, the robot can learn new tasks from a single real-world demonstration.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 36 references that are in the map. First posted on arXiv 8 Oct 2018.

Builds on

19 of 36 references.

References · 19

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016Layer Normalizationcited by 5,579
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2016Matching Networks for One Shot Learningcited by 2,246
  8. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  9. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  12. 2017Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperationcited by 286
7 more
  1. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  2. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  3. 2017Transferring End-to-End Visuomotor Control from Simulation to Real World for a Multi-Stage Taskcited by 66
  4. 2013A Survey on Metric Learning for Feature Vectors and Structured Datacited by 57
  5. 2017Few-Shot Learning Through an Information Retrieval Lenscited by 51
  6. 20163D Simulation for Robot Arm Control with Deep Q-Learningcited by 19
  7. 2015Deep Multimodal Embedding: Manipulating Novel Objects with Point-clouds, Language and Trajectoriescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 4; 2021: 12; 2022: 8; 2023: 16; 2024: 13; 2025: 8; 2026: 3.

65 papers.

Sort:

By area:

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  3. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  4. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
  5. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  6. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  7. 2023RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shotcited by 173
  8. 2021Coarse-to-Fine Q-attention: Efficient Learning for Visual Robotic Manipulation via Discretisationcited by 120
  9. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  10. 2021Coarse-to-Fine Imitation Learning: Robot Manipulation from a Single Demonstrationcited by 92
  11. 2024EquiBot: SIM(3)-Equivariant Diffusion Policy for Generalizable and Data Efficient Learningcited by 72
  12. 2024Keypoint Action Tokens Enable In-Context Imitation Learning in Roboticscited by 71
38 more
  1. 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Roboticscited by 68
  2. 2023Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasetscited by 56
  3. 2022Imitation Learning by Estimating Expertise of Demonstratorscited by 50
  4. 2021Towards More Generalizable One-shot Visual Imitation Learningcited by 48
  5. 2021AGENT: A Benchmark for Core Psychological Reasoningcited by 42
  6. 2020Provable Representation Learning for Imitation Learning via Bi-level Optimizationcited by 41
  7. 2024RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulationcited by 40
  8. 2023Generalization to New Sequential Decision Making Tasks with In-Context Learningcited by 39
  9. 2024Surgical Robot Transformer (SRT): Imitation Learning for Surgical Taskscited by 38
  10. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  11. 2023Hyper-Decision Transformer for Efficient Online Policy Adaptationcited by 32
  12. 2019Watch, Try, Learn: Meta-Learning from Demonstrations and Rewardcited by 26
  13. 2021Learning Transferable Motor Skills with Hierarchical Latent Mixture Policiescited by 26
  14. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  15. 2021Bayesian Meta-Learning for Few-Shot Policy Adaptation Across Robotic Platformscited by 22
  16. 2020Scalable Multi-Task Imitation Learning with Autonomous Improvementcited by 21
  17. 2021Learning Multi-Stage Tasks with One Demonstration via Self-Replaycited by 21
  18. 2022On the Effectiveness of Fine-tuning Versus Meta-reinforcement Learningcited by 18
  19. 2023Invariant Causal Imitation Learning for Generalizable Policiescited by 15
  20. 2024Gaussian Splatting to Real World Flight Navigation Transfer with Liquid Networkscited by 12
  21. 2020Episodic Self-Imitation Learning with Hindsightcited by 10
  22. 2023EC^2: Emergent Communication for Embodied Controlcited by 10
  23. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  24. 2024Effective Tuning Strategies for Generalist Robot Manipulation Policiescited by 10
  25. 2022Sim-to-Real via Sim-to-Seg: End-to-end Off-road Autonomous Driving Without Real Datacited by 9
  26. 2022Multi-Task Meta Learning: learn how to adapt to unseen taskscited by 8
  27. 2023Contrastive Modules with Temporal Attention for Multi-Task Reinforcement Learningcited by 8
  28. 2025cVLA: Towards Efficient Camera-Space VLAscited by 8
  29. 2025Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuningcited by 8
  30. 2024Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learningcited by 6
  31. 2020SAFARI: Safe and Active Robot Imitation Learning with Imaginationcited by 5
  32. 2022PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scalecited by 5
  33. 2023Few-Shot Learning of Force-Based Motions From Demonstration Through Pre-training of Haptic Representationcited by 5
  34. 2023One-shot Visual Imitation via Attributed Waypoints and Demonstration Augmentationcited by 4
  35. 2023Get Back Here: Robust Imitation by Return-to-Distribution Planningcited by 4
  36. 2025Behavioral Exploration: Learning to Explore via In-Context Adaptationcited by 4
  37. 2025See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrationscited by 4
  38. 2023Comparing the Efficacy of Fine-Tuning and Meta-Learning for Few-Shot Policy Imitationcited by 3

50 most cited of 65 All 65 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2019Learning One-Shot Imitation from Humans without Humans
  2. 2024Meta-Controller: Few-Shot Imitation of Unseen Embodiments and Tasks in Continuous Control
  3. 2022Learning Sensorimotor Primitives of Sequential Manipulation Tasks from Visual Demonstrations
  4. 2023Few-Shot In-Context Imitation Learning via Implicit Graph Alignment
  5. 2019Meta Adaptation using Importance Weighted Demonstrations
  6. 2022Learning and Extrapolation of Robotic Skills using Task-Parameterized Equation Learner Networks
  7. 2026Task Parameter Extrapolation via Learning Inverse Tasks from Forward Demonstrations ↗
  8. 2018One-Shot Learning of Multi-Step Tasks from Observation via Activity Localization in Auxiliary Video