mlsanityData to 14 Oct 2026

Instant Policy: In-Context Imitation Learning via Graph Diffusion

arXiv 2411.12633 ↗First posted 19 Nov 2024cs.RO cs.AI cs.CVcited by 39

On the map: Offline RL and imitation learning from data → Robot manipulation imitation learning

Abstract (arXiv)

Following the impressive capabilities of in-context learning with large transformers, In-Context Imitation Learning (ICIL) is a promising opportunity for robotics. We introduce Instant Policy, which learns new tasks instantly (without further training) from just one or two demonstrations, achieving ICIL through two key components. First, we introduce inductive biases through a graph representation and model ICIL as a graph generation problem with a learned diffusion process, enabling structured reasoning over demonstrations, observations, and actions. Second, we show that such a model can be trained using pseudo-demonstrations - arbitrary trajectories generated in simulation - as a virtually infinite pool of training data. Simulated and real experiments show that Instant Policy enables rapid learning of various everyday robot tasks. We also show how it can serve as a foundation for cross-embodiment and zero-shot transfer to language-defined tasks. Code and videos are available at https://www.robot-learning.uk/instant-policy.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 40 of its 48 references that are in the map. First posted on arXiv 19 Nov 2024.

Builds on

40 of 48 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024ALOHA Unleashed: A Simple Recipe for Robot Dexterity17 Oct
  2. 2024In-Context Imitation Learning via Next-Token Prediction28 Aug
  3. 2024One-Shot Imitation Learning with Invariance Matching for Robotic Manipulation21 May
  4. 2024Keypoint Action Tokens Enable In-Context Imitation Learning in Robotics28 Mar
  5. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations26 Oct
  6. 2023Few-Shot In-Context Imitation Learning via Implicit Graph Alignment18 Oct
  7. 2023NeRF in the Palm of Your Hand: Corrective Augmentation for Robotics via Novel-View Synthesis18 Jan
  8. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learning4 Feb

Other references · 32

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023Segment Anythingcited by 8,234
  7. 2016Layer Normalizationcited by 5,669
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  9. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  10. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  11. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  12. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
20 more
  1. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  2. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  3. 2021Perceiver IO: A General Architecture for Structured Inputs & Outputscited by 629
  4. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 583
  5. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  6. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 506
  7. 2021OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphscited by 329
  8. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
  9. 2017One-Shot Imitation Learningcited by 233
  10. 2023Scaling Robot Learning with Semantically Imagined Experiencecited by 177
  11. 2017Heterogeneous Information Network Embedding for Recommendationcited by 133
  12. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusioncited by 127
  13. 2022DALL-E-Bot: Introducing Web-Scale Diffusion Models to Roboticscited by 106
  14. 2023XMem++: Production-level Video Segmentation From Few Annotated Framescited by 53
  15. 2024Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformerscited by 52
  16. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 38
  17. 2024Body Transformer: Leveraging Robot Embodiment for Policy Learningcited by 32
  18. 2021My House, My Rules: Learning Tidying Preferences with Graph Neural Networkscited by 27
  19. 2024Render and Diffuse: Aligning Image and Action Spaces for Diffusion-based Behaviour Cloningcited by 20
  20. 2022Where To Start? Transferring Simple Skills to Complex Environmentscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 22.

39 papers.

Sort:

By area:

  1. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 33
  2. 2025MAP-VLA: Memory-Augmented Prompting for Vision-Language-Action Model in Robotic Manipulationcited by 28
  3. 2025SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulationcited by 21
  4. 2025AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletonscited by 18
  5. 2025Action Tokenizer Matters in In-Context Imitation Learningcited by 14
  6. 2026Act-Observe-Rewrite: Multimodal Coding Agents as In-Context Policy Learners for Robot Manipulationcited by 9
  7. 2025One-Shot Dual-Arm Imitation Learningcited by 8
  8. 2026In-Context Robot Learning with VLM Agentscited by 8
  9. 2025Learning Affordances at Inference-Time for Vision-Language-Action Modelscited by 6
  10. 2025X-Diffusion: Training Diffusion Policies on Cross-Embodiment Human Demonstrationscited by 6
  11. 2025IA-VLA: Input Augmentation for Vision-Language-Action models in settings with semantically complex taskscited by 4
  12. 2025OMP: One-step Meanflow Policy with Directional Alignmentcited by 4
27 more
  1. 2025MaNGO - Adaptable Graph Network Simulators via Meta-Learningcited by 3
  2. 2025Knowledge-Driven Imitation Learning: Enabling Generalization Across Diverse Conditionscited by 2
  3. 2025Learning in ImaginationLand: Omnidirectional Policies through 3D Generative Models (OP-Gen)cited by 2
  4. 2025Generalizable Insights for Graph Transformers in Theory and Practicecited by 2
  5. 2026Ada3Drift: Adaptive Training-Time Drifting for One-Step 3D Visuomotor Robotic Manipulationcited by 2
  6. 2026Instant-Fold: In-Context Imitation Learning for Deformable Object Manipulationcited by 2
  7. 2025Large Pre-Trained Models for Bimanual Manipulation in 3Dcited by 1
  8. 2025Annotation-Free One-Shot Imitation Learning for Multi-Step Manipulation Taskscited by 1
  9. 2026RADAR: Closed-Loop Robotic Data Generation via Semantic Planning and Autonomous Causal Environment Resetcited by 1
  10. 2026A Hierarchical Spatiotemporal Action Tokenizer for In-Context Imitation Learning in Roboticscited by 1
  11. 2026SynthICL: Scalable In-context Imitation Learning with Synthetic Datacited by 1
  12. 2026Difference-Aware Retrieval Policies for Imitation Learningcited by 1
  13. 2026Adaptation of Generalist Robot Policies with Minimal Datacited by 1
  14. 2026ContextFlow: In-Context Flow Matching for Robot Manipulationcited by 1
  15. 2026Generalizing Manipulation Skills with a Local Coding Agentcited by 1
  16. 2026POIL: Point-based One-Shot Imitation Learning with Stable Dynamical Systemscited by 1
  17. 2026In-Context Learning for Robots: Methods and Applicationscited by 1
  18. 2025ManiLong-Shot: Interaction-Aware One-Shot Imitation Learning for Long-Horizon Manipulationcited by 0
  19. 2026Show, Don't Tell: Detecting Novel Objects by Watching Human Videoscited by 0
  20. 2026Decompose and Recompose: Reasoning New Skills from Existing Abilities for Cross-Task Robotic Manipulationcited by 0
  21. 2026Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulatorscited by 0
  22. 2026SIR: Structured Image Representations for Explainable Robot Learningcited by 0
  23. 2026Dynamics-Aware Meta-Imitation for Generalization to Unseen Robotic Manipulationcited by 0
  24. 2026RA-VLA: Retrieval-Augmented VLA for Test-Time Adaptationcited by 0
  25. 2026Does Imitation Learning Preserve Temporal Robustness in Dexterous Manipulation? An Expert-Learner Comparison Across Task Execution Speedscited by 0
  26. 2026RoboICL: Embodied In-Context Learning with GPT-6 Astracited by 0
  27. 2026Encore: Few-Shot Agentic Discovery of Manipulation Strategiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Training and Evaluating Diffusion Policies with Long Context Lengths
  2. 2026ICLR: In-Context Imitation Learning with Visual Reasoning
  3. 2025RoboSSM: Scalable In-context Imitation Learning via State-Space Models
  4. 2026Expanding Spatial and Temporal Context for Robotic Imitation Learning With Scene Graphs
  5. 2025Robust Instant Policy: Leveraging Student's t-Regression Model for Robust In-context Imitation Learning of Robot Manipulation
  6. 2024Learning from 10 Demos: Generalisable and Sample-Efficient Policy Learning with Oriented Affordance Frames
  7. 2025Learning Long-Context Diffusion Policies via Past-Token Prediction
  8. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning