mlsanityData to 7 Oct 2026

Consistent Zero-Shot Imitation with Contrastive Goal Inference

arXiv 2510.17059 ↗First posted 20 Oct 2025cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Zero-shot imitation learning requires an agent to reproduce expert behavior from a single demonstration without additional environment interaction or gradient updates at test time. We introduce Contrastive Inverse Reinforcement Learning (CIRL), a self-supervised framework for pre-training zero-shot imitation agents. Our methods rests on a key observation that many useful tasks can be summarized by a single goal state. We can thus convert the multi-task inverse RL problem into a more tractable goal-inference problem, and utilize state-of-the-art goal-conditioned RL methods to recover a policy that reaches the goal. During pre-training, CIRL jointly employs three components to learn without any rewards or demonstrations: (1) a variant of contrastive RL designed to learn maximum-entropy goal-conditioned policies, (2) an automatic goal proposal mechanism (GoalKDE) that drives exploration, and (3) a mean-field variational model that performs amortized goal inference from trajectories. We prove that this procedure consistently recovers the demonstrator's intent by accounting for the relative difficulty of reaching different states and show how structurally similar prior work may otherwise fail to infer the correct reward. Experiments on goal-conditioned and standard reward-maximizing control tasks show that CIRL outperforms prior zero-shot imitation methods, supporting the expressiveness of goals as a compact summary of behavior.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 33 of its 50 references that are in the map. First posted on arXiv 20 Oct 2025.

Builds on

33 of 50 references.

References · 33

  1. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  4. 2016Concrete Problems in AI Safetycited by 1,582
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  7. 2017Hindsight Experience Replaycited by 1,096
  8. 2016End to End Learning for Self-Driving Carscited by 1,000
  9. 2022A Generalist Agentcited by 802
  10. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  11. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  12. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 375
21 more
  1. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  2. 2018Visual Reinforcement Learning with Imagined Goalscited by 248
  3. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 241
  4. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 225
  5. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  6. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 146
  7. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  8. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  9. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
  10. 2018Zero-Shot Visual Imitationcited by 114
  11. 2021Learning One Representation to Optimize All Rewardscited by 110
  12. 2018Unsupervised Control Through Non-Parametric Discriminative Rewardscited by 102
  13. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  14. 2021Asymmetric self-play for automatic goal discovery in robotic manipulationcited by 71
  15. 2020Rewriting History with Inverse RL: Hindsight Inference for Policy Improvementcited by 59
  16. 2024Learning Temporal Distances: Contrastive Successor Features Can Provide a Metric Structure for Decision-Makingcited by 53
  17. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variablescited by 50
  18. 2024Accelerating Goal-Conditioned RL Algorithms and Researchcited by 18
  19. 2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learningcited by 15
  20. 2018Forward Amortized Inference for Likelihood-Free Variational Marginalizationcited by 14
  21. 2023Variational Curriculum Reinforcement Learning for Unsupervised Discovery of Skillscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Can We Really Learn One Representation to Optimize All Rewards?cited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Zero-Shot Offline Imitation Learning via Optimal Transport
  2. 2026Zero-shot Imitation Learning by Latent Topology Mapping
  3. 2023Action Inference by Maximising Evidence: Zero-Shot Imitation from Observation with World Models
  4. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation
  5. 2026Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning
  6. 2026Learning from Demonstrations via Capability-Aware Goal Sampling
  7. 2021XIRL: Cross-embodiment Inverse Reinforcement Learning
  8. 2019Reinforced Imitation in Heterogeneous Action Space