mlsanityData to 7 Oct 2026

CEIL: Generalized Contextual Imitation Learning

arXiv 2306.14534 ↗First posted 26 Jun 2023cs.LGcited by 12

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

In this paper, we present ContExtual Imitation Learning~(CEIL), a general and broadly applicable algorithm for imitation learning (IL). Inspired by the formulation of hindsight information matching, we derive CEIL by explicitly learning a hindsight embedding function together with a contextual policy using the hindsight embeddings. To achieve the expert matching objective for IL, we advocate for optimizing a contextual variable such that it biases the contextual policy towards mimicking expert behaviors. Beyond the typical learning from demonstrations (LfD) setting, CEIL is a generalist that can be effectively applied to multiple settings including: 1)~learning from observations (LfO), 2)~offline IL, 3)~cross-domain IL (mismatched experts), and 4) one-shot IL settings. Empirically, we evaluate CEIL on the popular MuJoCo tasks (online) and the D4RL dataset (offline). Compared to prior state-of-the-art baselines, we show that CEIL is more sample-efficient in most online IL tasks and achieves better or competitive performances in offline tasks.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 59 of its 76 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

59 of 76 references.

Closest ancestors · 26

Same direction, newest first

  1. 2023DITTO: Offline Imitation Learning with World Models6 Feb
  2. 2022Out-of-Dynamics Imitation Learning from Multimodal Demonstrations13 Nov
  3. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddings24 Sep
  4. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations20 Jul
  5. 2022Discriminator-Guided Model-Based Offline Imitation Learning1 Jul
  6. 2022Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos23 Jun
  7. 2022Latent Policies for Adversarial Imitation Learning22 Jun
  8. 2022Imitation Learning from Observations under Transition Model Disparity25 Apr
  9. 2022Robust Learning from Observation with Model Misspecification12 Feb
  10. 2022Imitation Learning by State-Only Distribution Matching9 Feb
  11. 2021Cross-Domain Imitation Learning via Optimal Transport7 Oct
  12. 2021Implicit Behavioral Cloning1 Sep
14 more
  1. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  2. 2021Cross-domain Imitation from Observations20 May
  3. 2021Off-Policy Imitation Learning from Observations25 Feb
  4. 2020Offline Learning from Demonstrations and Unlabeled Experience27 Nov
  5. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matching25 Jun
  6. 2020Primal Wasserstein Imitation Learning8 Jun
  7. 2020Energy-Based Imitation Learning20 Apr
  8. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  9. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  10. 2019State Alignment-based Imitation Learning21 Nov
  11. 2019Task-Relevant Adversarial Imitation Learning2 Oct
  12. 2019Imitation Learning as f-Divergence Minimization30 May
  13. 2018Generative Adversarial Imitation from Observation17 Jul
  14. 2018Behavioral Cloning from Observation4 May

Other references · 33

  1. 2017Neural Discrete Representation Learningcited by 4,650
  2. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  3. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  4. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  8. 2017Hindsight Experience Replaycited by 1,096
  9. 2021CLIP-Adapter: Better Vision-Language Models with Feature Adapterscited by 1,021
  10. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 882
  11. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  12. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
21 more
  1. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  2. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  3. 2017One-Shot Imitation Learningcited by 232
  4. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  5. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  6. 2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 143
  7. 2019Training Agents using Upside-Down Reinforcement Learningcited by 106
  8. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  9. 2019Reward-Conditioned Policiescited by 101
  10. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 73
  11. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  12. 2023Behavior Proximal Policy Optimizationcited by 43
  13. 2023Optimal Transport for Offline Imitation Learningcited by 42
  14. 2022DARA: Dynamics-Aware Reward Augmentation in Offline Reinforcement Learningcited by 40
  15. 2023Beyond Reward: Offline Preference-guided Policy Optimizationcited by 35
  16. 2023Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learningcited by 27
  17. 2022UniMASK: Unified Inference in Sequential Decision Problemscited by 23
  18. 2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learningcited by 20
  19. 2021Learn Goal-Conditioned Policy with Intrinsic Motivation for Deep Reinforcement Learningcited by 15
  20. 2021Offline Inverse Reinforcement Learningcited by 14
  21. 2022All You Need Is Supervised Learning: From Imitation Learning to Meta-RL With Upside Down RLcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 6; 2025: 2; 2026: 1.

12 papers.

  1. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  2. 2024Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulationcited by 10
  3. 2024Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designscited by 9
  4. 2024Robot Policy Learning with Temporal Optimal Transport Rewardcited by 8
  5. 2023Offline Imitation Learning with Variational Counterfactual Reasoningcited by 7
  6. 2025TDMPBC: Self-Imitative Reinforcement Learning for Humanoid Robot Controlcited by 6
  7. 2023SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentationcited by 5
  8. 2024Context-Former: Stitching via Latent Conditioned Sequence Modelingcited by 3
  9. 2026Semi-Supervised Cross-Domain Imitation Learningcited by 1
  10. 2024Imitating from auxiliary imperfect demonstrations via Adversarial Density Weighted Regressioncited by 0
  11. 2024State-of-the-art in Robot Learning for Multi-Robot Collaboration: A Comprehensive Surveycited by 0
  12. 2025RAD: Retrieval High-quality Demonstrations to Enhance Decision-makingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RoboSSM: Scalable In-context Imitation Learning via State-Space Models
  2. 2025Using Non-Expert Data to Robustify Imitation Learning via Offline Reinforcement Learning
  3. 2025Self-evolved Imitation Learning in Simulated World
  4. 2022Extraneousness-Aware Imitation Learning
  5. 2024IDIL: Imitation Learning of Intent-Driven Expert Behavior
  6. 2024Explorative Imitation Learning: A Path Signature Approach for Continuous Environments
  7. 2023Invariant Causal Imitation Learning for Generalizable Policies
  8. 2025Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations