# CEIL: Generalized Contextual Imitation Learning

arXiv: https://arxiv.org/abs/2306.14534 · HTML: https://mlsanity.com/paper/2306.14534/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 59 of its 76 references that are in the map.
- First posted on arXiv 26 Jun 2023.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 59)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [D4RL: Datasets for Deep Data-Driven Reinforcement Learning](https://mlsanity.com/paper/2004.07219/) (2020)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [A Minimalist Approach to Offline Reinforcement Learning](https://mlsanity.com/paper/2106.06860/) (2021)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [Offline Reinforcement Learning as One Big Sequence Modeling Problem](https://mlsanity.com/paper/2106.02039/) (2021)
- [Is Conditional Generative Modeling all you need for Decision-Making?](https://mlsanity.com/paper/2211.15657/) (2022)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Implicit Behavioral Cloning](https://mlsanity.com/paper/2109.00137/) (2021)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [One-Shot Imitation Learning](https://mlsanity.com/paper/1703.07326/) (2017)
- [IQ-Learn: Inverse soft-Q Learning for Imitation](https://mlsanity.com/paper/2106.12142/) (2021)
- [RvS: What is Essential for Offline RL via Supervised Learning?](https://mlsanity.com/paper/2112.10751/) (2021)
- [Generative Adversarial Imitation from Observation](https://mlsanity.com/paper/1807.06158/) (2018)
- [Imitation Learning via Off-Policy Distribution Matching](https://mlsanity.com/paper/1912.05032/) (2019)
- [Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation](https://mlsanity.com/paper/1707.03374/) (2017)
- [Prompting Decision Transformer for Few-Shot Policy Generalization](https://mlsanity.com/paper/2206.13499/) (2022)
- [Imitation Learning as f-Divergence Minimization](https://mlsanity.com/paper/1905.12888/) (2019)
- [Primal Wasserstein Imitation Learning](https://mlsanity.com/paper/2006.04678/) (2020)
- [Training Agents using Upside-Down Reinforcement Learning](https://mlsanity.com/paper/1912.02877/) (2019)
- [Generalized Decision Transformer for Offline Hindsight Information Matching](https://mlsanity.com/paper/2111.10364/) (2021)
- [Reward-Conditioned Policies](https://mlsanity.com/paper/1912.13465/) (2019)
- [Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrations](https://mlsanity.com/paper/2207.10050/) (2022)

## Built on by (most cited first, 13 of 13)

- [A Survey of Imitation Learning: Algorithms, Recent Developments, and Challenges](https://mlsanity.com/paper/2309.02473/) (2023)
- [Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation](https://mlsanity.com/paper/2402.07127/) (2024)
- [Scalable and Effective Arithmetic Tree Generation for Adder and Multiplier Designs](https://mlsanity.com/paper/2405.06758/) (2024)
- [Robot Policy Learning with Temporal Optimal Transport Reward](https://mlsanity.com/paper/2410.21795/) (2024)
- [Offline Imitation Learning with Variational Counterfactual Reasoning](https://mlsanity.com/paper/2310.04706/) (2023)
- [Residual-MPPI: Online Policy Customization for Continuous Control](https://mlsanity.com/paper/2407.00898/) (2024)
- [TDMPBC: Self-Imitative Reinforcement Learning for Humanoid Robot Control](https://mlsanity.com/paper/2502.17322/) (2025)
- [SAM-OCTA: Prompting Segment-Anything for OCTA Image Segmentation](https://mlsanity.com/paper/2310.07183/) (2023)
- [Context-Former: Stitching via Latent Conditioned Sequence Modeling](https://mlsanity.com/paper/2401.16452/) (2024)
- [Semi-Supervised Cross-Domain Imitation Learning](https://mlsanity.com/paper/2602.10793/) (2026)
- [Imitating from auxiliary imperfect demonstrations via Adversarial Density Weighted Regression](https://mlsanity.com/paper/2405.20351/) (2024)
- [State-of-the-art in Robot Learning for Multi-Robot Collaboration: A Comprehensive Survey](https://mlsanity.com/paper/2408.11822/) (2024)
- [RAD: Retrieval High-quality Demonstrations to Enhance Decision-making](https://mlsanity.com/paper/2507.15356/) (2025)

## Abstract (arXiv)

In this paper, we present ContExtual Imitation Learning~(CEIL), a general and broadly applicable algorithm for imitation learning (IL). Inspired by the formulation of hindsight information matching, we derive CEIL by explicitly learning a hindsight embedding function together with a contextual policy using the hindsight embeddings. To achieve the expert matching objective for IL, we advocate for optimizing a contextual variable such that it biases the contextual policy towards mimicking expert behaviors. Beyond the typical learning from demonstrations (LfD) setting, CEIL is a generalist that can be effectively applied to multiple settings including: 1)~learning from observations (LfO), 2)~offline IL, 3)~cross-domain IL (mismatched experts), and 4) one-shot IL settings. Empirically, we evaluate CEIL on the popular MuJoCo tasks (online) and the D4RL dataset (offline). Compared to prior state-of-the-art baselines, we show that CEIL is more sample-efficient in most online IL tasks and achieves better or competitive performances in offline tasks.