# imitation: Clean Imitation Learning Implementations

arXiv: https://arxiv.org/abs/2211.11972 · HTML: https://mlsanity.com/paper/2211.11972/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 6 of its 15 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 6 of 6)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Dota 2 with Large Scale Deep Reinforcement Learning](https://mlsanity.com/paper/1912.06680/) (2019)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [What Matters for Adversarial Imitation Learning?](https://mlsanity.com/paper/2106.00672/) (2021)
- [APReL: A Library for Active Preference-based Reward Learning Algorithms](https://mlsanity.com/paper/2108.07259/) (2021)

## Built on by (most cited first, 30 of 34)

- [A Survey of Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2312.14925/) (2023)
- [Human-compatible driving partners through data-regularized self-play reinforcement learning](https://mlsanity.com/paper/2403.19648/) (2024)
- [Towards Solving Fuzzy Tasks with Human Feedback: A Retrospective of the MineRL BASALT 2022 Competition](https://mlsanity.com/paper/2303.13512/) (2023)
- [RLHF-Blender: A Configurable Interactive Interface for Learning from Diverse Human Feedback](https://mlsanity.com/paper/2308.04332/) (2023)
- [Diffusion Imitation from Observation](https://mlsanity.com/paper/2410.05429/) (2024)
- [CurricuVLM: Towards Safe Autonomous Driving via Personalized Safety-Critical Curriculum Learning with Vision-Language Models](https://mlsanity.com/paper/2502.15119/) (2025)
- [Residual Q-Learning: Offline and Online Policy Customization without Value](https://mlsanity.com/paper/2306.09526/) (2023)
- [X-IL: Exploring the Design Space of Imitation Learning Policies](https://mlsanity.com/paper/2502.12330/) (2025)
- [Reward Learning from Multiple Feedback Types](https://mlsanity.com/paper/2502.21038/) (2025)
- [BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy Tasks](https://mlsanity.com/paper/2312.02405/) (2023)
- [RICE: Breaking Through the Training Bottlenecks of Reinforcement Learning with Explanation](https://mlsanity.com/paper/2405.03064/) (2024)
- [SPACeR: Self-Play Anchoring with Centralized Reference Models](https://mlsanity.com/paper/2510.18060/) (2025)
- [WOFOSTGym: A Crop Simulator for Learning Annual and Perennial Crop Management Strategies](https://mlsanity.com/paper/2502.19308/) (2025)
- [Interactive Groupwise Comparison for Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2507.04340/) (2025)
- [Safe Imitation Learning of Nonlinear Model Predictive Control for Flexible Robots](https://mlsanity.com/paper/2212.02941/) (2022)
- [C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory](https://mlsanity.com/paper/2402.16349/) (2024)
- [Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shaping](https://mlsanity.com/paper/2410.03847/) (2024)
- [Leveraging Genetic Algorithms for Efficient Demonstration Generation in Real-World Reinforcement Learning Environments](https://mlsanity.com/paper/2507.00762/) (2025)
- ["Give Me an Example Like This": Episodic Active Reinforcement Learning from Demonstrations](https://mlsanity.com/paper/2406.03069/) (2024)
- [Synergistic Reinforcement and Imitation Learning for Vision-driven Autonomous Flight of UAV Along River](https://mlsanity.com/paper/2401.09332/) (2024)
- [Multi Task Inverse Reinforcement Learning for Common Sense Reward](https://mlsanity.com/paper/2402.11367/) (2024)
- [Preserving the Privacy of Reward Functions in MDPs through Deception](https://mlsanity.com/paper/2407.09809/) (2024)
- [A Graph-based Adversarial Imitation Learning Framework for Reliable & Realtime Fleet Scheduling in Urban Air Mobility](https://mlsanity.com/paper/2407.12113/) (2024)
- [On Discovering Algorithms for Adversarial Imitation Learning](https://mlsanity.com/paper/2510.00922/) (2025)
- [Synthetically Generating Human-like Data for Sequential Decision Making Tasks via Reward-Shaped Imitation Learning](https://mlsanity.com/paper/2304.07280/) (2023)
- [Dynamic value alignment through preference aggregation of multiple objectives](https://mlsanity.com/paper/2310.05871/) (2023)
- [Supervised Reward Inference](https://mlsanity.com/paper/2502.18447/) (2025)
- [Whenever, Wherever: Towards Orchestrating Crowd Simulations with Spatio-Temporal Spawn Dynamics](https://mlsanity.com/paper/2503.16639/) (2025)
- [Learning Dolly-In Filming From Demonstration Using a Ground-Based Robot](https://mlsanity.com/paper/2509.00574/) (2025)
- [Buzz, Choose, Forget: A Meta-Bandit Framework for Bee-Like Decision Making](https://mlsanity.com/paper/2510.16462/) (2025)

## Abstract (arXiv)

imitation provides open-source implementations of imitation and reward learning algorithms in PyTorch. We include three inverse reinforcement learning (IRL) algorithms, three imitation learning algorithms and a preference comparison algorithm. The implementations have been benchmarked against previous results, and automated tests cover 98% of the code. Moreover, the algorithms are implemented in a modular fashion, making it simple to develop novel algorithms in the framework. Our source code, including documentation and examples, is available at https://github.com/HumanCompatibleAI/imitation