Data to 14 Oct 2026

Model-Free Imitation Learning with Policy Optimization

arXiv 1605.08478 ↗First posted 26 May 2016cs.LG cs.AIcited by 52

Not on the map. Why?

Cited by 36 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In imitation learning, an agent learns how to behave in an environment with an unknown cost function by mimicking expert demonstrations. Existing imitation learning algorithms typically involve solving a sequence of planning or reinforcement learning problems. Such algorithms are therefore not directly applicable to large, high-dimensional environments, and their performance can significantly degrade if the planning problems are not solved to optimality. Under the apprenticeship learning formalism, we develop alternative model-free algorithms for finding a parameterized stochastic policy that performs at least as well as an expert policy on an unknown cost function, based on sample trajectories from the expert. Our approach, based on policy gradients, scales to large continuous environments with guaranteed convergence to local minima.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). First posted on arXiv 26 May 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 3; 2019: 8; 2020: 5; 2021: 10; 2022: 7; 2023: 7; 2024: 7; 2025: 4; 2026: 1.

52 papers.

Sort:

By area:

2018Behavioral Cloning from Observationcited by 492
2018An Algorithmic Perspective on Imitation Learningcited by 459
2021Implicit Behavioral Cloningcited by 452
2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 148
2019Imitation Learning as f-Divergence Minimizationcited by 123
2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 91
2019Wasserstein Adversarial Imitation Learningcited by 60
2019Multi-Agent Adversarial Inverse Reinforcement Learningcited by 60
2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 54
2021Procedure Planning in Instructional Videos via Contextual Modeling and Model-based Policy Learningcited by 53
2020Modeling Human Driving Behavior through Generative Adversarial Imitation Learningcited by 45
2023PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedbackcited by 40

38 more

2020Grasping with Chopsticks: Combating Covariate Shift in Model-free Imitation Learning for Fine Manipulationcited by 38
2018Sample-Efficient Imitation Learning via Generative Adversarial Netscited by 36
2025CarPlanner: Consistent Auto-regressive Trajectory Planning for Large-scale Reinforcement Learning in Autonomous Drivingcited by 34
2021Online Apprenticeship Learningcited by 29
2022Proximal Point Imitation Learningcited by 26
2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learningcited by 16
2023Constrained Policy Optimization with Explicit Behavior Density for Offline Reinforcement Learningcited by 16
2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
2024Imitation Learning in Discounted Linear MDPs without exploration assumptionscited by 15
2021Training Value-Aligned Reinforcement Learning Agents Using a Normative Priorcited by 14
2024SPIRE: Synergistic Planning, Imitation, and Reinforcement Learning for Long-Horizon Manipulationcited by 11
2021Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrationscited by 10
2023Inverse Decision Modeling: Learning Interpretable Representations of Behaviorcited by 10
2019Model-based Behavioral Cloning with Future Image Similarity Learningcited by 8
2021Supervised Advantage Actor-Critic for Recommender Systemscited by 8
2023How To Guide Your Learner: Imitation Learning with Active Adaptive Expert Involvementcited by 8
2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 6
2020POPO: Pessimistic Offline Policy Optimizationcited by 5
2024Deep Reinforcement Learning for Modelling Protein Complexescited by 5
2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
2019On Value Discrepancy of Imitation Learningcited by 4
2021Robust Maximum Entropy Behavior Cloningcited by 4
2024Multi-Agent Generative Adversarial Interactive Self-Imitation Learning for AUV Formation Control and Obstacle Avoidancecited by 3
2021GAN-Based Interactive Reinforcement Learning from Demonstration and Human Evaluative Feedbackcited by 2
2022Fail-Safe Adversarial Generative Imitation Learningcited by 2
2024Towards Generalized Inverse Reinforcement Learningcited by 2
2024Conditional Latent ODEs for Motion Prediction in Autonomous Drivingcited by 2
2020Metric-Based Imitation Learning Between Two Dissimilar Anthropomorphic Robotic Armscited by 1
2021Stochastic convex optimization for provably efficient apprenticeship learningcited by 1
2022Profitable Strategy Design by Using Deep Reinforcement Learning for Trades on Cryptocurrency Marketscited by 1
2023Replicating Complex Dialogue Policy of Humans via Offline Imitation Learning with Supervised Regularizationcited by 1
2019Stochastic Inverse Reinforcement Learningcited by 0
2019Relational Mimic for Visual Adversarial Imitation Learningcited by 0
2021Learning Lipschitz Feedback Policies from Expert Demonstrations: Closed-Loop Guarantees, Generalization and Robustnesscited by 0
2022GAIL-PT: A Generic Intelligent Penetration Testing Framework with Generative Adversarial Imitation Learningcited by 0
2022Some Supervision Required: Incorporating Oracle Policies in Reinforcement Learning via Epistemic Uncertainty Metricscited by 0
2022Understanding Hindsight Goal Relabeling from a Divergence Minimization Perspectivecited by 0
2024Randomized algorithms and PAC bounds for inverse reinforcement learning in continuous spacescited by 0

50 most cited of 52 All 52 (JSON)