Adversarial Imitation Learning On Aggregated Data
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
Inverse Reinforcement Learning (IRL) learns an optimal policy, given some expert demonstrations, thus avoiding the need for the tedious process of specifying a suitable reward function. However, current methods are constrained by at least one of the following requirements. The first one is the need to fully solve a forward Reinforcement Learning (RL) problem in the inner loop of the algorithm, which might be prohibitively expensive in many complex environments. The second one is the need for full trajectories from the experts, which might not be easily available. The third one is the assumption that the expert data is homogeneous rather than a collection from various experts or possibly alternative solutions to the same task. Such constraints make IRL approaches either not scalable or not usable on certain existing systems. In this work we propose an approach which removes these requirements through a dynamic, adaptive method called Adversarial Imitation Learning on Aggregated Data (AILAD). It learns conjointly both a non linear reward function and the associated optimal policy using an adversarial framework. The reward learner only uses aggregated data. Moreover, it generates diverse behaviors producing a distribution over the aggregated data matching that of the experts.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 17 of its 30 references that are in the map. First posted on arXiv 14 Nov 2023.
Builds on
17 of 30 references.
References · 17
- 2013Playing Atari with Deep Reinforcement Learningcited by 3,221
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,975
- 2015Trust Region Policy Optimizationcited by 2,513
- 2016Generative Adversarial Imitation Learningcited by 1,577
- 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 486
- 2015Illuminating search spaces by mapping elitescited by 415
- 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
- 2020Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Surveycited by 388
- 2018Unity: A General Platform for Intelligent Agentscited by 232
- 2016Sample Efficient Actor-Critic with Experience Replaycited by 186
- 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
- 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
5 more
- 2021What Matters for Adversarial Imitation Learning?cited by 70
- 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
- 2022Automated Play-Testing Through RL Based Human-Like Play-Styles Generationcited by 11
- 2017Inverse Reinforcement Learning from Summary Datacited by 5
- 2020Playing Catan with Cross-dimensional Neural Networkcited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
- 2020Off-Policy Adversarial Inverse Reinforcement Learning
- 2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment
- 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
- 2020ADAIL: Adaptive Adversarial Imitation Learning
- 2018Adversarial Imitation via Variational Inverse Reinforcement Learning
- 2024Inverse Delayed Reinforcement Learning
- 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms