mlsanityData to 7 Oct 2026

OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning

arXiv 1709.06683 ↗First posted 20 Sep 2017cs.LGcited by 40

Not on the map. Why?

Cited by 33 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning has shown promise in learning policies that can solve complex problems. However, manually specifying a good reward function can be difficult, especially for intricate tasks. Inverse reinforcement learning offers a useful paradigm to learn the underlying reward function directly from expert demonstrations. Yet in reality, the corpus of demonstrations may contain trajectories arising from a diverse set of underlying reward functions rather than a single one. Thus, in inverse reinforcement learning, it is useful to consider such a decomposition. The options framework in reinforcement learning is specifically designed to decompose policies in a similar light. We therefore extend the options framework and propose a method to simultaneously recover reward options in addition to policy options. We leverage adversarial methods to learn joint reward-policy options using only observed expert states. We show that this approach works well in both simple and complex continuous control tasks and shows significant performance increases in one-shot transfer learning.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Sep 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2018: 2; 2019: 5; 2020: 6; 2021: 10; 2022: 5; 2023: 6; 2024: 4; 2025: 2.

40 papers.

Sort:

By area:

  1. 2018An Algorithmic Perspective on Imitation Learningcited by 455
  2. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  3. 2019Recent Advances in Imitation Learning from Observationcited by 92
  4. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimalitycited by 44
  5. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  6. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learningcited by 17
  7. 2023Hierarchical Imitation Learning with Vector Quantized Modelscited by 15
  8. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  9. 2024Diffusion Imitation from Observationcited by 12
  10. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Taskscited by 10
  11. 2022Transfering Hierarchical Structure with Dual Meta Imitation Learningcited by 7
  12. 2025Intention-Conditioned Flow Occupancy Modelscited by 7
28 more
  1. 2019Randomized Adversarial Imitation Learning for Autonomous Drivingcited by 6
  2. 2022SHAIL: Safety-Aware Hierarchical Adversarial Imitation Learning for Autonomous Driving in Urban Environmentscited by 5
  3. 2019Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learningcited by 4
  4. 2020Forward and inverse reinforcement learning sharing network weights and hyperparameterscited by 4
  5. 2021Maximum Entropy Reinforcement Learning with Mixture Policiescited by 4
  6. 2021DEALIO: Data-Efficient Adversarial Learning for Imitation from Observationcited by 4
  7. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actionscited by 3
  8. 2020Adversarial Imitation Learning via Random Searchcited by 3
  9. 2021Online Baum-Welch algorithm for Hierarchical Imitation Learningcited by 3
  10. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learningcited by 3
  11. 2022Beyond Rewards: a Hierarchical Perspective on Offline Multiagent Behavioral Analysiscited by 3
  12. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3
  13. 2019Option Compatible Reward Inverse Reinforcement Learningcited by 2
  14. 2021LiMIIRL: Lightweight Multiple-Intent Inverse Reinforcement Learningcited by 2
  15. 2022Divide & Conquer Imitation Learningcited by 2
  16. 2018Injective State-Image Mapping facilitates Visual Adversarial Imitation Learningcited by 1
  17. 2020HILONet: Hierarchical Imitation Learning from Non-Aligned Observationscited by 1
  18. 2021A Review on Oracle Issues in Machine Learningcited by 1
  19. 2021Parallelized and Randomized Adversarial Imitation Learning for Safety-Critical Self-Driving Vehiclescited by 1
  20. 2023Adversarial Imitation Learning On Aggregated Datacited by 1
  21. 2023Active Third-Person Imitation Learningcited by 1
  22. 2024Imitation Learning with Additional Constraints on Motion Style using Parametric Biascited by 1
  23. 2020Preventing Imitation Learning with Adversarial Policy Ensemblescited by 0
  24. 2023Adversarial Policy Optimization in Deep Reinforcement Learningcited by 0
  25. 2023FP-IRL: Fokker--Planck Inverse Reinforcement Learning -- A Physics-Constrained Approach to Markov Decision Processescited by 0
  26. 2023CasIL: Cognizing and Imitating Skills via a Dual Cognition-Action Architecturecited by 0
  27. 2024Segmenting Action-Value Functions Over Time-Scales in SARSA via TD(Delta)cited by 0
  28. 2025SPLASH! Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrationscited by 0