mlsanityData to 7 Oct 2026

Learn to Exceed: Stereo Inverse Reinforcement Learning with Concurrent Policy Optimization

arXiv 2009.09577 ↗First posted 21 Sep 2020cs.LG cs.RO stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

In this paper, we study the problem of obtaining a control policy that can mimic and then outperform expert demonstrations in Markov decision processes where the reward function is unknown to the learning agent. One main relevant approach is the inverse reinforcement learning (IRL), which mainly focuses on inferring a reward function from expert demonstrations. The obtained control policy by IRL and the associated algorithms, however, can hardly outperform expert demonstrations. To overcome this limitation, we propose a novel method that enables the learning agent to outperform the demonstrator via a new concurrent reward and action policy learning approach. In particular, we first propose a new stereo utility definition that aims to address the bias in the interpretation of expert demonstrations. We then propose a loss function for the learning agent to learn reward and action policies concurrently such that the learning agent can outperform expert demonstrations. The performance of the proposed method is first demonstrated in OpenAI environments. Further efforts are conducted to experimentally validate the proposed method via an indoor drone flight scenario.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 28 references that are in the map. First posted on arXiv 21 Sep 2020.

Builds on

12 of 28 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 11

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2016OpenAI Gymcited by 1,684
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  7. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  8. 2017Reinforcement Learning with a Corrupted Reward Channelcited by 68
  9. 2019Hyperbolic Discounting and Learning over Multiple Horizonscited by 58
  10. 2019Deceptive Reinforcement Learning Under Adversarial Manipulations on Cost Signalscited by 50
  11. 2018Bipedal Walking Robot using Deep Deterministic Policy Gradientcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025PAC Apprenticeship Learning with Bayesian Active Inverse Reinforcement Learning
  2. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  3. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  4. 2025Symmetry-Guided Multi-Agent Inverse Reinforcement Learning
  5. 2025Inverse Reinforcement Learning via Convex Optimization
  6. 2019Multi-Agent Adversarial Inverse Reinforcement Learning
  7. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
  8. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids