mlsanityData to 14 Oct 2026

Reward function shape exploration in adversarial imitation learning: an empirical study

arXiv 2104.06687 ↗First posted 14 Apr 2021cs.LG cs.AIcited by 2

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

For adversarial imitation learning algorithms (AILs), no true rewards are obtained from the environment for learning the strategy. However, the pseudo rewards based on the output of the discriminator are still required. Given the implicit reward bias problem in AILs, we design several representative reward function shapes and compare their performances by large-scale experiments. To ensure our results' reliability, we conduct the experiments on a series of Mujoco and Box2D continuous control tasks based on four different AILs. Besides, we also compare the performance of various reward function shapes using varying numbers of expert trajectories. The empirical results reveal that the positive logarithmic reward function works well in typical continuous control tasks. In contrast, the so-called unbiased reward function is limited to specific kinds of tasks. Furthermore, several designed reward functions perform excellently in these environments as well.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 12 of its 28 references that are in the map. First posted on arXiv 14 Apr 2021.

Builds on

12 of 28 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020Addressing reward bias in Adversarial Imitation Learning with neutral reward functions20 Sep
  2. 2020Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration5 Jun
  3. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 9

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  4. 2017Improved Training of Wasserstein GANscited by 2,235
  5. 2016Generative Adversarial Imitation Learningcited by 1,597
  6. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  7. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 316
  8. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  9. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 0; 2025: 1.

2 papers.

  1. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspectivecited by 4
  2. 2025On Generalization and Distributional Update for Mimicking Observations with Adequate Explorationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025On Discovering Algorithms for Adversarial Imitation Learning
  2. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  3. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  4. 2022ARC - Actor Residual Critic for Adversarial Imitation Learning
  5. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  6. 2024Provably and Practically Efficient Adversarial Imitation Learning with General Function Approximation
  7. 2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemble
  8. 2023PAGAR: Taming Reward Misalignment in Inverse Reinforcement Learning-Based Imitation Learning with Protagonist Antagonist Guided Adversarial Reward