mlsanityData to 14 Oct 2026

Inverse Reinforcement Learning from Non-Stationary Learning Agents

arXiv 2410.14135 ↗First posted 18 Oct 2024cs.LG cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

In this paper, we study an inverse reinforcement learning problem that involves learning the reward function of a learning agent using trajectory data collected while this agent is learning its optimal policy. To address this problem, we propose an inverse reinforcement learning method that allows us to estimate the policy parameters of the learning agent which can then be used to estimate its reward function. Our method relies on a new variant of the behavior cloning algorithm, which we call bundle behavior cloning, and uses a small number of trajectories generated by the learning agent's policy at different points in time to learn a set of policies that match the distribution of actions observed in the sampled trajectories. We then use the cloned policies to train a neural network model that estimates the reward function of the learning agent. We provide a theoretical analysis to show a complexity result on bound guarantees for our method that beats standard behavior cloning as well as numerical experiments for a reinforcement learning problem that validate the proposed method.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 8 of its 22 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

8 of 22 references.

Closest ancestors · 2

Same direction, newest first

  1. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms1 Dec
  2. 2020Inverse Reinforcement Learning from a Gradient-based Learner15 Jul

Other references · 6

  1. 2017Constrained Policy Optimizationcited by 983
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  3. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 567
  4. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  5. 2020FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPscited by 233
  6. 2018Machine Theory of Mindcited by 226

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Spatiotemporal Attention-Augmented Inverse Reinforcement Learning for Multi-Agent Task Allocationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  2. 2024Stable Inverse Reinforcement Learning: Policies from Control Lyapunov Landscapes
  3. 2019Integrating Behavior Cloning and Reinforcement Learning for Improved Performance in Dense and Sparse Reward Environments
  4. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  5. 2023Coherent Soft Imitation Learning
  6. 2021Inverse Reinforcement Learning: A Control Lyapunov Approach
  7. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  8. 2026Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards