mlsanityData to 7 Oct 2026

Langevin Dynamics for Adaptive Inverse Reinforcement Learning of Stochastic Gradient Algorithms

arXiv 2006.11674 ↗First posted 20 Jun 2020cs.LG cs.SY eess.SYcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) aims to estimate the reward function of optimizing agents by observing their response (estimates or actions). This paper considers IRL when noisy estimates of the gradient of a reward function generated by multiple stochastic gradient agents are observed. We present a generalized Langevin dynamics algorithm to estimate the reward function R(θ); specifically, the resulting Langevin algorithm asymptotically generates samples from the distribution proportional to exp(R(θ)). The proposed IRL algorithms use kernel-based passive learning schemes. We also construct multi-kernel passive Langevin algorithms for IRL which are suitable for high dimensional data. The performance of the proposed IRL algorithms are illustrated on examples in adaptive Bayesian learning, logistic regression (high dimensional problem) and constrained Markov decision processes. We prove weak convergence of the proposed IRL algorithms using martingale averaging methods. We also analyze the tracking performance of the IRL algorithms in non-stationary environments where the utility function R(θ) jump changes over time as a slow Markov chain.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 3 of its 41 references that are in the map. First posted on arXiv 20 Jun 2020.

Builds on

3 of 41 references.

References · 3

  1. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  2. 2014Consistency and fluctuations for stochastic gradient Langevin dynamicscited by 65
  3. 2016Partially Observed Markov Decision Processes. Problem Sets and Internet Supplementcited by 34

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Finite-Sample Bounds for Adaptive Inverse Reinforcement Learning using Passive Langevin Dynamics
  2. 2026Malliavin Calculus for Counterfactual Gradient Estimation in Adaptive Inverse Reinforcement Learning
  3. 2025Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization
  4. 2023FP-IRL: Fokker--Planck Inverse Reinforcement Learning -- A Physics-Constrained Approach to Markov Decision Processes
  5. 2019Stochastic Inverse Reinforcement Learning
  6. 2024Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
  7. 2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization
  8. 2025Understanding Inverse Reinforcement Learning under Overparameterization: Non-Asymptotic Analysis and Global Optimality