mlsanityData to 7 Oct 2026

Finite-Sample Bounds for Adaptive Inverse Reinforcement Learning using Passive Langevin Dynamics

arXiv 2304.09123 ↗First posted 18 Apr 2023cs.LG stat.MLcited by 2

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

This paper provides a finite-sample analysis of a passive stochastic gradient Langevin dynamics (PSGLD) algorithm. This algorithm is designed to achieve adaptive inverse reinforcement learning (IRL). Adaptive IRL aims to estimate the cost function of a forward learner performing a stochastic gradient algorithm (e.g., policy gradient reinforcement learning) by observing their estimates in real-time. The PSGLD algorithm is considered passive because it incorporates noisy gradients provided by an external stochastic gradient algorithm (forward learner), of which it has no control. The PSGLD algorithm acts as a randomized sampler to achieve adaptive IRL by reconstructing the forward learner's cost function nonparametrically from the stationary measure of a Langevin diffusion. This paper analyzes the non-asymptotic (finite-sample) performance; we provide explicit bounds on the 2-Wasserstein distance between PSGLD algorithm sample measure and the stationary measure encoding the cost function, and provide guarantees for a kernel density estimation scheme which reconstructs the cost function from empirical samples. Our analysis uses tools from the study of Markov diffusion operators. The derived bounds have both practical and theoretical significance. They provide finite-time guarantees for an adaptive IRL mechanism, and substantially generalize the analytical framework of a line of research in passive stochastic gradient algorithms.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 6 of its 26 references that are in the map. First posted on arXiv 18 Apr 2023.

Builds on

6 of 26 references.

Closest ancestors · 1

Same direction, newest first

  1. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr

Other references · 5

  1. 2016Cooperative Inverse Reinforcement Learningcited by 298
  2. 2017Non-convex learning via Stochastic Gradient Langevin Dynamics: a nonasymptotic analysiscited by 242
  3. 2017Underdamped Langevin MCMC: A non-asymptotic analysiscited by 131
  4. 2017User-friendly guarantees for the Langevin Monte Carlo with inaccurate gradientcited by 116
  5. 2020Multi-kernel Passive Stochastic Gradient Algorithms and Transfer Learningcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026GaiaFlow: Semantic-Guided Diffusion Tuning for Carbon-Frugal Searchcited by 0
  2. 2026Malliavin Calculus for Counterfactual Gradient Estimation in Adaptive Inverse Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020Langevin Dynamics for Adaptive Inverse Reinforcement Learning of Stochastic Gradient Algorithms
  2. 2026How Does the Lagrangian Guide Safe Reinforcement Learning through Diffusion Models?
  3. 2026Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning
  4. 2026Reverse Flow Matching: A Unified Framework for Online Reinforcement Learning with Diffusion and Flow Policies
  5. 2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training
  6. 2026Fast Rates for Inverse Reinforcement Learning
  7. 2023LS-IQ: Implicit Reward Regularization for Inverse Reinforcement Learning
  8. 2025Inverse Reinforcement Learning using Revealed Preferences and Passive Stochastic Optimization