mlsanityData to 7 Oct 2026

Diverse Policies Recovering via Pointwise Mutual Information Weighted Imitation Learning

arXiv 2410.15910 ↗First posted 21 Oct 2024cs.LG cs.AI stat.MLcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Recovering a spectrum of diverse policies from a set of expert trajectories is an important research topic in imitation learning. After determining a latent style for a trajectory, previous diverse policies recovering methods usually employ a vanilla behavioral cloning learning objective conditioned on the latent style, treating each state-action pair in the trajectory with equal importance. Based on an observation that in many scenarios, behavioral styles are often highly relevant with only a subset of state-action pairs, this paper presents a new principled method in diverse polices recovery. In particular, after inferring or assigning a latent style for a trajectory, we enhance the vanilla behavioral cloning by incorporating a weighting mechanism based on pointwise mutual information. This additional weighting reflects the significance of each state-action pair's contribution to learning the style, thus allowing our method to focus on state-action pairs most representative of that style. We provide theoretical justifications for our new objective, and extensive empirical evaluations confirm the effectiveness of our method in recovering diverse policies from expert data.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 41 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

23 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018An Algorithmic Perspective on Imitation Learning16 Nov

Other references · 22

  1. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  2. 2016Generative Adversarial Imitation Learningcited by 1,576
  3. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  7. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  8. 2018MINE: Mutual Information Neural Estimationcited by 680
  9. 2019Behavior Regularized Offline Reinforcement Learningcited by 654
  10. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  11. 2017End-to-end Driving via Conditional Imitation Learningcited by 479
  12. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
10 more
  1. 2022Behavior Transformers: Cloning k modes with one stonecited by 299
  2. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 294
  3. 2016Data Programming: Creating Large Training Sets, Quicklycited by 246
  4. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skillscited by 124
  5. 2018Variational Option Discovery Algorithmscited by 121
  6. 2020Explore, Discover and Learn: Unsupervised Discovery of State-Covering Skillscited by 120
  7. 2017Robust Imitation of Diverse Behaviorscited by 85
  8. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  9. 2018AGIL: Learning Attention from Human for Visuomotor Taskscited by 32
  10. 2019Learning Calibratable Policies using Programmatic Style-Consistencycited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2025Diversity-Aware Policy Optimization for Large Language Model Reasoningcited by 39
  2. 2025Offline Learning of Controllable Diverse Behaviorscited by 2
  3. 2026Offline Reinforcement Learning of High-Quality Behaviors Under Robust Style Alignmentcited by 0
  4. 2026Explicit Trajectory Diversity for RL-Based Post-Training of LLM Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024How to Leverage Diverse Demonstrations in Offline Imitation Learning
  2. 2023A Simple Solution for Offline Imitation from Observations and Examples with Possibly Incomplete Trajectories
  3. 2026On the Sample Efficiency of Inverse Dynamics Models for Semi-Supervised Imitation Learning
  4. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  5. 2024Towards Diverse Behaviors: A Benchmark for Imitation Learning with Human Demonstrations
  6. 2022Multi-Task Imitation Learning for Linear Dynamical Systems
  7. 2026Difference-Aware Retrieval Policies for Imitation Learning
  8. 2024Contractive Dynamical Imitation Policies for Efficient Out-of-Sample Recovery