mlsanityData to 7 Oct 2026

X-MEN: Guaranteed XOR-Maximum Entropy Constrained Inverse Reinforcement Learning

arXiv 2203.11842 ↗First posted 22 Mar 2022cs.LGcited by 4

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse Reinforcement Learning (IRL) is a powerful way of learning from demonstrations. In this paper, we address IRL problems with the availability of prior knowledge that optimal policies will never violate certain constraints. Conventional approaches ignoring these constraints need many demonstrations to converge. We propose XOR-Maximum Entropy Constrained Inverse Reinforcement Learning (X-MEN), which is guaranteed to converge to the optimal policy in linear rate w.r.t. the number of learning iterations. X-MEN embeds XOR-sampling -- a provable sampling approach that transforms the #P complete sampling problem into queries to NP oracles -- into the framework of maximum entropy IRL. X-MEN also guarantees the learned policy will never generate trajectories that violate constraints. Empirical results in navigation demonstrate that X-MEN converges faster to the optimal policies compared to baseline approaches and always generates trajectories that satisfy multi-state combinatorial constraints.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 36 references that are in the map. First posted on arXiv 22 Mar 2022.

Builds on

16 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms1 Dec
  2. 2020Deep Inverse Q-learning with Constraints4 Aug
  3. 2019Maximum Likelihood Constraint Inference for Inverse Reinforcement Learning12 Sep
  4. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 12

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  3. 2017Deep Reinforcement Learning: An Overviewcited by 327
  4. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  5. 2020Inverse Constrained Reinforcement Learningcited by 45
  6. 2014Distribution-Aware Sampling and Weighted Model Counting for SATcited by 15
  7. 2021Maximum Likelihood Constraint Inference from Stochastic Demonstrationscited by 15
  8. 2013Taming the Curse of Dimensionality: Discrete Integration by Hashing and Optimizationcited by 12
  9. 2018Learning Constraints from Demonstrationscited by 9
  10. 2018Inferring geometric constraints in human demonstrationscited by 5
  11. 2018Modeling Supervisor Safe Sets for Improving Collaboration in Human-Robot Teamscited by 2
  12. 2019Towards Efficient Discrete Integration via Adaptive Quantile Queriescited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 2; 2024: 1.

4 papers.

  1. 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learningcited by 16
  2. 2022Learning Soft Constraints From Constrained Expert Demonstrationscited by 13
  3. 2024A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challengescited by 10
  4. 2023Inverse Reinforcement Learning With Constraint Recoverycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
  2. 2026Fast Rates for Inverse Reinforcement Learning
  3. 2024Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
  4. 2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learning
  5. 2022Weighted Maximum Entropy Inverse Reinforcement Learning
  6. 2018A Framework and Method for Online Inverse Reinforcement Learning
  7. 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
  8. 2021A Generalised Inverse Reinforcement Learning Framework