mlsanityData to 14 Oct 2026

SPLASH! Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrations

arXiv 2507.08707 ↗First posted 11 Jul 2025cs.LG cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse Reinforcement Learning (IRL) presents a powerful paradigm for learning complex robotic tasks from human demonstrations. However, most approaches make the assumption that expert demonstrations are available, which is often not the case. Those that allow for suboptimality in the demonstrations are not designed for long-horizon goals or adversarial tasks. Many desirable robot capabilities fall into one or both of these categories, thus highlighting a critical shortcoming in the ability of IRL to produce field-ready robotic agents. We introduce Sample-efficient Preference-based inverse reinforcement learning for Long-horizon Adversarial tasks from Suboptimal Hierarchical demonstrations (SPLASH), which advances the state-of-the-art in learning from suboptimal demonstrations to long-horizon and adversarial settings. We empirically validate SPLASH on a maritime capture-the-flag task in simulation, and demonstrate real-world applicability with sim-to-real translation experiments on autonomous unmanned surface vehicles. We show that our proposed methods allow SPLASH to significantly outperform the state-of-the-art in reward learning from suboptimal demonstrations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 36 references that are in the map. First posted on arXiv 11 Jul 2025.

Builds on

19 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr
  2. 2018Inverse reinforcement learning for video games24 Oct
  3. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 16

  1. 2017Deep reinforcement learning from human preferencescited by 3,912
  2. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,745
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  7. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  8. 2018Behavioral Cloning from Observationcited by 492
  9. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  10. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  11. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  12. 2020Agent57: Outperforming the Atari Human Benchmarkcited by 303
4 more
  1. 2020Learning from Suboptimal Demonstration via Self-Supervised Reward Regressioncited by 84
  2. 2019Learning Reward Functions by Integrating Human Demonstrations and Preferencescited by 64
  3. 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
  4. 2022Human-level Atari 200x fastercited by 28

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control
  2. 2020oIRL: Robust Adversarial Inverse Reinforcement Learning with Temporally Extended Actions
  3. 2022Autonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learning
  4. 2021Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
  5. 2022Fast Lifelong Adaptive Inverse Reinforcement Learning from Demonstrations
  6. 2024Toward Information Theoretic Active Inverse Reinforcement Learning
  7. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matching
  8. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learning